[c-nsp] maximum dCEF routes on GSR engine 2 cards
John van Oppen
john at vanoppen.com
Sun Jul 29 05:48:21 EDT 2007
Yep, that is what I thought. With that last command it looks like MPLS
could be the culprit, any advice to that end would be welcome. In any
case, here are the details (I never knew about that last command):
cr1 #show ip cef sum
IP Distributed CEF with switching (Table Version 411705), flags=0x0
222833 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 222497
410597 instant recursive resolutions, 0 used background process
0 load sharing elements, 0 references
0 in-place/0 aborted modifications
37906144 bytes allocated to the FIB table data structures
universal per-destination load sharing algorithm, id BA6A1595
2(0) CEF resets
Resolution Timer: Exponential (currently 1s, peak 1s)
Tree summary:
16-4-4-4-4 stride pattern
short mask protection disabled
222833 leaves, 58284 nodes using 11165764 bytes
Transient memory used: 199036, max: 24248008
Table epoch: 0 (222833 entries at this epoch)
Hardware resource allocation status summary
Green (Normal), Yellow (Caution) Red (Alarm)
Slot HW Resource Name Util Alert
2 E2_Rx_PLU 29 G
2 E2_Rx_TLU 99 R
3 E2_Rx_PLU 29 G
3 E2_Rx_TLU 100 R
Adjacency Table has 154 adjacencies
138 IPv4 adjacencies
16 IPv6 adjacencies
cr1 #show ip route sum
IP routing table name is Default-IP-Routing-Table(0)
Route Source Networks Subnets Overhead Memory (bytes)
connected 3 21 1680 3648
static 2 6 512 1216
ospf 1 0 1 64 152
Intra-area: 0 Inter-area: 0 External-1: 0 External-2: 1
NSSA External-1: 0 NSSA External-2: 0
bgp XXXXX 125146 97444 14245760 33844900
External: 222565 Internal: 25 Local: 0
internal 2134 2501048
Total 127285 97472 14248016 36350964
cr1-sea#
cr1#exec slot 3 show contr psa mem
========= Line Card (Slot 3) =========
Total Memory
------------
PLU SDRAM: 67108864 bytes
TLU SDRAM: 67108864 bytes
PSA SSRAM: 1048576 bytes
Free Memory (Free Memory excluding chunks)
-----------
PLU SDRAM: 47534592 bytes (47458304 bytes)
TLU SDRAM: 2742112 bytes ( 641152 bytes)
PSA SSRAM: 737152 bytes ( 726272 bytes)
Allocated Memory (Allocated Memory including chunks)
----------------
PLU SDRAM:
Mtrie 13225984 bytes (13238272 bytes)
Tag lookup table 4244480 bytes ( 4308480 bytes)
Frame switching table 0 bytes
ACL tables 0 bytes
Load balancing table 2097152 bytes
FRoM Imposition table 0 bytes
PHoM imposition table 0 bytes
TLU SDRAM:
Adjacency table 22208 bytes ( 161600 bytes)
LB Adjacency table 0 bytes ( 161280 bytes)
LB Adjacency count 0
Loadsharing information 0 bytes ( 172800 bytes)
Multicast hardware mstates 0 bytes
Mtrie prefix 3264 bytes ( 164544 bytes)
Mtrie prefix 128 ACL 13023936 bytes (13224960 bytes)
Mtrie prefix 448 ACL 0 bytes ( 0 bytes)
Tag rewrite structures 51150816 bytes (52577280 bytes)
Tag loadbalance structures 0 bytes ( 172800 bytes)
Frame switching 0 bytes ( 0 bytes)
Frame switching (policing) 0 bytes ( 0 bytes)
FRoM imposition 0 bytes ( 0 bytes)
PHoM imposition 0 bytes ( 0 bytes)
ACL tables 0 bytes
SSRAM:
Tag loadbalance structures 128 bytes ( 11008 bytes)
Loadbalance structures 21760 bytes ( 32640 bytes)
ACL usage 20480 bytes
Other 269056 bytes ( 279936 bytes)
VPN usage 128 bytes
BGP PA usage 65536 bytes
CoS Transparency usage 128 bytes
PIRC usage 512 bytes
SRC MAC Acct usage 6144 bytes
VLAN usage 196608 bytes
Largest Free Block
------------------
PLU SDRAM: 16251904 bytes
TLU SDRAM: 161280 bytes
PSA SSRAM: 726272 bytes
-----Original Message-----
From: Oliver Boehmer (oboehmer) [mailto:oboehmer at cisco.com]
Sent: Sunday, July 29, 2007 1:00 AM
To: John van Oppen; cisco-nsp at puck.nether.net
Subject: RE: [c-nsp] maximum dCEF routes on GSR engine 2 cards
John,
number of prefixes for an Engine2 linecard depends on the features used
(VRFs, ACLs) and the way CEF allocates their memory (for example number
of loadinfos when you do any form of multipath).
Can you show a "show ip route sum", "show ip cef summ" as well as an
"execute-on slot 3 show contr psa mem"?
You can also check the Release-notes of CSCdt34374 for some info.
Engine0 don't have any dedicated forwarding memory, so Engine0 prefix
capacity is limited by LC DRAM only.
With the growth of the Internet routing table, E2 fowarding capactiy can
reach its limits depending on ACLs configured on the box.
oli
John van Oppen <> wrote on Sunday, July 29, 2007 8:10 AM:
> I am running into a problem where I am running out of dCEF resources
> on a couple of engine 2 line cards (both of the suspect cards are
> three port gigE cards) in my 12008s. Anyone have hints on getting
> this usage under control or at least some actual numbers on what the
> actual maximum routing table size in the hardware forwarding table
> (since the E2_RX_TLU indicator seems to indicate forwarding entries).
>
> Interestingly this is not affecting the engine 0 line cards in the
> routers, and I was wondering if anyone else with a full view of the
> routing table was having this happen? It seems on my side to just
> have started one day and immediately have gone from a yellow alarm to
> 100% utilization, so it seems to me like something could be wonky.
>
>
> router>show ip cef re
>
> Hardware resource allocation status summary
>
> Green (Normal), Yellow (Caution) Red (Alarm)
>
> Slot HW Resource Name Util Alert
>
> 2 E2_Rx_PLU 29 G
>
> 2 E2_Rx_TLU 100 R
>
> 3 E2_Rx_PLU 29 G
>
> 3 E2_Rx_TLU 100 R
>
>
>
>
>
> Both cards are the following:
>
>
>
> router#show cef line 3
>
> CEF linecard slot 3, status up, fibhwidb sync, fibidb sync, adj sync
>
> Sequence number 20688, Maximum sequence number expected 21193, Seq
> Epoch 2
>
> Send failed 0, Out Of Sequence 0, drops 0
>
> Linecard CEF reset 2, reloaded 2
>
> 2091179 elements packed in 32971 messages(72011038 bytes) sent
>
> 32 elements cleared
>
> 0/0/0 xdr elements in LowQ/MediumQ/HighQ
>
> 247773/34/286 peak elements on LowQ/MediumQ/HighQ
>
> Keepalive timer will expire in 00:05:59
>
> Init window timer is not running
>
> Input packets 1904878439, bytes 1322133076937
>
> Output packets 1631996819, bytes 1062121911533, drops 0
>
> UTI input packets 0, bytes 0
>
> UTI output packets 0, bytes 0
>
> 0 proc sw, 0 proc sw fail, 0 hdr error
>
>
>
> CEF Table statistics:
>
> Table name Version Prefix-xdr Status
>
> Default 404607 702947 Active, sync, table-up
>
>
>
>
>
> router#show diag 3
>
>
>
> SLOT 3 (RP/LC 3 ): 3 Port Gigabit Ethernet
>
> MAIN: type 68, 800-6376-05 rev B0
>
> Deviation: 0
>
> HW config: 0x00 SW key: 00-00-00
>
> PCA: 73-4775-07 rev C0 ver 2
>
> Design Release 2.0 S/N SAD09100D9E
>
> MBUS: Embedded Agent
>
> Test hist: 0x00 RMA#: 00-00-00 RMA hist: 0x00
>
> DIAG: Test count: 0x00000000 Test results: 0x00000000
>
> FRU: Linecard/Module: 3GE-GBIC-SC=
>
> Processor Memory: MEM-GRP/LC-256=
>
> Packet Memory: MEM-LC1-PKT-256=
>
> L3 Engine: 2 - Backbone OC48 (2.5 Gbps)
>
> MBUS Agent Software version 1.100 (RAM) (ROM version is 2.41)
>
> ROM Monitor version 16.12
>
> Fabric Downloader version used n/a (ROM version is 10.1)
>
> Primary clock is CSC 1
>
> Board is analyzed
>
> Board State is Line Card Enabled (IOS RUN )
>
> Insertion time: 15w2d (1d09h ago)
>
> Processor Memory size: 268435456 bytes
>
> TX Packet Memory size: 134217728 bytes, Packet Memory pagesize: 8192
> bytes
>
> RX Packet Memory size: 134217728 bytes, Packet Memory pagesize: 8192
> bytes
>
> 0 crashes since restart
>
>
>
>
>
> Any ideas on optimization would be appreciated.
>
>
>
> John
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list