[c-nsp] 3550 high cpu & process switched traffic
Church, Chuck
cchurch at multimax.com
Fri Aug 18 15:07:16 EDT 2006
The SDM indicates a maximum of 8K routes. But your route table shows
almost 11,000 subnets. That would explain a lot of CPU-forwarding,
wouldn't it?
Chuck Church
Network Engineer
CCIE #8776, MCNE, MCSE
Multimax, Inc.
Enterprise Network Engineering
Home Office - 864-335-9473
Cell - 864-266-3978
cchurch at multimax.com
-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Tassos
Chatzithomaoglou
Sent: Friday, August 18, 2006 2:57 PM
To: Arie Vayner (avayner)
Cc: cisco-nsp
Subject: Re: [c-nsp] 3550 high cpu & process switched traffic
Here they are:
3550#sh tcam inacl 1 stat
Ingress ACL TCAM#1: Number of active labels: 11
Ingress ACL TCAM#1: Number of masks allocated: 78, available: 130
Ingress ACL TCAM#1: Number of entries allocated: 189, available: 1475
3550#sh tcam outacl 1 stat
Egress ACL TCAM#1: Number of active labels: 4
Egress ACL TCAM#1: Number of masks allocated: 15, available: 193
Egress ACL TCAM#1: Number of entries allocated: 35, available: 1629
3550#sh tcam pbr 1 stat
PBR TCAM#1: Number of active labels: 0
PBR TCAM#1: Number of masks allocated: 0
PBR TCAM#1: Number of entries allocated: 0
3550#sh tcam qos 1 stat
QoS TCAM#1: Number of active labels: 5
QoS TCAM#1: Number of masks allocated: 5, available: 411
QoS TCAM#1: Number of entries allocated: 16, available: 3312
3550#sh sdm prefer
The current template is the default template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.
number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 1K
number of security aces: 1K
number of unicast routes: 8K
number of multicast routes: 1K
Arie Vayner (avayner) wrote on 18/8/2006 21:53:
> Sorry:
>
> sh tcam inacl 1 stat
> sh tcam outacl 1 stat
> sh tcam pbr 1 stat
> sh tcam qos 1 stat
>
> And
>
> show sdm prefer
>
> Arie
>
> -----Original Message-----
> From: Arie Vayner (avayner)
> Sent: Friday, August 18, 2006 21:51 PM
> To: 'Tassos Chatzithomaoglou'
> Cc: cisco-nsp
> Subject: RE: [c-nsp] 3550 high cpu & process switched traffic
>
> Tassos,
>
> Can you please send:
>
> sh tcam inacl 1 statistics
> show sdm prefer
>
> Thanks
> Arie
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Tassos
> Chatzithomaoglou
> Sent: Friday, August 18, 2006 21:43 PM
> To: Clinton Work
> Cc: cisco-nsp
> Subject: Re: [c-nsp] 3550 high cpu & process switched traffic
>
> Hi Clinton,
>
> Check my inline comments for the outputs you requested...
>
> Clinton Work wrote on 18/8/2006 21:23:
>> I have seen this issue many times with the 3550s. Your ACLs are
>> probably too large / complicated and they are over-running the 3550
>> hardware TCAM.
>>
>> Other possible causes:
>> - Too many routes
>
> 3550#sh ip cef sum
> IP CEF with switching (Table Version 2714880), flags=0x0
> 11194 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 3
> 11197 leaves, 683 nodes, 2233168 bytes, 2714439 inserts, 2703242
> invalidations
> 1 load sharing elements, 336 bytes, 1 references
> universal per-destination load sharing algorithm, id BAF66A0D
> 2(0) CEF resets, 446 revisions of existing leaves
> Resolution Timer: Exponential (currently 1s, peak 1s)
> 444 in-place/0 aborted modifications
> refcounts: 196815 leaf, 175104 node
>
> Table epoch: 0 (11197 entries at this epoch)
>
> Adjacency Table has 58 adjacencies
>
> 3550#sh ip route sum
> IP routing table name is Default-IP-Routing-Table(0)
> Route Source Networks Subnets Overhead Memory (bytes)
> connected 0 12 912 1920
> static 1 6 804 1120
> ospf xxx 192 10889 709248 1776020
> Intra-area: 0 Inter-area: 188 External-1: 23 External-2: 10870
> NSSA External-1: 0 NSSA External-2: 0
> internal 489 577020
> Total 682 10907 710964 2356080
>
> 3550#sh l3tcam shadow
> L3 TCAM: total 72 bit entries = 9216, used entries = 8774
> L3 TCAM: total 144 bit entries = 4608, used entries = 0
>
>> - Too many ARP entries. Note, ARP entry uses one IP CEF routing
entry.
>>
>
> "sh arp" displays around 75 entries.
>
>> Please see:
>>
>> Understand and Configure the Switching Database Manager on Catalyst
>> 3550 Series Switches:
>> http://www.cisco.com/warp/public/473/145.html
>>
>>
>> Check the output from these commands:
>> show controller cpu
>
> 3550#sh contr cpu
> stp packets : 17395785 retrieved, 0 dropped, 0 errors ram access
packets
> : 88713983 retrieved, 0 dropped, 0 errors routing protocol packets :
> 75142574 retrieved, 0 dropped, 0 errors forwarding packets : 0
> retrieved, 0 dropped, 0 errors routing packets : 3323805100 retrieved,
0
> dropped, 0 errors
> L2 protocol packets : 1491847 retrieved, 0 dropped, 0 errors igmp
> snooping protocol packets : 6113814 retrieved, 0 dropped, 0 errors
> queue7 : 0 retrieved, 0 dropped, 0 errors icmp redirect packets : 0
> retrieved, 0 dropped, 0 errors icmp unreachable packets : 0 retrieved,
0
> dropped, 0 errors logging packets : 0 retrieved, 0 dropped, 0 errors
> addr learning packets : 0 retrieved, 0 dropped, 0 errors rpffail
packets
> : 0 retrieved, 0 dropped, 0 errors
> queue13 : 50 retrieved, 0 dropped, 0 errors
> queue14 : 0 retrieved, 0 dropped, 0 errors
> queue15 : 0 retrieved, 0 dropped, 0 errors
>
>> show access-lists hardware counters
>>
>
> 3550#sh access-lists hardware counters
> Input Drops: 0 matches (0 bytes)
> Output Drops: 340637 matches (22547288 bytes)
> Input Forwarded: 78500081633 matches (38770143343719 bytes)
> Output Forwarded: 57823143638 matches (29808587886766 bytes)
> Input Bridge Only: 3310978 matches (239194398 bytes)
> Bridge and Route in CPU: 0 matches (0 bytes)
> Route in CPU: 13764993 matches (1259523078 bytes)
>
> after 2 mins:
>
> 3550#sh access-lists hardware counters
> Input Drops: 0 matches (0 bytes)
> Output Drops: 340639 matches (22547416 bytes)
> Input Forwarded: 78502363549 matches (38771255482430 bytes)
> Output Forwarded: 57824317525 matches (29809415168851 bytes)
> Input Bridge Only: 3311091 matches (239201856 bytes)
> Bridge and Route in CPU: 0 matches (0 bytes)
> Route in CPU: 13765101 matches (1259535021 bytes)
>
>
>> Try removing and re-applying your ACLs and see if the following
>> message is logged:
>> show logging | inc FM-3-UNLOADING
>>
>
> I tried it, but i didn't got any error message.
>
>> You can look at the "show tcam inacl | outacl" commands, but they are
>> useless for troubleshooting TCAM utilization issues. In my opinion,
>> the show tcam commands should show you total requested TCAM size in
>> addition to the TCAM space currently used. Reporting TCAM utilization
>> only for the ACLs that fit into the TCAM isn't very useful!
>>
>
>
> 3550#sh tcam inacl 1 stat
> Ingress ACL TCAM#1: Number of active labels: 11
> Ingress ACL TCAM#1: Number of masks allocated: 78, available: 130
> Ingress ACL TCAM#1: Number of entries allocated: 189, available: 1475
>
> 3550#sh tcam outacl 1 stat
> Egress ACL TCAM#1: Number of active labels: 4
> Egress ACL TCAM#1: Number of masks allocated: 15, available: 193
> Egress ACL TCAM#1: Number of entries allocated: 35, available: 1629
>
>> 3550#show tcam inacl 1 stat
>> Ingress ACL TCAM#1: Number of active labels: 5
>> Ingress ACL TCAM#1: Number of masks allocated: 46, available:
370
>> Ingress ACL TCAM#1: Number of entries allocated: 74, available:
3254
>>
>>
>>
>> Tassos Chatzithomaoglou wrote:
>>> 3550#sh proc cpu | exc 0.00
>>> CPU utilization for five seconds: 93%/91%; one minute: 93%; five
>>> minutes: 93%
>>> PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY
> Process
>>> 20 4544896 123670508 36 0.57% 0.63% 0.62% 0
Vegas
>
>>> LED Proces
>>> 27 30048240 14764148 2035 0.40% 0.48% 0.45% 0
Vegas
>
>>> Statistics
>>> 33 4701728 12413685 378 0.08% 0.06% 0.07% 0
> L3MD_STAT
>>> 35 919348 69239506 13 0.08% 0.07% 0.08% 0
> VegasPM
>>> 36 11606324 9735382 1192 0.08% 0.14% 0.15% 0
>>> VUR_MGR bg proce
>>> 39 30030680 110922911 270 0.40% 0.40% 0.40% 0 IP
> Input
>>> 90 813456 7361206 110 0.08% 0.04% 0.04% 0 CEF
>>> process
>>> 99 13504708 28875655 467 0.24% 0.10% 0.15% 0 OSPF
>>> Router
>>>
>>> Of course, there isn't any debug running in the background...
>>>
>>> omar parihuana wrote on 18/8/2006 20:42:
>>>> Hi,
>>>>
>>>> There is something wrong.. I have a 3550 with near 100SVI and my
CPU
>
>>>> process not reach 20%
>>>>
>>
>
_______________________________________________
cisco-nsp mailing list cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list