[c-nsp] 3550 high cpu & process switched traffic
Tassos Chatzithomaoglou
achatz at forthnet.gr
Fri Aug 18 14:53:18 EDT 2006
Hi Arie,
Below you can find the outputs you requested.
Arie Vayner (avayner) wrote on 18/8/2006 21:50:
> Tassos,
>
> Can you please send:
>
> sh tcam inacl 1 statistics
3550#sh tcam inacl 1 statistics
Ingress ACL TCAM#1: Number of active labels: 11
Ingress ACL TCAM#1: Number of masks allocated: 78, available: 130
Ingress ACL TCAM#1: Number of entries allocated: 189, available: 1475
> show sdm prefer
3550#sh sdm prefer
The current template is the default template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.
number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 1K
number of security aces: 1K
number of unicast routes: 8K
number of multicast routes: 1K
>
> Thanks
> Arie
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Tassos
> Chatzithomaoglou
> Sent: Friday, August 18, 2006 21:43 PM
> To: Clinton Work
> Cc: cisco-nsp
> Subject: Re: [c-nsp] 3550 high cpu & process switched traffic
>
> Hi Clinton,
>
> Check my inline comments for the outputs you requested...
>
> Clinton Work wrote on 18/8/2006 21:23:
>> I have seen this issue many times with the 3550s. Your ACLs are
>> probably too large / complicated and they are over-running the 3550
>> hardware TCAM.
>>
>> Other possible causes:
>> - Too many routes
>
> 3550#sh ip cef sum
> IP CEF with switching (Table Version 2714880), flags=0x0
> 11194 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 3
> 11197 leaves, 683 nodes, 2233168 bytes, 2714439 inserts, 2703242
> invalidations
> 1 load sharing elements, 336 bytes, 1 references
> universal per-destination load sharing algorithm, id BAF66A0D
> 2(0) CEF resets, 446 revisions of existing leaves
> Resolution Timer: Exponential (currently 1s, peak 1s)
> 444 in-place/0 aborted modifications
> refcounts: 196815 leaf, 175104 node
>
> Table epoch: 0 (11197 entries at this epoch)
>
> Adjacency Table has 58 adjacencies
>
> 3550#sh ip route sum
> IP routing table name is Default-IP-Routing-Table(0)
> Route Source Networks Subnets Overhead Memory (bytes)
> connected 0 12 912 1920
> static 1 6 804 1120
> ospf xxx 192 10889 709248 1776020
> Intra-area: 0 Inter-area: 188 External-1: 23 External-2: 10870
> NSSA External-1: 0 NSSA External-2: 0
> internal 489 577020
> Total 682 10907 710964 2356080
>
> 3550#sh l3tcam shadow
> L3 TCAM: total 72 bit entries = 9216, used entries = 8774
> L3 TCAM: total 144 bit entries = 4608, used entries = 0
>
>> - Too many ARP entries. Note, ARP entry uses one IP CEF routing entry.
>>
>
> "sh arp" displays around 75 entries.
>
>> Please see:
>>
>> Understand and Configure the Switching Database Manager on Catalyst
>> 3550 Series Switches:
>> http://www.cisco.com/warp/public/473/145.html
>>
>>
>> Check the output from these commands:
>> show controller cpu
>
> 3550#sh contr cpu
> stp packets : 17395785 retrieved, 0 dropped, 0 errors ram access packets
> : 88713983 retrieved, 0 dropped, 0 errors routing protocol packets :
> 75142574 retrieved, 0 dropped, 0 errors forwarding packets : 0
> retrieved, 0 dropped, 0 errors routing packets : 3323805100 retrieved, 0
> dropped, 0 errors
> L2 protocol packets : 1491847 retrieved, 0 dropped, 0 errors igmp
> snooping protocol packets : 6113814 retrieved, 0 dropped, 0 errors
> queue7 : 0 retrieved, 0 dropped, 0 errors icmp redirect packets : 0
> retrieved, 0 dropped, 0 errors icmp unreachable packets : 0 retrieved, 0
> dropped, 0 errors logging packets : 0 retrieved, 0 dropped, 0 errors
> addr learning packets : 0 retrieved, 0 dropped, 0 errors rpffail packets
> : 0 retrieved, 0 dropped, 0 errors
> queue13 : 50 retrieved, 0 dropped, 0 errors
> queue14 : 0 retrieved, 0 dropped, 0 errors
> queue15 : 0 retrieved, 0 dropped, 0 errors
>
>> show access-lists hardware counters
>>
>
> 3550#sh access-lists hardware counters
> Input Drops: 0 matches (0 bytes)
> Output Drops: 340637 matches (22547288 bytes)
> Input Forwarded: 78500081633 matches (38770143343719 bytes)
> Output Forwarded: 57823143638 matches (29808587886766 bytes)
> Input Bridge Only: 3310978 matches (239194398 bytes)
> Bridge and Route in CPU: 0 matches (0 bytes)
> Route in CPU: 13764993 matches (1259523078 bytes)
>
> after 2 mins:
>
> 3550#sh access-lists hardware counters
> Input Drops: 0 matches (0 bytes)
> Output Drops: 340639 matches (22547416 bytes)
> Input Forwarded: 78502363549 matches (38771255482430 bytes)
> Output Forwarded: 57824317525 matches (29809415168851 bytes)
> Input Bridge Only: 3311091 matches (239201856 bytes)
> Bridge and Route in CPU: 0 matches (0 bytes)
> Route in CPU: 13765101 matches (1259535021 bytes)
>
>
>> Try removing and re-applying your ACLs and see if the following
>> message is logged:
>> show logging | inc FM-3-UNLOADING
>>
>
> I tried it, but i didn't got any error message.
>
>> You can look at the "show tcam inacl | outacl" commands, but they are
>> useless for troubleshooting TCAM utilization issues. In my opinion,
>> the show tcam commands should show you total requested TCAM size in
>> addition to the TCAM space currently used. Reporting TCAM utilization
>> only for the ACLs that fit into the TCAM isn't very useful!
>>
>
>
> 3550#sh tcam inacl 1 stat
> Ingress ACL TCAM#1: Number of active labels: 11
> Ingress ACL TCAM#1: Number of masks allocated: 78, available: 130
> Ingress ACL TCAM#1: Number of entries allocated: 189, available: 1475
>
> 3550#sh tcam outacl 1 stat
> Egress ACL TCAM#1: Number of active labels: 4
> Egress ACL TCAM#1: Number of masks allocated: 15, available: 193
> Egress ACL TCAM#1: Number of entries allocated: 35, available: 1629
>
>> 3550#show tcam inacl 1 stat
>> Ingress ACL TCAM#1: Number of active labels: 5
>> Ingress ACL TCAM#1: Number of masks allocated: 46, available: 370
>> Ingress ACL TCAM#1: Number of entries allocated: 74, available: 3254
>>
>>
>>
>> Tassos Chatzithomaoglou wrote:
>>> 3550#sh proc cpu | exc 0.00
>>> CPU utilization for five seconds: 93%/91%; one minute: 93%; five
>>> minutes: 93%
>>> PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY
> Process
>>> 20 4544896 123670508 36 0.57% 0.63% 0.62% 0 Vegas
>
>>> LED Proces
>>> 27 30048240 14764148 2035 0.40% 0.48% 0.45% 0 Vegas
>
>>> Statistics
>>> 33 4701728 12413685 378 0.08% 0.06% 0.07% 0
> L3MD_STAT
>>> 35 919348 69239506 13 0.08% 0.07% 0.08% 0
> VegasPM
>>> 36 11606324 9735382 1192 0.08% 0.14% 0.15% 0
>>> VUR_MGR bg proce
>>> 39 30030680 110922911 270 0.40% 0.40% 0.40% 0 IP
> Input
>>> 90 813456 7361206 110 0.08% 0.04% 0.04% 0 CEF
>>> process
>>> 99 13504708 28875655 467 0.24% 0.10% 0.15% 0 OSPF
>>> Router
>>>
>>> Of course, there isn't any debug running in the background...
>>>
>>> omar parihuana wrote on 18/8/2006 20:42:
>>>> Hi,
>>>>
>>>> There is something wrong.. I have a 3550 with near 100SVI and my CPU
>
>>>> process not reach 20%
>>>>
>>
>
> --
> ***************************************
> Tassos Chatzithomaoglou
> Network Design & Development Department
> FORTHnet S.A.
> <achatz at forthnet.gr>
> ***************************************
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
--
***************************************
Tassos Chatzithomaoglou
Network Design & Development Department
FORTHnet S.A.
<achatz at forthnet.gr>
***************************************
More information about the cisco-nsp
mailing list