[c-nsp] 3750 High Cpu IP Input
Richard Gallagher
rgallagh at cisco.com
Fri Apr 24 09:37:30 EDT 2009
Input queue was full of packets like this:
Buffer information for RxQ3 buffer at 0x2E792F0
data_area 0x7BB2AB0, refcount 1, next 0x2E7E210, flags 0x200
linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1
if_input 0x3ABBAE0 (Vlan217), if_output 0x0 (None)
inputtime 00:00:00.000 (elapsed never)
outputtime 00:00:00.000 (elapsed never), oqnumber 65535
datagramstart 0x7BB2AF6, datagramsize 82, maximum size 2196
mac_start 0x7BB2AF6, addr_start 0x7BB2AF6, info_start 0x0
network_start 0x7BB2B04, transport_start 0x7BB2B18, caller_pc
0x6D1024
source: 74.212.165.187, destination: 224.0.0.252, id: 0x3CDA, ttl: 1,
TOS: 0 prot: 17, source port 58064, destination port 5355
Buffer information for RxQFB buffer at 0x2672BB0
data_area 0x758C35C, refcount 1, next 0x263960C, flags 0x200
linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1
if_input 0x3ABBAE0 (Vlan217), if_output 0x0 (None)
inputtime 00:00:00.000 (elapsed never)
outputtime 00:00:00.000 (elapsed never), oqnumber 65535
datagramstart 0x758C3A2, datagramsize 64, maximum size 2196
mac_start 0x758C3A2, addr_start 0x758C3A2, info_start 0x0
network_start 0x758C3B0, transport_start 0x0, caller_pc 0x6D1024
source: 74.212.165.187, destination: 224.0.0.252, id: 0x3CDA, ttl: 1,
TOS: 0 prot: 17, source port 58064, destination port 5355
These TTL=1 are causing the high CPU.
On 24 Apr 2009, at 14:26, Chris Lane wrote:
> Richard Gallagher found that it was one of my customers sending mcast
> packets with a TTL 1. Tried adding ACL's to lower CPU but this
> didn't fix.
> We shutdown Vlan to verify and CPU came down 40% to adequate levels.
>
> I have a call into out customer notifying them to fix.
>
> Thanks to all for your input
>
> Regards
> Chris
>
> 2009/4/24 Chris Lane <clane1875 at gmail.com>
>
>> Yes with a high preference.
>>
>> 2009/4/24 junior <drrtuy at ya.ru>
>>
>> Hello.
>>>
>>> Does this switch have default route?
>>>
>>> Chris Lane wrote:
>>>
>>>> sh ip traffic IP statistics:
>>>> Rcvd: 37788273 total, 24253 local destination
>>>> 0 format errors, 0 checksum errors, 9771492 bad hop count
>>>> 0 unknown protocol, 27979860 not a gateway
>>>> 0 security failures, 0 bad options, 7762670 with options
>>>> Opts: 0 end, 0 nop, 0 basic security, 0 loose source route
>>>> 0 timestamp, 0 extended security, 0 record route
>>>> 0 stream ID, 0 strict source route, 7762670 alert, 0
>>>> cipso, 0 ump
>>>> 0 other
>>>> Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
>>>> 0 fragmented, 0 couldn't fragment
>>>> Bcast: 2884 received, 87 sent
>>>> Mcast: 2334 received, 2209 sent
>>>> Sent: 24621 generated, 8328118 forwarded
>>>> Drop: 4258 encapsulation failed, 0 unresolved, 83 no adjacency
>>>> 69 no route, 0 unicast RPF, 0 forced drop
>>>> 0 options denied, 0 source IP address zero
>>>>
>>>> ICMP statistics:
>>>> Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 0
>>>> unreachable
>>>> 9560 echo, 0 echo reply, 0 mask requests, 0 mask replies, 0
>>>> quench
>>>> 0 parameter, 0 timestamp, 0 info request, 0 other
>>>> 0 irdp solicitations, 0 irdp advertisements
>>>> Sent: 0 redirects, 3129 unreachable, 0 echo, 9560 echo reply
>>>> 0 mask requests, 0 mask replies, 0 quench, 0 timestamp
>>>> 0 info reply, 47 time exceeded, 0 parameter problem
>>>> 0 irdp solicitations, 0 irdp advertisements
>>>>
>>>> TCP statistics:
>>>> Rcvd: 7710 total, 8 checksum errors, 1 no port
>>>> Sent: 6762 total
>>>>
>>>> UDP statistics:
>>>> Rcvd: 4615 total, 0 checksum errors, 1430 no port
>>>> Sent: 2909 total, 0 forwarded broadcasts
>>>>
>>>> IP-EIGRP statistics:
>>>> Rcvd: 0 total
>>>> Sent: 0 total
>>>>
>>>> BGP statistics:
>>>> Rcvd: 162 total, 1 opens, 0 notifications, 1 updates
>>>> 160 keepalives, 0 route-refresh, 0 unrecognized
>>>> Sent: 159 total, 1 opens, 0 notifications, 0 updates
>>>> 158 keepalives, 0 route-refresh
>>>>
>>>> PIMv2 statistics: Sent/Received
>>>> Total: 0/0, 0 checksum errors, 0 format errors
>>>> Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,
>>>> Hellos:
>>>> 0/0
>>>> Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
>>>> Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
>>>> State-Refresh: 0/0
>>>>
>>>> IGMP statistics: Sent/Received
>>>> Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
>>>> Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0 DVMRP:
>>>> 0/0, PIM:
>>>> 0/0
>>>>
>>>> OSPF statistics:
>>>> Rcvd: 2363 total, 0 checksum errors
>>>> 1900 hello, 12 database desc, 2 link state req
>>>> 345 link state updates, 104 link state acks
>>>>
>>>> Sent: 2231 total
>>>> 1904 hello, 11 database desc, 4 link state req
>>>> 223 link state updates, 89 link state acks
>>>>
>>>> ARP statistics:
>>>> Rcvd: 2254 requests, 82 replies, 0 reverse, 0 other
>>>> Sent: 4178 requests, 2447 replies (2 proxy), 0 reverse
>>>> Drop due to input queue full: 0
>>>>
>>>> Thanks for looking.
>>>>
>>>> On Fri, Apr 24, 2009 at 7:45 AM, junior <drrtuy at ya.ru <mailto:
>>>> drrtuy at ya.ru>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Did You check TAC cases?
>>>> Can You post this switch current configuration with sh ip traffic
>>>> command output?
>>>>
>>>> WBR
>>>> Roman A. Nozdrin
>>>>
>>>> Chris Lane wrote:
>>>>
>>>> 1 routed interface.sh platform ip unicast failed route
>>>> Total of 0 covering fib entries
>>>>
>>>> Thanks for reply.. I checked earlier regarding sdm.
>>>> Its the same on all of my 3750's i have about 20 of them
>>>> throughout the
>>>> states, this is probably the quietest one in regards to
>>>> bandwidth and
>>>> services.
>>>>
>>>>
>>>>
>>>> On Fri, Apr 24, 2009 at 7:21 AM, Brian Turnbow <b.turnbow at twt.it
>>>> <mailto:b.turnbow at twt.it>> wrote:
>>>>
>>>> how many routed interfaces do you have ( sh ip int brief
>>>> with ip
>>>> addresses ) ?
>>>> if more than 8 change the sdm template to routing
>>>>
>>>> you can use sh platform ip unicast failed route to see
>>>> if
>>>> routes are
>>>> failing to be programmed into tcam
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* Chris Lane [mailto:clane1875 at gmail.com
>>>> <mailto:clane1875 at gmail.com>]
>>>> *Sent:* venerdě 24 aprile 2009 11.17
>>>>
>>>> *To:* Brian Turnbow
>>>> *Cc:* Peter Rathlev; cisco-nsp at puck.nether.net
>>>> <mailto:cisco-nsp at puck.nether.net>
>>>>
>>>>
>>>> *Subject:* Re: [c-nsp] 3750 High Cpu IP Input
>>>>
>>>> sh controllers cpu-interface
>>>> ASIC Rxbiterr Rxunder Fwdctfix Txbuflos
>>>> Rxbufloc
>>>> Rxbufdrain
>>>>
>>>> -------------------------------------------------------------------------
>>>> ASIC0 0 0 0 0 0
>>>> 0
>>>> ASIC1 0 0 0 0 0
>>>> 0
>>>>
>>>>
>>>> cpu-queue-frames retrieved dropped invalid hol-
>>>> block
>>>> stray
>>>> ----------------- ---------- ---------- ----------
>>>> ---------- ----------
>>>> rpc 0 0 0 0
>>>> 0
>>>> stp 1807 0 0 0
>>>> 0
>>>> ipc 0 0 0 0
>>>> 0
>>>> routing protocol 1516326 0 0 0
>>>> 0
>>>> L2 protocol 27 0 0 0
>>>> 0
>>>> remote console 0 0 0 0
>>>> 0
>>>> sw forwarding 915 0 0 0
>>>> 0
>>>> host 2014 0 0 0
>>>> 0
>>>> broadcast 1766 0 0 0
>>>> 0
>>>> cbt-to-spt 0 0 0 0
>>>> 0
>>>> igmp snooping 1518651 0 0 0
>>>> 0
>>>> icmp 45 0 0 0
>>>> 0
>>>> logging 0 0 0 0
>>>> 0
>>>> rpf-fail 0 0 0 0
>>>> 0
>>>> queue14 0 0 0 0
>>>> 0
>>>> cpu heartbeat 14116 0 0 0
>>>> 0
>>>>
>>>> ODD i have disabled IGMP SNOOPING...
>>>>
>>>> On Fri, Apr 24, 2009 at 4:19 AM, Brian Turnbow
>>>> <b.turnbow at twt.it <mailto:b.turnbow at twt.it>> wrote:
>>>>
>>>> You can use show controller cpu to help see whats
>>>> going to the cpu
>>>> Make sure you have no ip redirects and no proxy arp
>>>> on
>>>> all the interfaces.
>>>> How many routed interfaces do you have ?
>>>> The output below for "max" is for 8 routed
>>>> interfaces if
>>>> you have more you
>>>> should change to the desktop switching template.
>>>> With your roughly your values for indirectly
>>>> connected
>>>> routes and 13 ip
>>>> interfaces on a box I needed to switch the template
>>>> "sdm
>>>> prefer routing"
>>>> requies reload.
>>>>
>>>> Regards
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: cisco-nsp-bounces at puck.nether.net
>>>> <mailto:cisco-nsp-bounces at puck.nether.net> [mailto:
>>>> cisco-nsp-bounces at puck.nether.net
>>>> <mailto:cisco-nsp-bounces at puck.nether.net>] On
>>>> Behalf Of
>>>> Chris Lane
>>>> Sent: venerdě 24 aprile 2009 1.09
>>>> To: Peter Rathlev
>>>> Cc: cisco-nsp at puck.nether.net
>>>> <mailto:cisco-nsp at puck.nether.net>
>>>> Subject: Re: [c-nsp] 3750 High Cpu IP Input
>>>>
>>>> sh platform tcam utilization
>>>>
>>>> CAM Utilization for ASIC# 0 Max
>>>> Used
>>>> Masks/
>>>> Values
>>>> Masks/values
>>>>
>>>> Unicast mac addresses:
>>>> 784/6272
>>>> 37/235
>>>> IPv4 IGMP groups + multicast routes:
>>>> 144/1152
>>>> 6/26
>>>> IPv4 unicast directly-connected routes:
>>>> 784/6272
>>>> 37/235
>>>> IPv4 unicast indirectly-connected routes:
>>>> 272/2176
>>>> 52/326
>>>> IPv4 policy based routing aces: 0/0
>>>> 0/0
>>>> IPv4 qos aces:
>>>> 528/528
>>>> 18/18
>>>> IPv4 security aces:
>>>> 1024/1024
>>>> 57/57
>>>>
>>>> Note: Allocation of TCAM entries per feature uses
>>>> a complex algorithm. The above information is meant
>>>> to provide an abstract view of the current TCAM
>>>> utilization
>>>>
>>>> Hope this helps.
>>>>
>>>> On Thu, Apr 23, 2009 at 4:41 PM, Peter Rathlev
>>>> <peter at rathlev.dk <mailto:peter at rathlev.dk>> wrote:
>>>>
>>>> On Thu, 2009-04-23 at 16:15 -0400, Chris Lane
>>>> wrote:
>>>>
>>>> This box has been in production for over a
>>>> year
>>>> and doesn't really do
>>>> to much as you can see from my orig thread it
>>>> moves about 11MB.
>>>>
>>>> This just started late last night yet we
>>>> didn't
>>>> add any new customer
>>>> nor did anybody even touch switch as the
>>>> device
>>>> is remote.
>>>>
>>>> I read in an older thread regarding same
>>>> thing
>>>> that the person
>>>> rebooted and of course it resolved issue. I
>>>> am
>>>> planning to do that
>>>> Early tomorrow am, but
>>>> i really want to know what the heck is
>>>> causing
>>>> this.
>>>>
>>>> Yes CEF is running.
>>>>
>>>> What about TCAM utilisation ("show platform tcam
>>>> utilization")?
>>>>
>>>> Regards,
>>>> Peter
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> //CL
>>>> _______________________________________________
>>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>>> <mailto:cisco-nsp at puck.nether.net>
>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>> archive at http://puck.nether.net/pipermail/cisco-
>>>> nsp/
>>>>
>>>>
>>>>
>>>> --
>>>> //CL
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> //CL
>>>>
>>>
>>>
>>
>>
>> --
>> //CL
>>
>
>
>
> --
> //CL
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list