[c-nsp] troubleshooting SVI input drops on MSFC3
Sukumar Subburayan
sukumars at cisco.com
Wed May 9 19:12:19 EDT 2007
The decision to redirect the packet for ICMP redirect is done using a bit
in the PFC HW and it is a per-vlan bit, and not per flow. This bit can
either be on or off and is controlled by having '<no> ip redirects'. That
is why you are seeing all traffic in the vlan getting punted to software,
if we don't have 'no ip redirects' configured.
sukumar
On Wed, 9 May 2007, Church, Charles wrote:
> I don't think the PFC has any logic in it to provide ICMP responses for
> any of the message types. Someone from Cisco would know for sure
> though. Without the ability for the PFC to respond, those packets would
> get forwarded to the MSFC. It seems that it should only do it once per
> flow, but it sounds like it was doing it for all the traffic, since your
> CPU went down so dramatically.
>
>
> Chuck Church
> Multimax Principal Network Engineer, CCIE #8776
> EDS Contractor, Multimax - Navy Marine Corps Intranet (NMCI)
> 1210 N. Parker Rd. | Greenville, SC 29609
> Office: 864-335-9473 | Cell: 864-266-3978
> cchurch at multimax.com
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of barney gumbo
> Sent: Wednesday, May 09, 2007 5:39 PM
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] troubleshooting SVI input drops on MSFC3
>
> I disabled IP redirects on the SVI and traffic is no longer seen on the
> SVI interface, no longer seeing SVI input drops, and CPU returned to
> normal.
>
> Why does the router sending an IP redirect cause traffic to hit the
> MSFC?
>
> Or, does the traffic hit the MSFC because the CEF adjacency is the same
> interface for all prefixes, i.e. router-on-a-stick?
>
> I've read references to both as being the reason one would see high CPU
> on an MSFC along with SVI input drops, just trying to understand the
> hardware better. It seems the magic number is around 40-50 Mbps which
> resulted in 75% CPU usage and input drops. That just seems low for such
> powerful hardware...
>
>
> On 5/9/07, barney gumbo <barney.gumbo at gmail.com> wrote:
>>
>> I see ICMP redirects (which is not disabled on that SVI) and they are
>> incrementing. I can understand high CPU as a result of the router
>> being forced to trx alot of ICMP redirects, however that (the MSFC
>> sending ICMP
>> redirects) doesnt cause the traffic to be process-switched, does it?
>>
>> ICMP statistics:
>> Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 318
> unreachable
>> 10817442 echo, 3142088 echo reply, 0 mask requests, 0 mask
>> replies, 0 quench
>> 0 parameter, 0 timestamp, 0 info request, 0 other
>> 0 irdp solicitations, 0 irdp advertisements
>> Sent: 115887 redirects, 2262 unreachable, 3144305 echo, 10817442
>> echo reply
>> 0 mask requests, 0 mask replies, 0 quench, 0 timestamp
>> 0 info reply, 1740292 time exceeded, 0 parameter problem
>> 0 irdp solicitations, 0 irdp advertisements
>>
>>
>> ICMP statistics:
>> Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 318
> unreachable
>> 10820192 echo, 3142088 echo reply, 0 mask requests, 0 mask
>> replies, 0 quench
>> 0 parameter, 0 timestamp, 0 info request, 0 other
>> 0 irdp solicitations, 0 irdp advertisements
>> Sent: 115939 redirects, 2262 unreachable, 3144305 echo, 10820192
>> echo reply
>> 0 mask requests, 0 mask replies, 0 quench, 0 timestamp
>> 0 info reply, 1740292 time exceeded, 0 parameter problem
>> 0 irdp solicitations, 0 irdp advertisements
>>
>>
>> On 5/9/07, Church, Charles <cchurch at multimax.com> wrote:
>>>
>>> Sending ICMP redirects for all those?
>>>
>>> Chuck
>>>
>>> --- Original Message ---
>>> From:"barney gumbo" <barney.gumbo at gmail.com> Sent:Wed 5/9/07 12:15
>>> pm To:"Dale W. Carder" <dwcarder at doit.wisc.edu>
>>> Cc:"cisco-nsp at puck.nether.net" < cisco-nsp at puck.nether.net>
>>> Subj:Re: [c-nsp] troubleshooting SVI input drops on MSFC3
>>>
>>> I have a little more info this time..
>>>
>>> It appears that the traffic is being process switched.
>>>
>>> It started when this switch became more of a "router on a stick".
>>> Previously most traffic flowed from one SVI to the other; this 6503
>>> is essentially an access or WAN router. Now traffic flows from
>>> routers on the same VLAN into the SVI and then back out of the SVI
>>> to get to the next hop, which is a firewall, all on the same VLAN.
>>>
>>> It seems like once the flow volume hits around 40 Mbps, the input
>>> drops begin.
>>>
>>> Going away from the router-on-a-stick design, where the bulk of the
>>> traffic transit's across the switch instead of in/out the same
>>> interface, is not a trivial change, so I would like to try and get
>>> some confidence that this is related to the problem before I start
>>> to make changes. This is all just a guess, but it's the only major
>>> thing that has changed in the last week.
>>>
>>> Any ideas if this could be my cause of process-switchng and input
> drops?
>>>
>>> Any ideas on how I can verify the router-on-a-stick forwarding is
>>> definitely to blame?
>>>
>>> On 5/9/07, Dale W. Carder <dwcarder at doit.wisc.edu> wrote:
>>>>
>>>>
>>>> Here's some commands to get you started:
>>>>
>>>> sh buffers input-interface
>>>> sh int vlan1234 switching
>>>> sh ip interface
>>>> sh ip traffic
>>>> sh cef drop
>>>> sh ip cache flow
>>>> sh cef not-cef-switched
>>>>
>>>> Some more help can be found here:
>>>> http://www.cisco.com/warp/public/63/queue_drops.html
>>>>
>>>> You also might want to verify that you didn't configure a feature
>>>> that causes punts.
>>>>
>>>> If you really want to get dirty, you can create a span session to
>>>> monitor traffic destined to the RP. This has been discussed on
>>>> this list once or twice, but it is a bit messy.
>>>>
>>>> Dale
>>>>
>>>>
>>>> On May 9, 2007, at 9:43 AM, barney gumbo wrote:
>>>>
>>>>> I am seeing high input interface drops on an SVI interface on an
>
>>>>> MSFC3. The
>>>>> MSFC3 is installed in a 6503 chassis with Sup720. The switch is
>
>>>>> running hybird mode.
>>>>>
>>>>> The traffic load has increased, and CPU is running high when the
>
>>>>> traffic load increases. I don't know why the SVI is showing
>>>>> increased traffic load because normally I don't see traffic
>>>>> through the SVI, it all get's
>>> MLS
>>>>> switched. Something in the last week has caused traffic to be
>>>>> switched through the SVI showing the high input drops. The
>>>>> overal load of traffic which should be routed (MLS switched) via
>
>>>>> the interface has not increased or decreased; all of a sudden in
>
>>>>> the last week traffic is being
>>>>> (seemingly)
>>>>> process switched through this SVI.
>>>>>
>>>>> Where do I begin troubleshooting high interface drops on an SVI?
>>>>
>>> _______________________________________________
>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
>>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
More information about the cisco-nsp
mailing list