[c-nsp] Slow Inter-Vlan traffic

Sat Apr 20 02:59:58 EDT 2013

Hi,

Thanks Pete for the tips.
Packets were actually punted to the CPU.

The problem is linked to a Bug in our IOS version (triggered by a "no
switchport" on an interface a few weeks ago). Consequence is a failure in
TCAM programming when we add new SVIs.

First of all we will reload the switches then an upgrade should resolve it
all.

2013/4/18 Pete Lumbis <alumbis at gmail.com>

> Sounds like you have process switched traffic. You can watch the traffic
> punted to the CPU with a NetDR capture "debug netdr capture...". This will
> capture up to 1024 packets to memory. These are packets that are already
> punted to the cpu so there is no additional impact from this command
> (unlike other debugs).
> https://supportforums.cisco.com/docs/DOC-15608
>
> Can you show the interface configurations? My guess is there is something
> that the hardware can't support like an ACL with a log keyword, or
> misconfigured PBR.
>
>
> On Wed, Apr 17, 2013 at 1:21 PM, Sébastien MALHEIRO <s.malher at gmail.com
> >wrote:
>
> > HHi,
> >
> >
> >
> > We’re having some issues with inter-vlan traffic going pretty slow on
> > 6504-E chassis working in VSS (VS-S720-10G / WS-X6748-GE-TX), running IOS
> > version 12.2(33)SXH5.
> >
> >
> >
> > Here’s the thing :
> >
> > We have added 2 new Vlans on the swtichs (Vlan 41 : Production / Vlan 95
> :
> > Admin).
> >
> > Our customer told us that things are going pretty slow on both vlans (we
> > have seen 1.5Mbits/s of maximum bandwidth on Vlan 41).
> >
> >
> >
> > Looking forward we saw that there is no distributed cache traffic on the
> > SVIs :
> >
> >
> >
> > #sh interfaces Vlan41 stats
> >
> > Vlan41
> >
> >           Switching path    Pkts In   Chars In   Pkts Out  Chars Out
> >
> >                Processor      64954    4280562     155332    9976305
> >
> >              Route cache    6116040  589593156    8427463 10028760624
> >
> >        Distributed cache          0          0          0          0
> >
> >                    Total    6180994  593873718    8582795 10038736929
> >
> >
> >
> > #sh interfaces Vlan95 stats
> >
> > Vlan95
> >
> >           Switching path    Pkts In   Chars In   Pkts Out  Chars Out
> >
> >                Processor     732380   46887018    2009253  128644656
> >
> >              Route cache    8377033 4951406934    5756730  480387249
> >
> >        Distributed cache          0          0          0          0
> >
> >                    Total    9109413 4998293952    7765983  609031905
> >
> >
> >
> > On older (working well) SVIs added on the switchs we saw that packets are
> > going to distributed cache.
> >
> > I’m not a expert on this but as far as I understand I can see that our
> > linecards are working in acef mode and that cef is enabled on the 2 new
> > SVIs (the sh cef interface output is the same on all the SVIs configured
> on
> > the switchs) :
> >
> > #sh cef interface Vlan 41
> >
> > Vlan41 is up (if_number 460)
> >
> >   Corresponding hwidb fast_if_number 460
> >
> >   Corresponding hwidb firstsw->if_number 460
> >
> >   Internet address is X.X.X.X/X
> >
> >   ICMP redirects are always sent
> >
> >   IP unicast RPF check is disabled
> >
> >   Output features: HW Shortcut Installation
> >
> >   Inbound access list is not set
> >
> >   Outbound access list is not set
> >
> >   IP policy routing is disabled
> >
> >   BGP based policy accounting on input is disabled
> >
> >   BGP based policy accounting on output is disabled
> >
> >   Hardware idb is Vlan41
> >
> >   Fast switching type 22, interface type 147
> >
> >   IP CEF switching enabled
> >
> >   IP CEF switching turbo vector
> >
> >   IP Null turbo vector
> >
> >   IP prefix lookup IPv4 mtrie generic
> >
> >   Input fast flags 0x0, Output fast flags 0x0
> >
> >   ifindex 459(459)
> >
> >   Slot unknown (-1) Slot unit 41 VC -1
> >
> >   Transmit limit accumulator 0x0 (0x0)
> >
> >   IP MTU 1500
> >
> >
> >
> > #sh cef interface vlan 95
> >
> > Vlan95 is up (if_number 455)
> >
> >   Corresponding hwidb fast_if_number 455
> >
> >   Corresponding hwidb firstsw->if_number 455
> >
> >   Internet address is X.X.X.X/X
> >
> >   ICMP redirects are always sent
> >
> >   IP unicast RPF check is disabled
> >
> >   Output features: HW Shortcut Installation
> >
> >   Inbound access list is not set
> >
> >   Outbound access list is not set
> >
> >   IP policy routing is disabled
> >
> >   BGP based policy accounting on input is disabled
> >
> >   BGP based policy accounting on output is disabled
> >
> >   Hardware idb is Vlan95
> >
> >   Fast switching type 22, interface type 147
> >
> >   IP CEF switching enabled
> >
> >   IP CEF switching turbo vector
> >
> >   IP Null turbo vector
> >
> >   IP prefix lookup IPv4 mtrie generic
> >
> >   Input fast flags 0x0, Output fast flags 0x0
> >
> >   ifindex 454(454)
> >
> >   Slot unknown (-1) Slot unit 95 VC -1
> >
> >   Transmit limit accumulator 0x0 (0x0)
> >
> >   IP MTU 1500
> >
> >
> >
> > Moreover the “show ip cef vlan” output show that the good prefixes are
> > attached to the SVIs.
> >
> > We also saw in the “show interface vlan” output that there are no “L3
> > in/out Switched” packets going through.
> >
> >
> >
> > We would like to know where we could look to find the origin of this
> > problem.
> >
> > Any Idea would be appreciated.
> >
> >
> >
> >  Thanks.
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>