[c-nsp] portchannel & dcef?

[Jon Lewis]

On Wed, 1 Dec 2004, Jon Lewis wrote:

> On Wed, 1 Dec 2004, Rodney Dunn wrote:
>
> > It shouldn't be process switched but rather CEF switched on the RSP.
> > That's why the route-cache line increments there.  We don't support
> > ip accounting in the dCEF path (use netflow) so that's why the
> > traffic was getting punted to the RSP for switching.
>
> Well, there seems to be yet another counter issue, because the portchannel
> stats said the traffic was being route cache (cef on the RSP) switched,
> but the portchannel member FEs were saying processor switched.  Can an
> RSP4 processor switch 50mbit/s from FE's into a POS?

I've got more strangeness to report with rsp-k91pv-mz.122-18.S5.bin.
Yesterday, we apparently had a physical cabling failure with one of the
FE's in the previously mentioned portchannel.  This really made a mess of
things as the router declared the interface to be line proto down and
removed it from the portchannel, but the switch (3550) saw nothing wrong
and kept that FE in the portchannel.

>From what I've read, keepalives are supposed to be on by default.  If
that's true, shouldn't the switch have noticed that the router had downed
its end of one of the member interface and removed that interface on its
side?

Because we've had dcef induced issues recently, one of the first things I
did before we found the bad cable was switch dcef off and back on quickly.
For some reason, this killed dcef.  According to show run and show ip
interface, dcef/distributed switching was enabled.  According to show int
stat and show proc, it definitely was not...everything was being
route-cache switched by the RSP and RSP CPU was up around 95%.  I screwed
around with route-caching algorithms on the interfaces trying to get them
to start doing dcef again with no effect.  I flipped dcef off/on several
more times (conf t, ip cef, ip cef dist) with no effect.  Just before
giving up, I decided to repeat the above, but with a longer pause between
cef and cef dist.  That finally got dcef back and RSP CPU load down to
normal levels.

Two more odd things resulted from this.  First, I've started seeing
Dec  7 05:00:32: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 06:00:45: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 07:00:59: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 08:01:11: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 09:01:24: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 10:01:37: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 11:01:50: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 12:02:03: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 13:02:17: %TFIB-7-SCANSABORTED: TFIB scan not completing.
Dec  7 14:02:30: %TFIB-7-SCANSABORTED: TFIB scan not completing.

Second, the issue of the portchannel saying traffic is being dcef switched
but the FE members saying its being processor switched (from show int
stat) is gone.  Now the FE members also say the traffic is dcef switched.

FastEthernet0/1/0
          Switching path    Pkts In   Chars In   Pkts Out  Chars Out
               Processor     629281   62803052     263488   22747508
             Route cache          0          0          0          0
       Distributed cache  769990862 341809026014  563387047 147930109164
                   Total  770620143 341871829066  563650535 147952856672


----------------------------------------------------------------------
 Jon Lewis                   |  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________