[nsp] Catalyst6509 GE interface hang without any indication

[Sukumar Subburayan]

Joe,

The hang problem I mentioned below happens due to a wraparound of an
internal value. This would happen even if nothing is connected to the
system. For the value to wrap around it could take upto 7 months and
could happen faster if you are doing any delete/squeeze operations on the
bootflash:

sukumar



On Fri, 4 Jun 2004, Joe Shen wrote:

> Thanks you all. Thanks for your kindly help.
>
> In past days, I'm trying to identify the exact source of problem.
>
> There has been three possible reasons:
>
> 1. Software bug with CatOS
> 2. Security problem with IOS12.1
> 3. Overloading of MSFC/PFC for netflow data collecting
>
> In order to make clear what's the exact reason, we set up a little
> testbed with a Catalyst6509 with the same CatOS & IOS version
> of the system which experienced problem.
>
> I've download a little C program which is announced to be used as
> remaking TCP security problem with IOS, and I tried with
> one of GE interface IP address, but it seems the GE interface kept on
> working. ( I'm not sure whether the program do covers all possible
> hacking problem.)
>
> We have also read all those page, you have kindly pointed out. And, find
> the software version is listed in the list.
>
> But,  I'm still not clear whether netflow data collecting could derive
> to system hang if the Catalyst6509's overall load is heavy?
> ( I used full mode of netflow data collecting )
>
>
> Has anyone or Cisco did experiments with sideeffect of netflow
> collecting on Catalyst6509 behavior?
>
> regards
>
> Joe
>
>
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Sukumar
> Subburayan
> Sent: Thursday, June 03, 2004 2:42 AM
> To: cisco-nsp at puck.nether.net
> Subject: Re: [nsp] Catalyst6509 GE interface hang without any indication
>
>
> Joe,
>
> Sorry, there was a typo in the bug-id I mentioned.
>
> It should be CSCeb39694 and *not* CSCed37694.
>
> sukumar
>
>
> >
> > >a) When the box hangs does the sup2 hang or is it just 15/1?
> >
> > With the first box suffered from interface hang, only ge-1/1  hang ( I
>
> > showed its info. in previous message), with the second on the whole
> > system hang. and,
>
> Your entire box hanging problem is related to a software caveat (please
> refer  CSCed37694). If you had a console to the box you probably would
> have seen some characters (like continuous "." or "R") printed.
>
> This caveat affects releases 7.6(1) thro 7.6(4) and only for Sup2. Since
> you are  running, 7.6(1), I think, this is the bug you are running into.
>
> Customers with Sup2 running 7.6.x code are encouraged to consider
> upgrading to 7.6(7).
>
> sukumar
>
> >
> > 6509C-msfc-hz>sh scp status
> > Rx 10691273,  Tx 7072654,  Sap 13
> > Id     Channel name    current/peak/retry/total  time(queue/process)
> > -- ------------------- ------------------------  -------------------
> > 0  SCP async: TCAM MGR  0/ 20/  0/  162         13412/13412/   4
> > 1  SCP async: Fake MCA  0/  0/  0/    0            0/   0/   0
> > 3  SCP async: RUN_CONF  0/ 33/  0/   59          176/ 172/ 172
> > 5  SCP async: HA SRM S  0/  2/  0/    7            0/   4/   0
> > 6  SCP async: HA SRM M  0/  1/  0/    6            0/   0/   0
> > 7  SCP Unsolicit:0      0/  3/  0/1233574            0/   0/ 396
> > 8  SCP async: Draco-NM  0/  1/  0/    2            0/ 424/   0
> > 9  SCP async: NMP       0/  0/  0/    0            0/   0/   0
> > 10 SCP async: constall  0/  0/  0/    0            0/   0/   0
> > 11 SCP async: [cfg] l3  0/ 16/  0/   19         1348/ 408/   8
> > 12 SCP async: [mls] l3  0/  2/  0/10248            8/ 472/  20
> >
> > but, show inband, sh scp failcnt  seems not work. how could I check
> > interface asicreg?
> >
> >
> > >4) To note a few things from your email below - 15/1: Is there any
> > correlation between Gig to M160 not sending/receiving traffic and 15/1
> > >freezing? OR is 15/1 freezing the only cause of traffic not being
> > forwarded. I guess I am saying the MSFC is problem and not the gig
> port
> > >between the 6500 and the m160. Is this valid?
> >
> > the problem went away when We restart ge-1/1. Other part seems good
> when
> > I noticed no traffic on the ge link.
> >
> > >5) Assuming that scp is not the problem I would set up a span and
> > capture traffic up until the msfc fails.  Then see if you can see what
> > >might be causing it (I have done this many times - Let me know if you
> > need some ideas).
> >
> > Maybe I have to do this, but the traffic on that switch is about
> > 800Mbps. Too heavy to my PC.
> >
> >
> > Thank you very much !
> >
> > Joe
> >
>