[c-nsp] RSP-6-TXSTUCK - software or hardware?
Rodney Dunn
rodunn at cisco.com
Fri Aug 11 09:24:56 EDT 2006
Wait until it's back up. Clear the counters and let it run
for a bit and send me the output again.
If it's not showing as distributed or route-cache in 'sh int stat'
we have to look at the features you have configured to see why
it's punting.
On Fri, Aug 11, 2006 at 02:25:12PM +0100, David Freedman wrote:
> oh, just in case you are wondering, the interface is currently down
> (transmission people are having another go at the cabling)
> but in case you are concerned by the "stat" output (does not show CEF
> for some reason) it should definately be CEF switching:
>
> #sh cef int s6/0/0
> Serial6/0/0 is down (if_number 6)
> Corresponding hwidb fast_if_number 6
> Corresponding hwidb firstsw->if_number 6
> Internet address is x.x.x.x/y
> ICMP redirects are always sent
> Per packet load-sharing is disabled
> IP unicast RPF check is disabled
> Inbound access list is not set
> Outbound access list is not set
> IP policy routing is disabled
> BGP based policy accounting is disabled
> Interface is marked as point to point interface
> Hardware idb is Serial6/0/0
> Fast switching type 4, interface type 11
> IP Distributed CEF switching enabled
> IP Flow switching turbo vector
> IP Flow CEF switching turbo vector
> Input fast flags 0x0, Output fast flags 0x0
> ifindex 5(5)
> Slot 6 Slot unit 0 Unit 0 VC -1
> Transmit limit accumulator 0xE8001A82 (0xE8001A82)
> IP MTU 4470
>
> #execute-on slot 6 show cef int stat | in Serial6
> show cef int stat | in Serial6 from slot 6:
> Serial6/0/0 2557769059 540490892955 4558247 933422662
>
> Thanks,
>
> Dave.
>
>
> David Freedman wrote:
> > Sure:
> >
> > Rodney Dunn wrote:
> >> David,
> >>
> >> What those messages were designed for was to help
> >> identify if there was tx accumulator leaks on a 75xx before
> >> it started affecting traffic.
> >> It was one of the code areas we had some bugs so we put
> >> that in to try and help. However it caused some false positives
> >> so we removed it for E1's and below from what I remember.
> >>
> >> Pretty unilikely your load is high enough to get to the DS3 rate
> >> as long as it's really clocking at a T3.
> >>
> >> Can you provide:
> >> sh int se 6/0/0 stat
> >
> > #sh int s6/0/0 stat
> > Serial6/0/0
> > Switching path Pkts In Chars In Pkts Out Chars Out
> > Processor 20039 2084968 20039 2084968
> > Route cache 0 0 0 0
> > Distributed cache 0 0 0 0
> > Total 20039 2084968 20039 2084968
> >
> >
> >
> >
> >
> >>
> >> and sh contr cbus
> >
> >
> > #show controllers cbus | beg slot6
> > slot6: VIP2 R5K, hw 2.00, sw 22.20, ccb F800FF70, cmdq E80000B0, vps 8192
> > software loaded from system
> > IOS (tm) VIP Software (SVIP-DW-M), Version 12.3(10d), RELEASE
> > SOFTWARE (fc2)
> > ROM Monitor version 115.0
> > Mx T3+(1) HW Revision 0x3, FW Revision 3.93
> > Serial6/0/0, applique is T3+ DTE
> > received clockrate 1365
> > gfreeq E8000180, lfreeq E80001B0 (4512 bytes)
> > rxlo 4, rxhi 78, rxcurr 0, maxrxcurr 3
> > txq E8001A80, txacc E8001A82 (value 76), txlimit 78
> > FastEthernet6/1/0, addr 0001.9700.78c8 (bia 0001.9700.78c8)
> > gfreeq E8000160, lfreeq E80001B8 (1536 bytes)
> > rxlo 4, rxhi 526, rxcurr 4, maxrxcurr 29
> > txq E8001A88, txacc E8001A8A (value 264), txlimit 265
> >
> >
> >>
> >> from the box.
> >>
> >> See what the value/limit is for that serial 6/0/0 interface.
> >>
> >> There is a remote remote remote chance it could be bad hadware
> >> causing buffer header corruption which results in txacc leaks.
> >> I've only seen that 3 times every in 8 years but it is a chance.
> >> Almost always it's a software bug.
> >>
> >> Rodney
> >>
> >> On Thu, Aug 10, 2006 at 01:15:06PM +0100, David Freedman wrote:
> >>>
> >>> Hardware: 75xx(dual RSP4) with VIP2-50 populated with PA-T3+ and PA-FE-TX
> >>>
> >>> What would cause the following messages to occur?
> >>>
> >>>
> >>> Aug 9 21:13:29 BST: %RSP-6-TXSTUCK: Txacc of Interface Serial6/0/0 is
> >>> at 2% of its txlimit
> >>>
> >>> Aug 10 10:08:48 BST: %RSP-6-TXSTUCK: Txacc of Interface Serial6/0/0 is
> >>> at 3% of its txlimit
> >>>
> >>> Aug 10 12:02:00 BST: %RSP-6-TXSTUCK: Txacc of Interface Serial6/0/0 is
> >>> at 2% of its txlimit
> >>>
> >>>
> >>> Traffic on this interface is miniscule (i.e around 8/10Mbps) and on the
> >>> FE interface around 16/20Mbps, both with appropriate packet rates.
> >>>
> >>> Checking the graphs for the times that these events were reported ,
> >>> nothing untoward was occuring at the time.
> >>>
> >>> We also graph the CPU of the VIPs and the VIP didn't rise about 10% CPU
> >>> at these times.
> >>>
> >>> The box is a legacy piece of kit which has been beefed up somewhat to
> >>> run 12.3 (as it has IPv6 requirements) , its currently running 12.3(10)d
> >>> but I can't find any known bugs that refer directly to this.
> >>>
> >>> The T3 has been experiencing Input errors for around a week before these
> >>> messages were appearing:
> >>>
> >>> Serial6/0/0 is up, line protocol is up
> >>> Hardware is cyBus PODS3+ Serial
> >>> Description: Protection T path DS3 over 100239/450845-0/2308
> >>> MTU 4470 bytes, BW 44210 Kbit, DLY 200 usec,
> >>> reliability 255/255, txload 5/255, rxload 39/255
> >>> Encapsulation HDLC, crc 16, loopback not set
> >>> Keepalive set (10 sec)
> >>> Restart-Delay is 0 secs
> >>> Last input 00:00:00, output 00:00:00, output hang never
> >>> Last clearing of "show interface" counters 1d19h
> >>> Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops:
> >>> 9714
> >>> Queueing strategy: fifo
> >>> Output queue: 0/40 (size/max)
> >>> 30 second input rate 6825000 bits/sec, 1325 packets/sec
> >>> 30 second output rate 914000 bits/sec, 320 packets/sec
> >>> 610766319 packets input, 3114706794 bytes, 0 no buffer
> >>> Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
> >>> 0 parity
> >>> 158422 input errors, 137421 CRC, 0 frame, 18741 overrun, 16
> >>> ignored, 2260 abort
> >>> 34167551 packets output, 1658107765 bytes, 0 underruns
> >>> 0 output errors, 0 applique, 0 interface resets
> >>> 0 output buffer failures, 0 output buffers swapped out
> >>> 236 carrier transitions
> >>> rxLOS inactive, rxLOF inactive, rxAIS inactive
> >>> txAIS inactive, rxRAI inactive, txRAI inactive
> >>>
> >>>
> >>> I've been blaming the transmission people for the past week , but they
> >>> are as mystified as me, is it possible the TXSTUCK messages are a result
> >>> of some kind of events occuring on the transmission network?
> >>>
> >>> For reference, its configured as a point-to-point DS3, as so:
> >>>
> >>> interface Serial6/0/0
> >>> description Protection T path DS3 over 100239/450845-0/2308
> >>> load-interval 30
> >>> dsu bandwidth 44210
> >>> scramble
> >>> framing c-bit
> >>> cablelength 50
> >>> serial restart-delay 0
> >>> no clns route-cache
> >>> hold-queue 2000 in
> >>> end
> >>>
> >>>
> >>>
> >>> If we look at the PA-FE:
> >>>
> >>>
> >>> FastEthernet6/1/0 is up, line protocol is up
> >>> Hardware is cyBus FastEthernet Interface, address is 0001.9700.78c8
> >>> (bia 0001.9700.78c8)
> >>> Description: Worker ET Path 100239/450845-0/2307
> >>> MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
> >>> reliability 255/255, txload 39/255, rxload 13/255
> >>> Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
> >>> Keepalive set (10 sec)
> >>> Full-duplex, 100Mb/s, 100BaseTX/FX
> >>> ARP type: ARPA, ARP Timeout 04:00:00
> >>> Last input 00:00:00, output 00:00:06, output hang never
> >>> Last clearing of "show interface" counters never
> >>> Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
> >>> Queueing strategy: fifo
> >>> Output queue: 0/40 (size/max)
> >>> 5 minute input rate 5381000 bits/sec, 2258 packets/sec
> >>> 5 minute output rate 15681000 bits/sec, 2623 packets/sec
> >>> 1271168625 packets input, 2747320900 bytes, 0 no buffer
> >>> Received 14588 broadcasts, 0 runts, 0 giants, 0 throttles
> >>> 36 input errors, 0 CRC, 0 frame, 0 overrun, 36 ignored
> >>> 0 watchdog
> >>> 0 input packets with dribble condition detected
> >>> 1476558580 packets output, 3223841169 bytes, 0 underruns
> >>> 0 output errors, 0 collisions, 0 interface resets
> >>> 0 babbles, 0 late collision, 0 deferred
> >>> 0 lost carrier, 0 no carrier
> >>> 0 output buffer failures, 0 output buffers swapped out
> >>>
> >>>
> >>> Nothing out of the ordinary.
> >>>
> >>>
> >>> Not quite sure how to proceed, I'm considering replacing the PA and/or
> >>> the VIP but if anybody can suggest this as being a s/w oddity I'm
> >>> willing to replace/upgrade as needed (bugtool is not turning up anything
> >>> of interest here)
> >>>
> >>>
> >>> Any help/advice on this muchly appreciated.
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> Dave.
> >>>
> >>> _______________________________________________
> >>> cisco-nsp mailing list cisco-nsp at puck.nether.net
> >>> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >>> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >> _______________________________________________
> >> cisco-nsp mailing list cisco-nsp at puck.nether.net
> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>
> >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list