[c-nsp] Weird IOS problem
Lawrence Wong
lawrencewong72 at yahoo.com
Wed Jan 5 14:48:25 EST 2005
I used to encounter similar errors as well.
Everything worked fine on 12.2.18(S) but starting from
S1 (most problematic) to S6, random DBUS errors
occured. 12.2.25S and 12.2.25S1 worked fine though
(and on the exact same hardware).
The hardware that I tested on are a mix of 7505/7507
with Single/Dual RSP2/RSP4 and VIP2-40/VIP2-50/VIP4-50
with PA-FE-TX.
Seems more like something to do with the IOS instead
of hardware.
One thing I did notice for 12.2.18S1 to 12.2.18S6 is
that when I did a "sh diagbus <slot>":
e.g.
Slot 1:
Physical slot 1, ~physical slot 0xE, logical
slot 1, CBus 0
Microcode Status 0x4
Master Enable, LED, WCS Loaded
Board is analyzed
Pending I/O Status: None <-----
EEPROM format version 1
Instead of saying "None", some other text is written
(I cannot recall it offhand). And when it is not
"None", the DBUS errors are more or less guaranteed to
occur within some hours.
Anyone else having similar problems?
--- Hank Nussbacher <hank at mail.iucc.ac.il> wrote:
> At 09:09 AM 05-01-05 +0100, Brian Turnbow wrote:
>
> Our VIP4-50 has a PA-2FE and PA-8T and we suspect
> the PA-2FE as being the
> problem. Thing is it had been working for 3 years
> without a problem till
> now. Weird. -Hank
>
> > I had a similar problem with (12.2(18)S7)on a
> 7507 rsp 4 and a vip4-50
> >With a combination STM-1 FE Pas. I changed the vip
> and Pas with no change.
> >Replacing the STM-1 with a E3 resolved everything
> and My router is now happy.
> >The other 3 VIPs with different PA combinations
> were ok.
> >
> >-----Original Message-----
> >From: cisco-nsp-bounces at puck.nether.net
> >[mailto:cisco-nsp-bounces at puck.nether.net] On
> Behalf Of Hank Nussbacher
> >Sent: mercoledì 5 gennaio 2005 7.16
> >To: cisco-nsp at puck.nether.net
> >Subject: [c-nsp] Weird IOS problem
> >
> >This is a swing in the dark that perhaps someone
> else has seen this
> >before. Our 7513 (w/ RSP16) running 12.2(18)S7
> seems to be having a bad
> >day. One particular VIP4-50 (slot1) started
> misbehaving and it causes all
> >other VIPs to lose their CEF and for that
> particular VIP to take itself
> >off-line!
> >
> >Log output:
> >Jan 5 07:50:57: %CBUS-3-CMDTIMEOUT: Cmd timed out,
> CCB 0xF800FF30, slot
> >1, cmd code 2 -Traceback= 404F8334 404F8BF8
> 404EFF44 404ECD40 404CE974
> >4045C4A4 403D6B94 Jan 5 07:50:58:
> %DBUS-3-DBUSINTERR: Slot 1, Internal
> >Error Jan 5 07:50:58: %CBUS-3-CCBCMDFAIL1:
> Controller 1, cmd (61
> >0x00000008) failed (0x8010) Jan 5 07:50:58:
> %CBUS-3-CCBCMDFAIL1:
> >Controller 1, cmd (36 0x00000008) failed (0x8010)
> Jan 5 07:50:58:
> >%CBUS-3-ADDRFILTR: Interface FastEthernet1/0/1,
> address filter write
> >command failed, code 0x8010 -Traceback= 404FE1F8
> 404FEA40 404FECD4
> >40403F08 40651774 409C5234 409C588C
> >40A09674 409E5960 40420C1C 404DDDFC 404DDF80
> 404E4990 404CE86C Jan 5
> >07:50:58: %CBUS-3-CCBCMDFAIL1: Controller 1, cmd
> (36 0x0000FFFF) failed
> >(0x8010) Jan 5 07:50:58: %CBUS-3-CCBCMDFAIL1:
> Controller 1, cmd (36
> >0x00000100) failed (0x8010) Jan 5 07:50:58:
> %CBUS-3-CCBCMDFAIL1:
> >Controller 1, cmd (36 0x00000100) failed (0x8010)
> Jan 5 07:50:58:
> >%CBUS-3-CCBCMDFAIL1: Controller 1, cmd (36
> 0x00003333) failed (0x8010)
> >Jan 5 07:50:58: %CBUS-3-CCBCMDFAIL1: Controller 1,
> cmd (36 0x00003333)
> >failed (0x8010) Jan 5 07:50:58:
> %CBUS-3-CCBCMDFAIL1: Controller 1, cmd
> >(36 0x00003333) failed (0x8010) Jan 5 07:50:58:
> %CBUS-3-CCBCMDFAIL1:
> >Controller 1, cmd (36 0x00003333) failed (0x8010)
> Jan 5 07:50:59:
> >%OSPF-5-ADJCHG: Process 378, Nbr 192.114.22.1 on
> >FastEthernet1/0/1 from 2WAY to DOWN, Neighbor Down:
> Interface down or
> >detached Jan 5 07:50:59: %OSPF-5-ADJCHG: Process
> 378, Nbr 192.114.99.52 on
> >FastEthernet1/0/1 from FULL to DOWN, Neighbor Down:
> Interface down or
> >detached Jan 5 07:51:12: %DBUS-3-SW_NOTRDY: DBUS
> software not ready after
> >HARD_RESET, elapsed 13012, status 0x0 -Traceback=
> 404DE264 404E5E64
> >404F4620 404F7E1C 404E49B4 404CE86C Jan 5
> 07:51:25: %DBUS-3-SW_NOTRDY:
> >DBUS software not ready after HARD_RESET, elapsed
> 13036, status 0x0
> >-Traceback= 404DE264 404E5E64 404F7EA0 404E49B4
> 404CE86C Jan 5 07:51:38:
> >%DBUS-3-SW_NOTRDY: DBUS software not ready after
> RESET, elapsed 13036,
> >status 0x0 -Traceback= 404DE264 404E5A78 404F7EB8
> 404E49B4 404CE86C Jan 5
> >07:51:51: %DBUS-3-SW_NOTRDY: DBUS software not
> ready after RESET, elapsed
> >13028, status 0x0 -Traceback= 404DE264 404DEA78
> 404E3B44 404DC470 404DC8B4
> >404DCD90 404F7EE4
> >404E49B4 404CE86C
> >Jan 5 07:52:02: %MDS-2-RP: MDFS is disabled on
> some line card(s). Use
> >"show ip mds stats linecard" to view status and
> "clear ip mds linecard" to
> >reset.
> >Jan 5 07:54:28: %DBUS-3-SW_NOTRDY: DBUS software
> not ready after
> >DOWNLOAD, elapsed 20024, status 0x40 -Traceback=
> 404DE264 404E3E60
> >404DC470 404DC8B4 404DCD90 404F7EE4 404E49B4
> 404CE86C Jan 5 07:54:28:
> >%DBUS-3-WCSLDERR: Slot 1, error loading WCS, status
> 0x40 cmd/data 0xB6 pos
> >4818686 Jan 5 07:54:28: %UCODE-3-LDFAIL: Unable to
> download ucode from
> >system image in slot 1, trying rom ucode Jan 5
> 07:54:42:
> >%DBUS-3-SW_NOTRDY: DBUS software not ready after
> HARD_RESET, elapsed
> >13064, status 0x0 -Traceback= 404DE264 404E1448
> 404E7BDC Jan 5 07:54:55:
> >%DBUS-3-SW_NOTRDY: DBUS software not ready after
> RESET, elapsed 13064,
> >status 0x0 -Traceback= 404DE264 404DEA78 404E3B44
> 404DC470 404DC8B4
> >404DCD90 404E1450 404E7BDC Jan 5 07:55:54:
> %FIB-3-FIBDISABLE: Fatal
> >error, slot/cpu 2/0: No window message, LC to RP
> IPC is non-operational
> >Jan 5 07:55:54: %FIB-3-FIBDISABLE: Fatal error,
> slot/cpu 4/0: No window
> >message, LC to RP IPC is non-operational Jan 5
> 07:55:54:
> >%FIB-3-FIBDISABLE: Fatal error, slot/cpu 10/0: No
> window message, LC to RP
> >IPC is non-operational Jan 5 07:55:54:
> %FIB-3-FIBDISABLE: Fatal error,
> >slot/cpu 3/0: No window message, LC to RP IPC is
> non-operational Jan 5
> >07:55:54: %FIB-3-FIBDISABLE: Fatal error, slot/cpu
> 8/0: No window message,
> >LC to RP IPC is non-operational Jan 5 07:55:54:
> %FIB-3-FIBDISABLE: Fatal
> >error, slot/cpu 11/0: No window message, LC to RP
> IPC is non-operational
> >Jan 5 07:55:54: %FIB-3-FIBDISABLE: Fatal error,
> slot/cpu 9/0: No window
> >message, LC to RP IPC is non-operational .Jan 5
> 07:57:33:
> >%DBUS-3-SW_NOTRDY: DBUS software not ready after
> DOWNLOAD, elapsed 20012,
> >status 0x40 -Traceback= 404DE264 404E3E60 404DC470
> 404DC8B4 404DCD90
> >404E1450 404E7BDC .Jan 5 07:57:33:
> %DBUS-3-WCSLDERR: Slot 1, error
> >loading WCS, status 0x40 cmd/data 0xB6 pos 4818686
> .Jan 5 07:57:33:
> >%UCODE-3-LDFAIL: Unable to download ucode from
> system image in slot 1,
> >trying rom ucode .Jan 5 07:57:33: %RSP-3-NOSTART:
> No microcode for
> >VIP4-50 RM5271 card, slot 1 .Jan 5 07:57:34:
> %MDS-2-LC_FAILED_IPC_ACK: RP
> >failed in getting Ack for IPC message of size 80 to
> LC in slot 1 with
> >sequence 40250, error = retry queue flush .Jan 5
> 07:57:55:
> >%SNMP-3-AUTHFAIL: Authentication failure for SNMP
> req from host
> >132.74.1.154 .Jan 5 08:00:46: %FIB-3-FIBDISABLE:
> Fatal error, slot/cpu
> >8/0: keepalive failure .Jan 5 08:00:46:
> %FIB-3-FIBDISABLE: Fatal error,
> >slot/cpu 4/0: keepalive failure .Jan 5 08:00:46:
> %FIB-3-FIBDISABLE: Fatal
> >error, slot/cpu 9/0: keepalive failure .Jan 5
> 08:00:46:
> >%FIB-3-FIBDISABLE: Fatal error, slot/cpu 11/0:
> keepalive failure .Jan 5
> >08:00:46: %FIB-3-FIBDISABLE: Fatal error, slot/cpu
> 10/0: keepalive failure
> >.Jan 5 08:00:46: %FIB-3-FIBDISABLE: Fatal error,
> slot/cpu 2/0: keepalive
> >failure .Jan 5 08:00:46: %FIB-3-FIBDISABLE: Fatal
> error, slot/cpu 3/0:
> >keepalive failure TAU-gp1#sho cef line
> > Slot/CPU MsgSent XDRSent Window LowQ
> MedQ HighQ Flags
> > 2 5669 160596 LC wait 0
> 0 0 disabled
> > 3 5669 160598 LC wait 0
> 0 0 disabled
> > 4 5738 161622 LC wait 0
> 0 0 disabled
> > 8 5742 161621 LC wait 0
> 0 0 disabled
> > 9 5668 160577 LC wait 0
> 0 0 disabled
> > 10 5668 160574 LC wait 0
> 0 0 disabled
> > 11 5668 160577 LC wait 0
> 0 0 disabled
> >
> >VRF Default, version 311259, 153409 routes
> > Slot/CPU Version CEF-XDR I/Fs State
> Flags
> > 2 310726 157274 9 Active
> table-disabled
> > 3 310726 157274 8 Active
> table-disabled
> > 4 310726 157484 6 Active
> table-disabled
> > 8 310726 157484 6 Active
> table-disabled
> > 9 310726 157274 8 Active
> table-disabled
> > 10 310726 157274 18 Active
> table-disabled
> > 11 310726 157274 6 Active
> table-disabled
> >TAU-gp1#show ip mds stats linecard
> >
> >Slot Status IPC(seq/max/window)
> Q(high/route) Reloads
> > 1 disabled 40455/44286/3831 0/0
> 1
> > 2 active 258 /4354 /4096 0/0
> 2
> > 3 active 267 /4363 /4096 0/0
> 2
> > 4 active 284 /4380 /4096 0/0
> 2
> > 8 active 239 /4335 /4096 0/0
> 2
> > 9 active 275 /4371 /4096 0/0
> 2
> > 10 active 231 /2279 /2048 0/0
> 2
> > 11 active 249 /4345 /4096 0/0
> 2
> >
> >Taking out VIP1 from the chassis and rebooting
> solves the problem. We
> >have already replaced the VIP in slot1 yesterday
> when it first happened
> >and life went on for about 12 hours before it
> happened again.
> >
> >Anyone seen anything like this? Cisco output
> intepreter was of no help
> >since it indicated things that we don't run at all.
> >
> >Thanks,
> >Hank
> >_______________________________________________
> >cisco-nsp mailing list cisco-nsp at puck.nether.net
> >https://puck.nether.net/mailman/listinfo/cisco-nsp
> >archive at
> http://puck.nether.net/pipermail/cisco-nsp/
> >
> > +++++++++++++++++++++++++++++++++++++++++++
> > This Mail Was Scanned By Mail-seCure System
> > at the Tel-Aviv University CC.
>
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at
> http://puck.nether.net/pipermail/cisco-nsp/
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
More information about the cisco-nsp
mailing list