[nsp] VIP2 reloading with %RSP-3-FOREVER

[Ed Ravin]

I've got a Cisco 7513 running 12.0(25)S1 that's complaining about
a VIP2 with one FastEthernet adapter in it:

  %VIP2-3-MSG: slot5 VIP-3-SVIP_RELOAD: SVIP Reload is called.
  %VIP2-3-MSG: slot5 VIP-3-SYSTEM_EXCEPTION: VIP System Exception occurred sig=20, code=0x90051391, context=0x609110F4
  %RSP-3-FOREVER: cmd 2 to VIP2 (slot 5) took 61 usecs, done 1
  -Traceback= 4043E35C 4043EDA8 40435F8C 404337BC 403FF7A0 403A1CA4 4035C0A0 40384884 40384870
  %DBUS-3-DBUSINTERRSWSET: Slot 5, Internal Error due to VIP crash
  %RSP-3-RESTART: cbus complex
  -Traceback= 403F5468 403F4DA8 403FF640 40384884 40384870

This first happened to us a week or two ago, then happened today three
times in the space of an hour.

I've turned on "service single-slot-reload-enable" to reduce the effect
of the line card reload (i.e. hopefully avoid the evil "cbus complex") the
next time this happens.  In the meantime, does anyone have a clue about whether
this is hardware or software?  I'm using the Output Interpreter, but I
get the feeling that I'm talking to the Oracle at Delphi - the robot says
with an authoritative air that this is a software error, then gives me a
list of bugs that are either fixed in the version I'm running or totally
non-relevant.  Not exactly confidence-inspiring.  I've also fed it the
results of "show diag 5", and got an even less relevant list of bugs.

One thing I see is that the boot loader isn't as up to date as the IOS:

  IOS (tm) RSP Software (RSP-K3PV-M), Version 12.0(25)S1, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
  [...]
  ROM: System Bootstrap, Version 11.1(8)CA1, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
  BOOTLDR: RSP Software (RSP-BOOT-M), Version 12.0(12)S, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)

The boot image version is mentioned in bug in 12.0(14)S ( CSCds89389 / VIP2
crash on 12.0(14)S with FE and CT3 PAs ).  There is indeed a CT3 on the
box, along with a couple of T3 and HSSI adapters.

Any thoughts on the subject, or things to explore that I haven't done so
yet?  I keep track of my router stats with Cricket and haven't seen
anything unusual as far as CPU load or traffic.

Thanks,

	-- Ed