[nsp] VCWare trouble

[Blaz Zupan]

After upgrading lots of AS5300 voice gateways to a new IOS and compatible
VCWare, one of the voice cards in a box decided not to cooperate, VCWare does
not come up and is stuck in "loading" mode. The other card in the same box
works fine.

For those that are not fammiliar with the specifics of AS5300, IOS for the box
does not contain the DSP firmware, it is available in a separate file called
"VCWare" that needs to be separately uploaded to the DSP board flash. The card
can be upgraded in two modes, VCWare mode and ROMMON mode.

I upgraded the troubled card in VCWare mode. After reboot, the card did not
come up. Here is what it has to say:

vgw#show vfc 1 dir
Voice Feature Card in slot 1 is not ready. It may be in ROM Monitor mode.
Please, check. If it is not, try later.
vgw#show vfc 1 board
VFC board state is VCWARE loading, vfc status initializing(0x3)
VFC board in slot 1 with 30 dsps
vgw#show vfc 1 int

 Rx: in ptr 0, outptr 0
 Tx: in ptr 0  outptr  0
 0 in hw queue, 0 queue head , 0 queue tail
 Hardware is VFC out-of-band channel
 Interface : state RESET DSP instance (0x6248AF68)
     dsp_number 0, Channel ID 0
     tx outstanding 3, max tx outstanding 0
 Received 0 packets, 0 bytes, 0 giant packets
 0 drops, 0 no buffers, 0 input errors
 168 bytes output,  3 frames output
 0 bounce errors 0
DSP module 1 is not installed
DSP module 2 is not installed
DSP module 3 is not installed
DSP module 4 is not installed
DSP module 5 is not installed


Huh? No DSP module installed? Well, it has been installed for the past two
years and used in production ever since :-)

Ok, so I decide that the upgrade was somehow hosed and that I'd like to repeat
the upgrade. I can't do it in VCWare mode, as the VCWare does not come up. So
I try to do it in ROMMON mode.

vgw#clear vfc 1 purge
Purging will erase the entire VFC flash and put VFC in ROM Monitor mode.
Do you really want to reset this card ? [y/n]: y
Purging will take sometime.

Please, wait...
Error: Timeout: VFC is not in Rom Monitor Mode. slot 1

vgw#show vfc 1 board
VFC board state is ROM DIAG, vfc status initializing(0x3)
VFC board in slot 1 with 30 dsps

So, "clear vfc 1 purge" puts the card in ROM DIAG mode. About 15 seconds
later, its back to "VCWARE loading":

vgw#show vfc 1 board
VFC board state is VCWARE loading, vfc status initializing(0x3)
VFC board in slot 1 with 30 dsps


So, at this point I'm pretty much stuck. I can't get the card to go into
either VCWare or ROMMON mode, where I could retry the VCWare upload. For now,
I'll probably replace the card. Can anyone suggest how to get out of this mess
except for "trash the card or return it to Cisco"?