[c-nsp] IOS upgrade on 6500 yields wacky result.

[Drew Weaver]

I noticed that one of our 6500s was running 12.2(17d)SXB and I wanted to upgrade it to SXI3.

I did my normal procedure of changing the boot variables and then reloading the slave, the slave booted into SXI3 normally so I then forced switchover to the slave and the old-master rebooted and came up in SXI3 normally (as I have done so many times before). So now I had a system in SSO with 2x Supervisor 720s with SXI3.

The switch was up for about maybe an hour, and then I noticed that the ports on the (now master) Supervisor 720 were down so I checked and it seems that it failed back over to the original active SUP and now the following oddness is occurring.

1: The slave sup is now back to 12.2(17d)SXB (???)
2: In sh bootvar I see:

Standby BOOT variable is unobtainable due to IPC error
Standby CONFIG_FILE variable is unobtainable due to IPC error
Standby BOOTLDR variable is unobtainable due to IPC error
Standby Configuration register is 0x2102 unobtainable due to IPC error

3: In show log I see:

.Oct 28 08:12:30.310 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:12:30.310 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:12:30.310 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:12:30.470 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:16:09.237 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:16:09.241 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:16:09.241 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:16:09.241 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:37:13.604 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:37:13.604 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:37:13.604 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]
.Oct 28 08:37:13.604 EST: %HA_CONFIG_SYNC-3-IPC: IPC operation failed for [ cann
ot open standby port ] with error code [ no such port ]

4: The ports on the slave supervisor are up and are passing traffic and routing protocols even though it seems that the master and slave are not communicating.

5: at the tail end of the log I see:

Oct 28 08:08:44.362 EST: %PFREDUN-SP-4-PHYSMEM_MISMATCH: Asymmetrical redundant
configuration: Active SP has (524288/65536K) memory, Standby has (458752/65536K)
.
Oct 28 08:08:44.362 EST: %PFREDUN-SP-4-PHYSMEM_MISMATCH: Asymmetrical redundant
configuration: Active RP has (524288/65536K) memory, Standby has (458752/65536K)
.

(they're the same card...)

and

.Oct 28 08:08:57.570 EST: %MLS_RATE-4-DISABLING: The global switching mode is no
w 'truncated'. Disabling the Layer2 Rate Limiters.

It also seems that the slave supervisor in slot 6 is not responding to the console, either.

Anyone have any clue what I did wrong here?

Prior to this upgrade this box was up for over 3 years and working fine but I needed some new functionality.

-Drew