[f-nsp] NI-MLX-10Gx4 shown as "Invalid Module" maybe due to very old code on lp while chassis is on 5.3

Wed Jul 24 17:22:03 EDT 2013

Hello Wilbur,

thank you very much for your reply. 

First of all I have now removed the manual module-type-setting for slot 4,
unplugged the lp from slot 4 and reinserted it.

Now the MLX says: S4: Invalid Module  CARD_STATE_REBOOT   0
000c.dbe1.2f90

This is where I tried “lp boot system interactive 4” which is being
confirmed in syslog with “cr1.fra1.mainlab.net Module 4 is reset by mgmt
(reason: CLI command)”.

Unfortunately the lp stays in “CARD_STATE_REBOOT”. The same applies then
powering off and on again. Interactive-boot mode didn’t help 

L

My software - including mbridge - is up to date:

#show ver

Boot     : Version 5.3.0T165 Copyright (c) 1996-2009 Brocade Communications
Systems, Inc.

Monitor  : Version 5.3.0T165 Copyright (c) 1996-2009 Brocade Communications
Systems, Inc.

IronWare : Version 5.3.0eT163 Copyright (c) 1996-2009 Brocade Communications
Systems, Inc.

Board ID : 00 MBRIDGE Revision : 37

Do you have any other idea how I can get the module out of the reboot-loop
and access is?

Your help is being appreciated – thanks in advance.

Gunther

Von:

 Wilbur Smith [mailto:wsmith at brocade.com] 

Gesendet:
 Mittwoch, 24. Juli 2013 22:30

An:
 Gunther Stammwitz; foundry-nsp at puck.nether.net

Betreff:
 RE: [f-nsp] NI-MLX-10Gx4 shown as "Invalid Module" maybe due to
very old code on lp whlile chassis is on 5.3

Gunther,

Usually when this happens to me, I can still force the LP to boot into
“interactive mode” with the command I mentioned
 and then manually powering on and off the LP ( power-off lp 4 , power-on lp
4). Since the MP’s access to the LP uses a separate 
out-of-band link, the MP should still be able to upgrade the LP if it is in
interactive mode.

The MP uses a separate FPGA image to allow it to talk to the chassis
backplane; we call this the MBRIDGE image. I would make sure that the
MBRIDGE image is the recommended release for your version of code and update
if needed. The required version is listed in the release notes for the
specific release of code (just search for mbridge). In rare cases, a patch
release may need a newer FPGA or MBRIDGE image to fix a problem, so make
sure you check the release notes for the exact release (5.3.0 vs. 5.3.0C). 

The error you are seeing when you tried to push the fpga code to the LP is
triggered because that LP is stuck in reboot. The MP can’t communicate with
that LP to see if it needs the newer version; we need to stop the reboot
loop before we can update the LP. I connected to one of my MLX routers in a
lab and double-checked the command you will need. Make sure you are using
this command to reboot a module in slot 4:

SSH at EA_CORE-1#lp boot system interactive 4

Let me know if this helps with your issue.

Wilbur

From:

 foundry-nsp [
mailto:foundry-nsp-bounces at puck.nether.net
] 
On Behalf Of 
Gunther Stammwitz

Sent:
 Tuesday, July 23, 2013 11:46 AM

To:

foundry-nsp at puck.nether.net

Subject:
 [f-nsp] NI-MLX-10Gx4 shown as "Invalid Module" maybe due to very
old code on lp whlile chassis is on 5.3

Hello colleagues,

I have more or less a similar problem. I have pushed a NI-MLX-10gX4 into
 a chassis running 5300e and it didn’t work: The module in slot 4 is not
even being recognized: it is an “invalid module”.

Most probably this is due to very old code on the lp while the chassis runs
a newer code.

#show mod

 Module

 Status 

Ports
 Starting MAC

M1 (left): NI-MLX-MR Management Module

 Active

M2 (right):

F1: NI-X-SF
 Switch Fabric Module

 Active

F2: NI-X-SF
 Switch Fabric Module

 Active

F3:

S4: Invalid Module
 CARD_STATE_REBOOT

 0

 xxx

(S4: Configured as NI-MLX-10Gx4 4-port 10GbE Module)

I can neither access it with “rconsole 4” nor does “boot lp boot system
interactive 4” help.

Power-off and power-on didn’t help.

Wilbur wrote that MP and LP cannot communicate and that the FPGA needs to be
upgraded. I tried to do so, but “copy tftp lp 1.2.3.4 lpfpga05300e.bin
fpga-all 4” didn’t work.

Copying FPGA images to the applicable slot(s), this may take several
minutes... 

à

 No FPGA image to be copied. 

L

Using copy tftp lp with individual-fpga-images doesn’t work either, the
system always claims that the fpga doesn’t match.

Any idea how to get a LP with very old code working in a chassis running
5.3-code?

You help is being appreciated.

Kind regards

Gunther

Von:

 foundry-nsp [
mailto:foundry-nsp-bounces at puck.nether.net
] 
Im Auftrag von 
Wilbur Smith

Gesendet:
 Freitag, 5. Juli 2013 00:53

An:
 Jeroen Wunnink | Atrato IP Networks; 
foundry-nsp at puck.nether.net

Betreff:
 Re: [f-nsp] NI-MLX-10Gx8-D falls to CARD_STATE_INTERACTIVE mode

Hello Folks,

Using auto lp-syc is a good improvement over the previous methods of
upgrading a line card, but there are times it may not do the trick. Because
the management modules also use an FPGA (the mbridge) to communicate with
the backplane, you can run into a situation where the older module’s FPGA
image cannot talk to the management module. The LP and management modules
have their own separate out-of-band network that bypasses the backplane, so
auto-lp will attempt to stop the reboot cycle and use this connection to
push the correct code to a misbehaving LP. Unfortunately, this may not work
in every situation, so you may need to manually upgrade the code.

If you have a linecard constantly rebooting, you should use “boot lp 3
interactive” from Enable mode to force the card to start in interactive mode
(this will stop the reboot loop). You can then push the correct software and
fpga image to the card. I tend to use the older “copy tftp image” method
because it will automatically copy the correct IronWare LP code and you can
target a specific LP with it vs. the whole chassis. I also use “copy tftp lp

. Fpga-all lp 3” to automatically sync the correct FPGA code to a specific
module.

Alternately, you can use the new “copy tftp system” method to automatically
up grade all code and fpga images on an LP. The command should only push to
a module that needs to be upgraded, but I would make sure you have an outage
window if you do this on a live system. 

There is some good guides on doing this at the Brocade Communities site and
the software upgrade guide for each version of IronWare has details notes on
how to use the methods I mentioned and how to fix common problems when
things go wrong. Check my.brocade.com for more info.

Wilbur

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/foundry-nsp/attachments/20130724/2fbd21b2/attachment-0001.html>