[f-nsp] BI-4XG errors

Wed Nov 3 12:24:29 EDT 2010

Still trying to figure out why my 4XG card isn't working properly, and I
thought I'd try this list again.  I switched out the SFMs, and I'm not
getting FAP errors, but the logs are showing path diagnostic failures, and
I'm getting some packet loss and packet corruption.

The logs are giving me a brand new error, at this point.  There are a number
of them like this:

INFO:Nov  3 08:13:50: LP3/NP3 packet path diagnostic failure (tx 115269, rx
115269, tx_err 0, rx_err 0, rx_data_err 115269)
INFO:Nov  3 03:42:39: LP3/NP3 packet path diagnostic failure (tx 113468, rx
113468, tx_err 0, rx_err 0, rx_data_err 113468)
ALARM:Nov  3 03:01:10: TM DRAM CRC: LP2/TM3  (Reg: 0xa50c, Value: 0x12265)
INFO:Nov  2 23:12:20: LP3/NP3 packet path diagnostic failure (tx 111667, rx
111667, tx_err 0, rx_err 0, rx_data_err 111667)
ALARM:Nov  2 20:58:19: TM DRAM CRC: LP2/TM3  (Reg: 0xa50c, Value: 0x11b5c)

and this:

SFM/FE| #|LP/TM | #|   CRC  |misalign|code grp|sig lock|link dwn

 ------+--+------+--+--------+--------+--------+--------+--------

  2/ 3 |31| 3/ 2 | 4|00000001|00000000|00000001|00000000|00000000

LP/TM | #|SFM/FE| #|   CRC  |misalign|code grp|sig lock|link dwn

 ------+--+------+--+--------|--------+--------+--------+--------

  2/ 4 | 4| 2/ 3 | 4|00000001|00000000|00000001|00000000|00000000

 LP/NP: PktPath|  TxCnt |  RxCnt |  TxErr |  RxErr |DataErr
--------------+--------+--------+--------+--------+--------
 2/ 3:        |0001c5d1|0001a968|00000000|00000001|00004377
 3/ 3:        |0001c5d0|0001c5d0|00000000|00000000|0001c5d0

LP/TM: DRAM CRC |       Cnt
----------------+----------
 2/ 3           |    76425

Only one port on the 4XG is spitting out bad packets.  Am I looking at a bad
memory chip, maybe?  Anyone have any thoughts on this?

Thanks again to all of you.

--Chuck

On Tue, Aug 31, 2010 at 11:19 AM, Chris Marlatt <cmarlatt at rxsec.com> wrote:

> On 08/31/2010 12:42 PM, Chuck Ufarley wrote:
> > Hey everybody--
> >
> > I've got a 4XG card giving out errors on one port.
> >
> > The pertinent information in the logs seems to be: LP4/TM3 has shutdown
> > (ALARM:TM DRAM CRC: LP4/TM3  (Reg: 0xa50c, Value: 0x5) (shutdown)
> >
> > I'm also seeing this in the show tech output:
> > Slot 4/FAP3/Link 1 (N)-- SNM1/FE3/Link45 : down (FAP shutdown)
> > Slot 4/FAP3/Link 3 (A)-- SNM2/FE3/Link45 : down (FAP shutdown)
> > Slot 4/FAP3/Link 4 (A)-- SNM2/FE3/Link 5 : down (FAP shutdown)
> > Slot 4/FAP3/Link 5 (B)-- SNM1/FE1/Link43 : down (FAP shutdown)
> > Slot 4/FAP3/Link 7 (N)-- SNM2/FE1/Link43 : down (FAP shutdown)
> > Slot 4/FAP3/Link 8 (N)-- SNM1/FE1/Link 3 : down (FAP shutdown)
> > Slot 4/FAP3/Link 9 (N)-- SNM2/FE1/Link 3 : down (FAP shutdown)
>
> This is definitely where you want to concentrate. This error says the
> port has lost it's connectivity to the switch fabric.
>
> The following output from a healthy 4XG module:
>
> BI#show snm-links by-lp 8
> Slot 8/FAP1/Link 1 (N)-- SNM1/FE3/Link58 : up
> Slot 8/FAP1/Link 2 (N)-- SNM3/FE3/Link58 : up
> Slot 8/FAP1/Link 3 (A)-- SNM2/FE3/Link58 : up
> Slot 8/FAP1/Link 4 (A)-- SNM2/FE3/Link18 : up
> Slot 8/FAP1/Link 5 (A)-- SNM3/FE1/Link58 : up
> Slot 8/FAP1/Link 6 (N)-- SNM3/FE1/Link18 : up
> Slot 8/FAP1/Link 7 (N)-- SNM2/FE1/Link58 : up
> Slot 8/FAP1/Link 8 (N)-- SNM1/FE1/Link18 : up
> Slot 8/FAP1/Link 9 (N)-- SNM2/FE1/Link18 : up
>
> Slot 8/FAP2/Link 1 (N)-- SNM1/FE3/Link57 : up
> Slot 8/FAP2/Link 2 (N)-- SNM3/FE3/Link57 : up
> Slot 8/FAP2/Link 3 (A)-- SNM2/FE3/Link57 : up
> Slot 8/FAP2/Link 4 (A)-- SNM2/FE3/Link17 : up
> Slot 8/FAP2/Link 5 (A)-- SNM3/FE1/Link57 : up
> Slot 8/FAP2/Link 6 (N)-- SNM3/FE1/Link17 : up
> Slot 8/FAP2/Link 7 (N)-- SNM2/FE1/Link57 : up
> Slot 8/FAP2/Link 8 (N)-- SNM1/FE1/Link17 : up
> Slot 8/FAP2/Link 9 (N)-- SNM2/FE1/Link17 : up
>
> Slot 8/FAP3/Link 1 (N)-- SNM1/FE3/Link33 : up
> Slot 8/FAP3/Link 2 (N)-- SNM3/FE3/Link33 : up
> Slot 8/FAP3/Link 3 (A)-- SNM2/FE3/Link33 : up
> Slot 8/FAP3/Link 4 (A)-- SNM2/FE3/Link 9 : up
> Slot 8/FAP3/Link 5 (A)-- SNM3/FE1/Link33 : up
> Slot 8/FAP3/Link 6 (N)-- SNM3/FE1/Link 9 : up
> Slot 8/FAP3/Link 7 (N)-- SNM2/FE1/Link33 : up
> Slot 8/FAP3/Link 8 (N)-- SNM1/FE1/Link 9 : up
> Slot 8/FAP3/Link 9 (N)-- SNM2/FE1/Link 9 : up
>
> Slot 8/FAP4/Link 1 (N)-- SNM1/FE3/Link34 : up
> Slot 8/FAP4/Link 2 (N)-- SNM3/FE3/Link34 : up
> Slot 8/FAP4/Link 3 (A)-- SNM2/FE3/Link34 : up
> Slot 8/FAP4/Link 4 (A)-- SNM2/FE3/Link10 : up
> Slot 8/FAP4/Link 5 (A)-- SNM3/FE1/Link34 : up
> Slot 8/FAP4/Link 6 (N)-- SNM3/FE1/Link10 : up
> Slot 8/FAP4/Link 7 (N)-- SNM2/FE1/Link34 : up
> Slot 8/FAP4/Link 8 (N)-- SNM1/FE1/Link10 : up
> Slot 8/FAP4/Link 9 (N)-- SNM2/FE1/Link10 : up
>
> BI#
>
> BI#show module | i S8
> S8: RX-BI-4XG 4-port 10GbE Module  CARD_STATE_UP    4   0012.f29c.cd00
> BI#
>
> As you can see it would appear as though each port asic has 9 links to
> the switch fabric.
>
> Perhaps cycling the module or SFM would resolve the issue.
> Brocade/Foundry TAC may have a more graceful solution though.
>
> Regards,
>
>        Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/foundry-nsp/attachments/20101103/77401904/attachment.html>