[f-nsp] MAC address table issues with ICX 6610 stack running 7.4.00f switch code

[Frank Bulk]

We replicated the issue with BTAC and continue troubleshooting.  BTAC
believes it's a packet processor issue -- we'll be more sure when we flip
the active member of the cross-stack LAG to the other stack member.

Frank

-----Original Message-----
From: foundry-nsp [mailto:foundry-nsp-bounces at puck.nether.net] On Behalf Of
Frank Bulk
Sent: Saturday, August 30, 2014 3:01 PM
To: foundry-nsp at puck.nether.net
Subject: [f-nsp] MAC address table issues with ICX 6610 stack running
7.4.00f switch code

A few weeks ago a customer alerted us to a packet loss issue that we
eventually traced down to a loop in a LAG on some access gear (not Brocade
gear).  

In the process of troubleshooting and looking at MAC address tables on the
intermediate gear we connected the WAN interface of a simple consumer-grade
router on ethernet 1/1/23 of ICX 6610 #1 so we had a pingable host.  What I
noticed, when graphing that port, is that we were seeing a lot of traffic
egressing the 1/1/23 -- anywhere from 200 kbps to 15 Mbps over the day!
That seemed like a lot more than the usual amount of broadcast traffic on
this 2500 host VLAN.  

This is an ICX 6610 stack running 7.4 switch code with almost all
connections being a cross-stack LAG.

Curious as to what was going on, I packet captured the port's output and
discovered a lot of unicast traffic flooding out of 1/1/23.  By doing some
troubleshooting I uncovered three different situations:
a) there are times the ICX 6610 lists the correct MAC address and port in
its table for a host yet it still floods (some) unicast traffic for that
host out of 1/1/23.
b) despite having a static mac address entry in the switch to a LAG port the
ICX6610 floods some traffic to that host out of 1/1/23 instead out of the
statically specified LAG port.
c) there are times the ICX 6610 has no MAC address table entry for a host,
even though it should have learned it because it's getting traffic from that
host on the LAG.  Entering a static MAC address and then removing it then
results in the switch learning it dynamically!

The only traffic I should be seeing out of 1/1/23 is spanning tree, ARP,
broadcast traffic, and any traffic for a host that has not yet been learned
by the switch.  

We opened up two cases with Brocade TAC and I was able to able to confirm
one of the two items with the tech, but since 7.4.00a had some MAC address
table issues (Defect ID 437017 is one of them), rather than troubleshoot
extensively we decided to start with a more current release and upgraded to
7.4.00f Thursday morning .... but I have already re-confirmed items (a) and
(c).

Has anyone else seen this issue?  I don't think you would really notice it
unless you really did some packet captures and looked for it.

Frank

_______________________________________________
foundry-nsp mailing list
foundry-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/foundry-nsp