[nsp] Cisco 3548/3550 MAC strangeness

joe lin jlin at doradosoftware.com
Thu Feb 13 09:30:17 EST 2003


What kind of boxes is running the linux boxes?  Once had linux running on a
certain vendor that would spew broadcast storms til the boxes got rebooted.


----- Original Message -----
From: "Deepak Jain" <deepak at ai.net>
To: "Michael Loftis" <mloftis at wgops.com>; "Cisco-Nsp at Puck. Nether. Net"
<cisco-nsp at puck.nether.net>
Sent: Thursday, February 13, 2003 3:08 PM
Subject: RE: [nsp] Cisco 3548/3550 MAC strangeness


>
> These are almost entirely modern Linux or BSD based systems. The two or
> three Windows boxes on the network don't have anywhere near the traffic
> levels (observed) to be responsible for even 1% of the broadcast storm,
but
> I can have them inspected for proper netmasks and segmented to see if the
> symptoms disappear.
>
> Thanks,
>
> DJ
>
> > -----Original Message-----
> > From: cisco-nsp-bounces at puck.nether.net
> > [mailto:cisco-nsp-bounces at puck.nether.net]On Behalf Of Michael Loftis
> > Sent: Thursday, February 13, 2003 1:34 AM
> > To: deepak at ai.net; Cisco-Nsp at Puck. Nether. Net
> > Subject: Re: [nsp] Cisco 3548/3550 MAC strangeness
> >
> >
> > Uhhhmmm...are these 400 windows systems, or other mixed systems?  This
> > smells badly like a broadcast storm being started by hosts incorrectly
> > responding to queries to the broadcast address(es) of their subnet.  Or
> > misconfigured host(s) with the wrong idea of what the subnet is
> > sending out
> > broadcasts when they really intended to send unicast packets.
> >
> > Say host A has 255.255.255.0 as it's mask and 10.1.1.0 as it's
> > network when
> > it is supposed to be 255.255.0.0 and 10.1.0.0 -- host  A sends a
> > broadcast
> > packet to 10.1.1.255 instead of ARPing and sending a unicast packet.
> > Having this happen enough, along with hosts that incorrectly respond to
> > this broadcast packet when it's incorrectly addressed can quickly
> > fire up a
> > nice broadcast storm.
> >
> > --On Wednesday, February 12, 2003 10:52 PM -0500 Deepak Jain
> > <deepak at ai.net> wrote:
> >
> > >
> > > Hi,
> > >
> > > I am working with a customer who has the following server farm
> > > arrangement.
> > >
> > > About 400 servers connected to approximately 10 Cisco 3548 and 3550
> > > switches. No VLANs, no loops, no Spanning Tree, no command
> > clusters. They
> > > each have about 20 IP addresses and share a single router. The
switches
> > > link in a left-to-right fashion use SX-GBICs and uplink from the last
> > > switch by single-attach SX-GE.
> > >
> > > The problem is that there are unicast broadcast floods on
> > the order of
> > > 100mb/s (at times, completely unpredictable) on many (not all)
> > ports from
> > > the switch ports to each of the servers. The exact amount per server
is
> > > not always the same, nor is the amount each switch is broadcasting.
> > >
> > > The machines are not rebooting, bridging or doing anything
> > funky. They
> > > are simply there. With a default MAC-aging time of 4 hours and only
400
> > > (+1 router) MAC addresses to learn, I can't understand why so much
> > > broadcast traffic is being generated. Even when the MAC-address aging
> > > time is set to 14400 minutes there is no change in the behavior.
> > >
> > > If "port block unicast" is turned on all the FE ports, the
> > floods (to the
> > > servers) stop, but the switches no longer report all the
> > machines they are
> > > attached to "show mac" nor do they report these MACs to their
> > neighboring
> > > switches even when direct connections are attempted. A clear ARP (on
the
> > > router) solves the problem for anywhere from a few seconds to a few
> > > minutes. Even if the switch has learned the MAC address, unless
> > > continuous traffic is sent to it, it will lose the server, even if its
> > > directly attached to the switch.
> > >
> > > Cisco TAC's solution was to hard-code EACH server's MAC
> > address into EACH
> > > switch via "mac-address secure xxxx.xxxx.xxxx f0/xx" which is
cumbersome
> > > and feels silly.
> > >
> > > We have eliminated individual pieces of hardware as any
> > switch of these
> > > series that are put in place repeat the behavior. IOS versions do not
> > > affect it either. VLANS would solve the problem, but the Customer is
> > > vehemently opposed to them, and I can't think of a reason why he needs
> > > them. We are only talking about a small amount of address space and no
> > > secondaries on the interfaces.
> > >
> > > Please give me an idea of what I need to look at to either solve the
> > > problem or prove why VLANs or some other solution would solve
> > it. I talked
> > > to a non-Cisco engineer and they seemed to feel that its a
> > problem at the
> > > ASIC-level of these units, but it seems so fundamental for
> > essentially an
> > > enterprise switch, I have a hard time believing that either.
> > >
> > > Thanks in advance,
> > >
> > > DJ
> > >
> > > _______________________________________________
> > > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > > http://puck.nether.net/mailman/listinfo/cisco-nsp
> > > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> >
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > http://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> >
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> http://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>



More information about the cisco-nsp mailing list