[j-nsp] Strange behavior on directly connected interfaces?

[Frances Albemuth]

  Hi Hannes,

  Thanks for your response.  When I'm sniffing on the segment I see a
massive stream of ICMP TTL exceeded messages being returned by the
"mystery node".  The topology is definitely loop-free and the
"loop-ish" behavior that we see only seems to occur when data is
transmitted to unreachable destinations.

 I assume by forwarding loop you mean an Ethernet loop? I would agree
that it behaves this way in some respects.  Of course, if I had a
genuine loop the problems would be more serious and would occur
regardless of routed traffic (the Ethernet topology with a handful of
hosts would cripple itself).

  Also interesting: the node returning the TTL exceeded "storm" lives
behind a link with a maximum synchronous capacity of 10M.  The "storm"
itself results in 10M of traffic pushing consistently over all ports
where the VLAN lives.  It thusly only cripples other devices with a
10M maximum synchronous bandwidth.

 Thanks!

  -FC

On 5/16/06, Hannes Gredler <hannes at juniper.net> wrote:
> frances,
>
> looks like you have a forwarding loop in your setup;
>
> for further troubleshooting attach a packet-sniffer to
> the subnet in question and spot for the source MAC-adress
> that is bouncing back your traffic.
>
> /hannes
>
>
> Frances Albemuth wrote:
> >  Hi,
> >
> >  This is my first post to the list and I would like to preface this by
> > stating that I doubt this problem is actually related specifically to
> > Juniper equipment (perhaps a configuration error involving Juniper
> > equipment, however). I'm hoping the issue I'm working on right now
> > might ring bells in the heads of others, and in any case I figure this
> > is as good a place as any to find yourself beaten by the clue stick.
> >
> >   I have a directly connected interface facing a large, flat Ethernet
> > infrastructure.  There are dozens of IP's mapped to the interface in
> > question (this is a legacy aspect of the design, but migration to a
> > more hierarchical infrastructure is a long process).  Periodically,
> > when packets are transmitted with an unreachable destination IP
> > residing on the directly connected interface,  a massive series of
> > ICMP TTL exceeded packets is returned by a different host residing on
> > a different logical interface.  Traceroutes to the unreachable IP
> > similarly show a one-node loop (the same IP responds until the TTL=0).
> >  The node is always the same, but if unmitigated ICMP traffic is
> > permitted to and from addresses on the logical interface, sniffing the
> > wire shows this behavior occurring to and from a number of nodes.  I
> > haven't managed to duplicate the multi-node behavior in a
> > semi-controlled environment.
> >
> >   When sniffing the segment in question, the ICMP is clearly visible,
> > so for whatever reason it is universally broadcast, even though both
> > nodes involved in the ICMP communication are legitimate unicast
> > destinations.  If a ping is left running, these TTL exceeded messages
> > will continue an accelerate ad nauseum until a de facto pseudo
> > broadcast storm occurs, crippling access on every switching node where
> > the VLAN in question is mapped.  Usually (but not always) the
> > anomalies halt when the ping is killed.  The issue is largely
> > mitigated by denying all ICMP to and from addresses mapped to the
> > logical interface.
> >
> >   That's all I'm comfortable asserting about the issue at this time.
> > What I'm really digging for here is an explanation as to why when the
> > Juniper tries to transmit to an unreachable node, it doesn't discover
> > the node is unreachable due to a lack of response from an ARP request
> > and return ICMP unreachables on it's own.  I may have missed something
> > obvious here (I'm sort of hoping so) and would appreciate any
> > suggestions or experience from others.  If I've sent this message to a
> > woefully inappropriate list I would greatly appreciate a suggestion as
> > to a better place to bring my question(s).
> >
> >  Thanks,
> >
> >   -FC
> >
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp at puck.nether.net
> > http://puck.nether.net/mailman/listinfo/juniper-nsp
>