[j-nsp] Strange behavior on directly connected interfaces?

[Frances Albemuth]

  The issue can thus far be mitigated (believe it or not) by filtering
ICMP to and from the "mystery node", or by filtering ICMP to and from
every network on interface "A".  I'm in possession of the MAC of the
"mystery node" and I know exactly where it lives on the network, but
it doesn't seem to correspond oddly with anything and I haven't
identified anything quirky about the network configuration.  What else
should I be keeping an eye out for?

  -FC

On 5/17/06, Hannes Gredler <hannes at juniper.net> wrote:
> frances,
>
> to mitigate the problem while diagnosing
> you could configure a firewall that discards
> traffic from non-local-subnet sources.
>
> but lets focus on the loop:
>    what is the mac-adress of the mystery node ?
>
> /hannes
>
> Frances Albemuth wrote:
> >  Hi Hannes,
> >
> >  Thanks for your response.  When I'm sniffing on the segment I see a
> > massive stream of ICMP TTL exceeded messages being returned by the
> > "mystery node".  The topology is definitely loop-free and the
> > "loop-ish" behavior that we see only seems to occur when data is
> > transmitted to unreachable destinations.
> >
> > I assume by forwarding loop you mean an Ethernet loop? I would agree
> > that it behaves this way in some respects.  Of course, if I had a
> > genuine loop the problems would be more serious and would occur
> > regardless of routed traffic (the Ethernet topology with a handful of
> > hosts would cripple itself).
> >
> >  Also interesting: the node returning the TTL exceeded "storm" lives
> > behind a link with a maximum synchronous capacity of 10M.  The "storm"
> > itself results in 10M of traffic pushing consistently over all ports
> > where the VLAN lives.  It thusly only cripples other devices with a
> > 10M maximum synchronous bandwidth.
> >
> > Thanks!
> >
> >  -FC
> >
> > On 5/16/06, Hannes Gredler <hannes at juniper.net> wrote:
> >
> >> frances,
> >>
> >> looks like you have a forwarding loop in your setup;
> >>
> >> for further troubleshooting attach a packet-sniffer to
> >> the subnet in question and spot for the source MAC-adress
> >> that is bouncing back your traffic.
> >>
> >> /hannes
> >>
> >>
> >> Frances Albemuth wrote:
> >> >  Hi,
> >> >
> >> >  This is my first post to the list and I would like to preface this by
> >> > stating that I doubt this problem is actually related specifically to
> >> > Juniper equipment (perhaps a configuration error involving Juniper
> >> > equipment, however). I'm hoping the issue I'm working on right now
> >> > might ring bells in the heads of others, and in any case I figure this
> >> > is as good a place as any to find yourself beaten by the clue stick.
> >> >
> >> >   I have a directly connected interface facing a large, flat Ethernet
> >> > infrastructure.  There are dozens of IP's mapped to the interface in
> >> > question (this is a legacy aspect of the design, but migration to a
> >> > more hierarchical infrastructure is a long process).  Periodically,
> >> > when packets are transmitted with an unreachable destination IP
> >> > residing on the directly connected interface,  a massive series of
> >> > ICMP TTL exceeded packets is returned by a different host residing on
> >> > a different logical interface.  Traceroutes to the unreachable IP
> >> > similarly show a one-node loop (the same IP responds until the TTL=0).
> >> >  The node is always the same, but if unmitigated ICMP traffic is
> >> > permitted to and from addresses on the logical interface, sniffing the
> >> > wire shows this behavior occurring to and from a number of nodes.  I
> >> > haven't managed to duplicate the multi-node behavior in a
> >> > semi-controlled environment.
> >> >
> >> >   When sniffing the segment in question, the ICMP is clearly visible,
> >> > so for whatever reason it is universally broadcast, even though both
> >> > nodes involved in the ICMP communication are legitimate unicast
> >> > destinations.  If a ping is left running, these TTL exceeded messages
> >> > will continue an accelerate ad nauseum until a de facto pseudo
> >> > broadcast storm occurs, crippling access on every switching node where
> >> > the VLAN in question is mapped.  Usually (but not always) the
> >> > anomalies halt when the ping is killed.  The issue is largely
> >> > mitigated by denying all ICMP to and from addresses mapped to the
> >> > logical interface.
> >> >
> >> >   That's all I'm comfortable asserting about the issue at this time.
> >> > What I'm really digging for here is an explanation as to why when the
> >> > Juniper tries to transmit to an unreachable node, it doesn't discover
> >> > the node is unreachable due to a lack of response from an ARP request
> >> > and return ICMP unreachables on it's own.  I may have missed something
> >> > obvious here (I'm sort of hoping so) and would appreciate any
> >> > suggestions or experience from others.  If I've sent this message to a
> >> > woefully inappropriate list I would greatly appreciate a suggestion as
> >> > to a better place to bring my question(s).
> >> >
> >> >  Thanks,
> >> >
> >> >   -FC
> >> >
> >> > _______________________________________________
> >> > juniper-nsp mailing list juniper-nsp at puck.nether.net
> >> > http://puck.nether.net/mailman/listinfo/juniper-nsp
> >>
>