[j-nsp] ARP resolution algorithm? Storage of MX transit packets?

[Saku Ytti]

Hey Clarke,

On Thu, 31 Jan 2019 at 02:19, Clarke Morledge <chmorl at wm.edu> wrote:

> I am trying to wrap my head around how the MX handles ARP resolution,
> and how it stores packets waiting to be transmitted, while waiting for ARP
> to resolve.

This might answer some of your questions
http://blog.ip.fi/2014/02/junos-and-arp-glean.html

> In the event an ARP request is generated, and no response is received
> within x(???) seconds, it looks like another request is sent out, or more,
> perhaps? If no response ever comes back, after some period of time, I
> presume the router will drop the transit packet.

First transit packet being punted the hardware programs the adjacency
into a one which will drop subsequent packets. However programming is
not instantaneous, so if burst of packets come in, they honour the
original adjacency which will cause them to be punted.
The first packet being punted triggers the RE resolution process, and
what it does and how many times it tries is decoupled from the amount
of packets that are trying to transit.

> So, where is this transit packet being held, while it is waiting for the
> ARP reply to come back (if it even does come back)? How is the packet
> being stored? Are the packets stored via a hash in separate queues, of
> some sort, so that other transit traffic is not getting blocked?

This is good question. I believe it is not stored in HW at all, only
in control-plane.

> What is strange is that if there is a string of transit packets coming in,
> that have no corresponding ARP entry in the cache available, the way the
> RE sends out ARP requests does not exactly correspond to the order of
> transmit packets, as they come into the PFE.  I would have expected a
> FIFO-like mechanism, but this does not seem to be the case.

It is necessarily non-lock step due to delays needed to reinform hardware.


> Off the Wire, Just Before Traffic Enters the MX:
>
> 21:51:16.158064 IP 185.254.123.12.46185 > 100.64.101.189.3588:
> 21:51:16.297351 IP 92.63.194.38.47423 > 100.64.101.25.55126:
> 21:51:16.301438 IP 185.53.91.24.55823 > 100.64.101.88.5038:
> 21:51:16.385521 IP 185.176.27.34.58908 > 100.64.101.215.1288:
> 21:51:16.449858 IP 92.53.90.143.44499 > 100.64.101.192.282:
> 21:51:16.462591 IP 92.53.90.143.44499 > 100.64.101.181.282:
> 21:51:16.470221 IP 185.143.221.106.58528 > 100.64.101.1.4040:
> 21:51:16.492806 IP 92.63.194.38.47423 > 100.64.101.35.55126:
> 21:51:16.500132 IP 92.63.194.38.47423 > 100.64.101.58.55126:
>
> ARP Requests Coming Out of the RE:
>
> 21:51:16.158515 ARP, Request who-has 100.64.101.189 tell 100.64.101.3
> 21:51:16.227443 ARP, Request who-has 100.64.101.50 tell 100.64.101.3 # you dont have corresponding packet, so I assume this is retry of older punt
> 21:51:16.227985 ARP, Request who-has 100.64.101.158 tell 100.64.101.3 # same
> 21:51:16.297828 ARP, Request who-has 100.64.101.25 tell 100.64.101.3  #  order preserved in relation to 189
> 21:51:16.327204 ARP, Request who-has 100.64.101.59 tell 100.64.101.3  # again no packet seen above
> 21:51:16.327664 ARP, Request who-has 100.64.101.65 tell 100.64.101.3  # again no packet seen above
> 21:51:16.427452 ARP, Request who-has 100.64.101.2 tell 100.64.101.3 #  ..
> 21:51:16.428282 ARP, Request who-has 100.64.101.9 tell 100.64.101.3 # ..
> 21:51:16.473085 ARP, Request who-has 100.64.101.1 tell 100.64.101.3 # ..
> 21:51:16.527447 ARP, Request who-has 100.64.101.7 tell 100.64.101.3  # ..
> 21:51:16.528278 ARP, Request who-has 100.64.101.88 tell 100.64.101.3 # order preserved in relation to 189, 25

I see nothing odd there.

As an interesting side note, when JNPR implemented 'ddos-protection'
which is great feature, market-leading really. They totally broke ARP.
For around 3 years when ever glean packet needed punting, instead of
having special 'glean/resolve' classification it was classified by
what it was.
So say I was ddossing 100.64.101.50 (non-existing host) with BGP
packet, instead of causing 'glean/resolve' packets to be rate-limited,
I was causing BGP packets to be ratelimited, and you had _no
recourse_, I could bring down your BGP as I wish and not be subject to
our Lo0 filtering.

I wish some vendor would implement static DIP=>DADDR resolution, there
are many applications, such as VM only LANs, where you anyhow generate
MAC programmatically, you could generate it deterministically from DIP
and inform router about this, so then you'd never need to
glean/resolve punt in this interface.

-- 
  ++ytti