[j-nsp] EVPN IRB / gateway selection

[Nathan Ward]

Hi all,

Doing some EVPN work at the moment, and have come across something we’re struggling with, and can’t quite figure out if it’s a fundamental thing we’re missing or if we’ve missed a magic knob somewhere. Have tried the two sets of eyes thing, to no avail.

EVPN, eBGP underlay, iBGP overlay. Following the VLAN aware EVPN guide from the Juniper docs.

QFX10k spine, QFX5110 leaf. EVPN-VXLAN.

We have 2 “DCs” in the lab, each “DC" with 2 spines and one leaf (will be more leafs in prod, obviously!)

We are using a single EVPN domain at the moment, so there’s no real separation between the DCs other than there’s more hops to go inter-DC vs intra-DC - at the moment, again, lab, figuring out what is going to work best for us.

We are using the IRB virtual gateway functionality, on all 4 spines, and are finding that the IRB instance which gets selected by the leaf is the one which most recently came up - if we disable/undisable one, it starts attracting traffic from all leafs. Trace options on protocols evpn shows bits about MAC moves when an IRB comes up - for the autogenerated VRRP style gateway MAC.

Our expectation was that it would select the IRB on the shortest path - i.e. the shortest eBGP underlay path, but that is not at all what happens, the eBGP path appears to not be considered at all.

The ESI is the same (autogenerated) and is signalled as “all active”.
We have tried fiddling around with the gateway communities - no advertise, advertise, etc.

We have not (yet) tested an (mc-ish) LAG ESI, to see if the behaviour is the same there, or if it’s limited to IRB.

Is this a QFX5110 thing? There are notes in some docs about hardware limitations with ECMP, but this is not equal paths so not sure it’s related.
Have we missed a knob somewhere?

--
Nathan Ward