[j-nsp] BGP EVPN, VXLAN and ECMP

[Sebastian Wiesinger]

* Vincent Bernat <bernat at luffy.cx> [2018-03-28 16:26]:
> Hey!
> 
> I am trying to setup a Juniper QFX5100 as a VTEP with a very classic
> setup. Everything works as expected, but the setup is only using one
> possible path from the underlay network.
> 
> I have the route to the other VTEP like this:
> 
> # run show route 10.16.39.3
> 
> inet.0: 240 destinations, 1808 routes (240 active, 0 holddown, 0 hidden)
> + = Active Route, - = Last Active, * = Both
> 
> 10.16.39.3/32      *[BGP/140] 00:38:24, localpref 500, from 10.64.0.5
>                       AS path: I, validation-state: unverified
>                       to 10.64.0.23 via xe-0/0/46.181
>                     > to 10.64.128.23 via xe-0/0/47.183

Can you do a 'run show route 10.16.39.3 extensive'?

For the record this work on JunOS 17.3 here (ISIS as underlay):

Bridging domain: EVPN-TEST-1.evpn-vxlan [Index 7] 
VPLS:
Enabled protocols: Bridging, ACKed by all peers, EVPN VXLAN, 
    
Destination:  3c:8a:b0:db:28:83/48
  Learn VLAN: 0                        Route type: user                  
  Route reference: 0                   Route interface-index: 558 
  Multicast RPF nh index: 0         
  P2mpidx: 0              
  IFL generation: 136598               Epoch: 0   
  Sequence Number: 0                   Learn Mask: 0x4000000000000000010000000000000000000000
  L2 Flags: control_dyn
  Flags: sent to PFE
  Nexthop:  
  Next-hop type: composite             Index: 1723     Reference: 509  
  Next-hop type: indirect              Index: 131073   Reference: 3    
  Next-hop type: unilist               Index: 131070   Reference: 3    
  Nexthop: 172.17.1.0
  Next-hop type: unicast               Index: 1721     Reference: 6    
  Next-hop interface: et-0/0/48.0      Weight: 0x1  
  Nexthop: 172.17.2.0
  Next-hop type: unicast               Index: 1722     Reference: 6    
  Next-hop interface: et-0/0/49.0      Weight: 0x1  

One thing we discovered is that QFX5100 can only loadbalance in the
underlay (ECMP for the VTEP IP address) but not in the overlay for
ESIs. When you have an ESI that is reachable through two VTEPs, only
one will be used for forwarding. If that is a problem for you in
practice depends on where you attach stuff. If you have something
attached to the Spines (for example L3 to external) this might hit you
performance-wise.

It seems QFX5100 will do some sort of load-balancing for
Destination-MACs per ESI (reach MAC-A trough VTEP 1, reach MAC-B
trough VTEP 2) but I haven't tested that.

QFX10k on the other hand will install multiple next-hops for ESIs.

BTW: We're also seeing problems with third-party optics in JunOS 17.3
on QFX5k. CRC errors and problems with interfaces not coming up
instantly.

Regards

Sebastian

-- 
GPG Key: 0x93A0B9CE (F4F6 B1A3 866B 26E9 450A  9D82 58A2 D94A 93A0 B9CE)
'Are you Death?' ... IT'S THE SCYTHE, ISN'T IT? PEOPLE ALWAYS NOTICE THE SCYTHE.
            -- Terry Pratchett, The Fifth Elephant