[c-nsp] EVPN/VXLAN on ASR9001 - now, L2FIB/VXLAN weirdness

[Gert Doering]

Hi,

On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> I'm trying to make EVPN via VXLAN encapsulation work between two ASR9001
> (with the goal of eventually making it work between ASR9001 and Arista
> boxes, but right now I'm failing ASR9001 <-> ASR9001 already).

So, spent some more hours on this, ignoring non-existant documentation,
and found a different way to configure EVPN-with-VXLAN.

Bridge-group like this:

 bridge group vlandb
  bridge-domain v2799
   interface TenGigE0/0/2/2.2799
   !
   routed interface BVI2799
   !
   vni 102799

(so no "member vni ..." and no "evi ..." either)

EVPN like this:

evpn
 vni 102799
  bgp
   rd 195.30.3.252:2799
  !
  advertise-mac
   bvi-mac
  !
 !


VTEP like this:

interface nve1
 member vni 102799
  host-reachability protocol bgp
 !
 source-interface Loopback30
 ingress-replication protocol bgp          <<<< new knob, docs lacking
!


and BGP neighbour like this (without any frills):

router bgp 5539
 neighbor 195.30.3.251
  remote-as 5539
  description m34/evpn-vxlan-test
  update-source Loopback30
  address-family l2vpn evpn
   encapsulation-type vxlan
   soft-reconfiguration inbound always
  !
 !
!


with that, I get proper BGP signalling, and I see my EVPN VLAN and
the associated MAC addresses in "show evpn evi", "show evpn mac":

RP/0/RSP0/CPU0:M52#show evpn evi
VPN-ID     Encap  Bridge Domain                Type               
---------- ------ ---------------------------- -------------------
65535      MPLS   ES:GLOBAL                    Invalid            
102799     VXLAN  v2799                        EVPN               

RP/0/RSP0/CPU0:M52#show evpn evi inclusive-multicast 
Sun Mar 29 17:30:51.532 MEDST

VPN-ID     Encap  EtherTag   Originating IP                          
---------- ------ ---------- ----------------------------------------
102799     VXLAN  0          195.30.3.249                            
102799     VXLAN  0          195.30.3.251                            
102799     VXLAN  0          195.30.3.252                            

RP/0/RSP0/CPU0:M52#show evpn evi mac
VPN-ID     Encap  MAC address    IP address                               Nexthop                                 Label   
---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
65535      N/A    a80c.0d56.503d ::                                       Local                                   0       
102799     VXLAN  0050.569c.338e ::                                       195.30.3.251                            102799  
102799     VXLAN  0050.569c.338e 10.27.99.10                              195.30.3.251                            102799  
102799     VXLAN  3cfd.febd.7835 ::                                       TenGigE0/0/2/2.2799                     102799  
102799     VXLAN  3cfd.febd.7835 10.27.99.2                               TenGigE0/0/2/2.2799                     102799  

(... and more, everything I'd *expect* to be there)


and the VXLAN NVE VNI is "up":

RP/0/RSP0/CPU0:M52#sh nve vni
Sun Mar 29 17:30:27.100 MEDST
Interface  VNI          MCAST        VNI State        Mode
nve1       102799       N/A          Up               L2 Control


... so, generally speaking, this should be working now... alas, it 
doesn't.  


RP/0/RSP0/CPU0:M52#show l2vpn forw bridge-domain vlandb:v2799 mac loc 0/0/CPU0
Mac Address    Type    Learned from/Filtered on    LC learned Resync Age/Last Change Mapped to       
-------------- ------- --------------------------- ---------- ---------------------- --------------  
3cfd.febd.7835 dynamic Te0/0/2/2.2799              N/A        29 Mar 17:31:04        N/A             
9803.9b97.8f36 dynamic BD id: 0(nve1)              N/A        29 Mar 17:06:34        195.30.3.249    
a80c.0d56.503f routed  BD id: 0                    N/A        N/A                    N/A             


MAC addresses get not properly mapped to the NVE1 *unless* they are seen
from there first - so, the address above is something behind an Arista,
which happily does everything in a straightforward way.  Arista sends
packet, NVE1 decapsulates, and does mac-learning-from-VXLAN.  No other 
EVPN MAC addresses show up in the L2 forwarding table...

Said address shows up in "show evpn evi mac" in a "funky" way too:

RP/0/RSP0/CPU0:M52#show evpn evi mac
102799     VXLAN  9803.9b97.8f36 ::                                       Unknown(No Forwarder for XID)           102799  
102799     VXLAN  9803.9b97.8f36 ::                                       195.30.3.249                            102799  
102799     VXLAN  9803.9b97.8f36 10.27.99.202                             Unknown(No Forwarder for XID)           102799  



So it seems that some sort of disconnect still happens between L2 FIB
and EVPN MAC table.

I'm out of ideas how to debug this, or what further knobs to twiddle...


So - what I'd appreciate most, right now, is a working sample config
for "ASR9000 to ASR9000, basic L2, with EVPN and VXLAN transport".

No fancy L2/L3 stuff, no fancy route-target importing/exporting, no
stitching MPLS<->VXLAN, just barebones and *working*...

gert
-- 
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             gert at greenie.muc.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 630 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/cisco-nsp/attachments/20200329/55e2ffed/attachment.sig>