[c-nsp] NCS-5501 - EVPN L2VPN BVI mac-address weirdness

[Drikus Brits]

Hi folks,

Wondering if anybody came across an issue with NCS-5501 models doing
EVPN Active/Active with Anycast IRB gateways. We have a multitude of
BVI's, with L2VPN and EVPN configured between a pair ov NCSs and using
a ESI for Bundle-Ether40 connected to downstream Nexus's.

The config i'm using below:

<config>
interface Bundle-Ether40.1017 l2transport
 description CUSTOMER: TEST-BVI-MAC-ISSUE - 001
 encapsulation dot1q 1017
 rewrite ingress tag pop 1 symmetric


evpn
 evi 11017
  description CUSTOMER: TEST-BVI-MAC-ISSUE - 001
  control-word-disable
  advertise-mac


l2vpn
 bridge group GRP-000111-00
  bridge-domain BD--000111-00
   description CUSTOMER: TEST-BVI-MAC-ISSUE - 001
   interface Bundle-Ether40.1017
   routed interface BVI1017
   evi 11017


interface BVI101017
 description CUSTOMER: TEST-BVI-MAC-ISSUE - 001
 vrf TEST-BVI
 ipv4 address 172.31.175.1 255.255.255.0
 mac-address 0000.ff00.ffaa
</config>

The behaviour we're seeing and causing us some grief, is that whilst
on our Nexus's we see the mac address of 0000.ff00.ffaa in vlan 1017,
we're also seeing a second generate mac address 9ce1.7685.2000 closely
resembling that of NCS #1's Bundle-Ether40's BIA of 9ce1.7685.24df.
This behaviour seems to be causing fluctuations with reachability to
some applications, so far most notable are citrix orientated apps.
>From our packet captures, the netscalers fluctuates between knowing
the NCS's as the gateway of 172.31.175.1 with 0000.ff00.ffaa
mac-address and then randomly gets the mac-address as 9ce1.7685.2000
instead. Regardless which mac-address we configure on the BVI, the
fluctuations keeps happening.

When we change the mac address to 9ce1.7685.2000 on the BVI of both
NCSs, the issue appears to resolve itself and works, except that a
failure introduced on the 1st NCS breaks the connectivity and we see
the same happening with the 2nd NCS's BE40 of 7c31.0e21.5cdf and some
mac address of 7c31.0e21.5000.....fluctuating between that mac address
and 9ce1.7685.2000

Our Cisco tac guy seems to be scratching his head as well.. Any
thoughts or experiences like this?

Cheers,