[j-nsp] VXLAN, BGP EVPN, remote VTEP

Vincent Bernat bernat at luffy.cx
Thu May 10 15:10:32 EDT 2018


Hey!

Some update which may be useful to future readers.

I had to add back the "ingress-node-replication" to the
vlans. Otherwise, when a VNI has more than a handful VTEPs (30 in my
case), replication doesn't seem to be done at all.

I have also hit another surprising limitation: the outer MAC address for
remote VTEP was wrong (same MAC address for all VTEP, despite the
next-hop database on both FPCs being correct) when not doing replication
(it was correct for broadcast packets). I am using external loopback
cables with a separate instance for routing to workaround the
issue. This is not pretty, but it is OK for my use case.

I am using the QFX5100 in a virtual-chassis (cannot use BGP EVPN type-1
routes due to interoperability issues). Maybe this is the source of the
problems (as this is not an usual setup to run BGP EVPN on). I am
currently on 17.4R1-S2.2. rpd on 14.1X53-D46.7 was crashing when I added
more remote VTEPs. Didn't had any other difference from 15.1 to 18.1.
-- 
If you tell the truth you don't have to remember anything.
		-- Mark Twain

 ――――――― Original Message ―――――――
 From: Nitzan Tzelniker <nitzan.tzelniker at gmail.com>
 Sent: 11 avril 2018 21:02 GMT
 Subject: Re: [j-nsp] VXLAN, BGP EVPN, remote VTEP
 To: Vincent Bernat
 Cc: juniper-nsp at puck.nether.net

> Try to remove the  "ingress-node-replication" from the vlans and add " set
> protocols evpn multicast-mode ingress replication "
> few months ago a TAC engineer told it to me
>
> "This knob will add all the remote VTEPs under the VNIs on local device
> even though the remote devices do not have these VNIs configured. This
> causes unnecessary flooding from local device to the remote."
>
> But to be honest I didn't check it yet
>
> Thanks
>
> Nitzan
>
> On Tue, Apr 10, 2018 at 12:42 PM Vincent Bernat <bernat at luffy.cx> wrote:
>
>> Hey!
>>
>> I am noticing an odd behaviour when using BGP EVPN on a vQFX: it assumes
>> that all VTEP have all the local VNIs. For example, on the local system,
>> I have VNI 654, 655 and 656. However, on the remote VTEP, I have only
>> 654 and 655. However, the local system still assumes VNI 656 should be
>> present:
>>
>> juniper at QFX1> show version
>> fpc0:
>> --------------------------------------------------------------------------
>> Hostname: QFX1
>> Model: vqfx-10000
>> Junos: 17.4R1.16 limited
>> JUNOS Base OS boot [17.4R1.16]
>> JUNOS Base OS Software Suite [17.4R1.16]
>> JUNOS Crypto Software Suite [17.4R1.16]
>> JUNOS Online Documentation [17.4R1.16]
>> JUNOS Kernel Software Suite [17.4R1.16]
>> JUNOS Packet Forwarding Engine Support (qfx-10-f) [17.4R1.16]
>> JUNOS Routing Software Suite [17.4R1.16]
>> JUNOS jsd [i386-17.4R1.16-jet-1]
>> JUNOS SDN Software Suite [17.4R1.16]
>> JUNOS Enterprise Software Suite [17.4R1.16]
>> JUNOS Web Management [17.4R1.16]
>> JUNOS py-base-i386 [17.4R1.16]
>> JUNOS py-extensions-i386 [17.4R1.16]
>>
>> {master:0}
>> juniper at QFX1> show ethernet-switching vxlan-tunnel-end-point remote
>> Logical System Name       Id  SVTEP-IP         IFL   L3-Idx
>> <default>                 0   192.0.2.11       lo0.0    0
>>  RVTEP-IP         IFL-Idx   NH-Id
>>  192.0.2.13       570       1749
>>     VNID          MC-Group-IP
>>     656           0.0.0.0
>>     655           0.0.0.0
>>     654           0.0.0.0
>>
>> {master:0}
>> juniper at QFX1> show route table default-switch.evpn.0
>>
>> default-switch.evpn.0: 11 destinations, 11 routes (11 active, 0 holddown,
>> 0 hidden)
>> + = Active Route, - = Last Active, * = Both
>>
>> 1:192.0.2.11:1::010101010101010101::0/192 AD/EVI
>>                    *[EVPN/170] 00:03:49
>>                       Indirect
>> 2:192.0.2.11:1::654::50:54:33:00:00:0f/304 MAC/IP
>>                    *[EVPN/170] 00:02:43
>>                       Indirect
>> 2:192.0.2.11:1::655::50:54:33:00:00:0f/304 MAC/IP
>>                    *[EVPN/170] 00:01:20
>>                       Indirect
>> 2:192.0.2.11:1::656::50:54:33:00:00:0f/304 MAC/IP
>>                    *[EVPN/170] 00:03:54
>>                       Indirect
>> 2:192.0.2.13:1::654::50:54:33:00:00:11/304 MAC/IP
>>                    *[BGP/170] 00:02:41, localpref 100, from 192.0.2.100
>>                       AS path: I, validation-state: unverified
>>                     > to 203.0.113.13 via xe-0/0/0.0
>> 2:192.0.2.13:1::655::50:54:33:00:00:11/304 MAC/IP
>>                    *[BGP/170] 00:02:04, localpref 100, from 192.0.2.100
>>                       AS path: I, validation-state: unverified
>>                     > to 203.0.113.13 via xe-0/0/0.0
>> 3:192.0.2.11:1::654::192.0.2.11/248 IM
>>                    *[EVPN/170] 00:04:08
>>                       Indirect
>> 3:192.0.2.11:1::655::192.0.2.11/248 IM
>>                    *[EVPN/170] 00:04:07
>>                       Indirect
>> 3:192.0.2.11:1::656::192.0.2.11/248 IM
>>                    *[EVPN/170] 00:04:07
>>                       Indirect
>> 3:192.0.2.13:1::654::192.0.2.13/248 IM
>>                    *[BGP/170] 00:03:55, localpref 100, from 192.0.2.100
>>                       AS path: I, validation-state: unverified
>>                     > to 203.0.113.13 via xe-0/0/0.0
>> 3:192.0.2.13:1::655::192.0.2.13/248 IM
>>                    *[BGP/170] 00:03:55, localpref 100, from 192.0.2.100
>>                       AS path: I, validation-state: unverified
>>                     > to 203.0.113.13 via xe-0/0/0.0
>>
>> The remote system, 192.0.2.13 (also a vQFX), never send anything that
>> would hint it can handle VNI 656. I have also verified by sniffing on
>> the wire that the remote system is in fact receiving VXLAN packets for
>> VNI 656:
>>
>> Frame 807: 106 bytes on wire (848 bits), 106 bytes captured (848 bits)
>> Ethernet II, Src: MS-NLB-PhysServer-05_86:71:7e:03 (02:05:86:71:7e:03),
>> Dst: MS-NLB-PhysServer-05_86:71:69:03 (02:05:86:71:69:03)
>> Internet Protocol Version 4, Src: 192.0.2.11, Dst: 192.0.2.13
>> User Datagram Protocol, Src Port: 3471, Dst Port: 4789
>> Virtual eXtensible Local Area Network
>>     Flags: 0x0800, VXLAN Network ID (VNI)
>>     Group Policy ID: 0
>>     VXLAN Network Identifier (VNI): 656
>>     Reserved: 0
>> Ethernet II, Src: 50:54:33:00:00:0f (50:54:33:00:00:0f), Dst: Broadcast
>> (ff:ff:ff:ff:ff:ff)
>> Address Resolution Protocol (request)
>>
>> My BGP EVPN configuration is pretty simple and I rely on auto RT:
>>
>> protocols {
>>     bgp {
>>         group evpn {
>>             type internal;
>>             multipath;
>>             multihop;
>>             family evpn signaling;
>>         }
>>     }
>>     evpn {
>>         encapsulation vxlan;
>>         extended-vni-list all;
>>         multicast-mode ingress-replication;
>>     }
>> }
>>
>> switch-options {
>>     vtep-source-interface lo0.0;
>>     vrf-import EVPN-VRF-VXLAN;
>>     vrf-target {
>>         target:65000:1;
>>         auto;
>>     }
>> }
>> policy-options {
>>     policy-statement EVPN-VRF-VXLAN {
>>         then accept;
>>     }
>> }
>> vlans {
>>     vlan-client1-654 {
>>         vlan-id 654;
>>         vxlan {
>>             vni 654;
>>             ingress-node-replication;
>>         }
>>     }
>>     vlan-client1-655 {
>>         vlan-id 655;
>>         vxlan {
>>             vni 655;
>>             ingress-node-replication;
>>         }
>>     }
>>     vlan-client1-656 {
>>         vlan-id 656;
>>         vxlan {
>>             vni 656;
>>             ingress-node-replication;
>>         }
>>     }
>> }
>>
>> Do you observe the same "issue" on your setup? Is there a known way to
>> fix that?
>>
>> Thanks!
>> --
>> Soap and education are not as sudden as a massacre, but they are more
>> deadly in the long run.
>>                 -- Mark Twain
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp


More information about the juniper-nsp mailing list