[j-nsp] VXLAN, BGP EVPN, remote VTEP
Vincent Bernat
bernat at luffy.cx
Thu May 10 15:10:32 EDT 2018
Hey!
Some update which may be useful to future readers.
I had to add back the "ingress-node-replication" to the
vlans. Otherwise, when a VNI has more than a handful VTEPs (30 in my
case), replication doesn't seem to be done at all.
I have also hit another surprising limitation: the outer MAC address for
remote VTEP was wrong (same MAC address for all VTEP, despite the
next-hop database on both FPCs being correct) when not doing replication
(it was correct for broadcast packets). I am using external loopback
cables with a separate instance for routing to workaround the
issue. This is not pretty, but it is OK for my use case.
I am using the QFX5100 in a virtual-chassis (cannot use BGP EVPN type-1
routes due to interoperability issues). Maybe this is the source of the
problems (as this is not an usual setup to run BGP EVPN on). I am
currently on 17.4R1-S2.2. rpd on 14.1X53-D46.7 was crashing when I added
more remote VTEPs. Didn't had any other difference from 15.1 to 18.1.
--
If you tell the truth you don't have to remember anything.
-- Mark Twain
――――――― Original Message ―――――――
From: Nitzan Tzelniker <nitzan.tzelniker at gmail.com>
Sent: 11 avril 2018 21:02 GMT
Subject: Re: [j-nsp] VXLAN, BGP EVPN, remote VTEP
To: Vincent Bernat
Cc: juniper-nsp at puck.nether.net
> Try to remove the "ingress-node-replication" from the vlans and add " set
> protocols evpn multicast-mode ingress replication "
> few months ago a TAC engineer told it to me
>
> "This knob will add all the remote VTEPs under the VNIs on local device
> even though the remote devices do not have these VNIs configured. This
> causes unnecessary flooding from local device to the remote."
>
> But to be honest I didn't check it yet
>
> Thanks
>
> Nitzan
>
> On Tue, Apr 10, 2018 at 12:42 PM Vincent Bernat <bernat at luffy.cx> wrote:
>
>> Hey!
>>
>> I am noticing an odd behaviour when using BGP EVPN on a vQFX: it assumes
>> that all VTEP have all the local VNIs. For example, on the local system,
>> I have VNI 654, 655 and 656. However, on the remote VTEP, I have only
>> 654 and 655. However, the local system still assumes VNI 656 should be
>> present:
>>
>> juniper at QFX1> show version
>> fpc0:
>> --------------------------------------------------------------------------
>> Hostname: QFX1
>> Model: vqfx-10000
>> Junos: 17.4R1.16 limited
>> JUNOS Base OS boot [17.4R1.16]
>> JUNOS Base OS Software Suite [17.4R1.16]
>> JUNOS Crypto Software Suite [17.4R1.16]
>> JUNOS Online Documentation [17.4R1.16]
>> JUNOS Kernel Software Suite [17.4R1.16]
>> JUNOS Packet Forwarding Engine Support (qfx-10-f) [17.4R1.16]
>> JUNOS Routing Software Suite [17.4R1.16]
>> JUNOS jsd [i386-17.4R1.16-jet-1]
>> JUNOS SDN Software Suite [17.4R1.16]
>> JUNOS Enterprise Software Suite [17.4R1.16]
>> JUNOS Web Management [17.4R1.16]
>> JUNOS py-base-i386 [17.4R1.16]
>> JUNOS py-extensions-i386 [17.4R1.16]
>>
>> {master:0}
>> juniper at QFX1> show ethernet-switching vxlan-tunnel-end-point remote
>> Logical System Name Id SVTEP-IP IFL L3-Idx
>> <default> 0 192.0.2.11 lo0.0 0
>> RVTEP-IP IFL-Idx NH-Id
>> 192.0.2.13 570 1749
>> VNID MC-Group-IP
>> 656 0.0.0.0
>> 655 0.0.0.0
>> 654 0.0.0.0
>>
>> {master:0}
>> juniper at QFX1> show route table default-switch.evpn.0
>>
>> default-switch.evpn.0: 11 destinations, 11 routes (11 active, 0 holddown,
>> 0 hidden)
>> + = Active Route, - = Last Active, * = Both
>>
>> 1:192.0.2.11:1::010101010101010101::0/192 AD/EVI
>> *[EVPN/170] 00:03:49
>> Indirect
>> 2:192.0.2.11:1::654::50:54:33:00:00:0f/304 MAC/IP
>> *[EVPN/170] 00:02:43
>> Indirect
>> 2:192.0.2.11:1::655::50:54:33:00:00:0f/304 MAC/IP
>> *[EVPN/170] 00:01:20
>> Indirect
>> 2:192.0.2.11:1::656::50:54:33:00:00:0f/304 MAC/IP
>> *[EVPN/170] 00:03:54
>> Indirect
>> 2:192.0.2.13:1::654::50:54:33:00:00:11/304 MAC/IP
>> *[BGP/170] 00:02:41, localpref 100, from 192.0.2.100
>> AS path: I, validation-state: unverified
>> > to 203.0.113.13 via xe-0/0/0.0
>> 2:192.0.2.13:1::655::50:54:33:00:00:11/304 MAC/IP
>> *[BGP/170] 00:02:04, localpref 100, from 192.0.2.100
>> AS path: I, validation-state: unverified
>> > to 203.0.113.13 via xe-0/0/0.0
>> 3:192.0.2.11:1::654::192.0.2.11/248 IM
>> *[EVPN/170] 00:04:08
>> Indirect
>> 3:192.0.2.11:1::655::192.0.2.11/248 IM
>> *[EVPN/170] 00:04:07
>> Indirect
>> 3:192.0.2.11:1::656::192.0.2.11/248 IM
>> *[EVPN/170] 00:04:07
>> Indirect
>> 3:192.0.2.13:1::654::192.0.2.13/248 IM
>> *[BGP/170] 00:03:55, localpref 100, from 192.0.2.100
>> AS path: I, validation-state: unverified
>> > to 203.0.113.13 via xe-0/0/0.0
>> 3:192.0.2.13:1::655::192.0.2.13/248 IM
>> *[BGP/170] 00:03:55, localpref 100, from 192.0.2.100
>> AS path: I, validation-state: unverified
>> > to 203.0.113.13 via xe-0/0/0.0
>>
>> The remote system, 192.0.2.13 (also a vQFX), never send anything that
>> would hint it can handle VNI 656. I have also verified by sniffing on
>> the wire that the remote system is in fact receiving VXLAN packets for
>> VNI 656:
>>
>> Frame 807: 106 bytes on wire (848 bits), 106 bytes captured (848 bits)
>> Ethernet II, Src: MS-NLB-PhysServer-05_86:71:7e:03 (02:05:86:71:7e:03),
>> Dst: MS-NLB-PhysServer-05_86:71:69:03 (02:05:86:71:69:03)
>> Internet Protocol Version 4, Src: 192.0.2.11, Dst: 192.0.2.13
>> User Datagram Protocol, Src Port: 3471, Dst Port: 4789
>> Virtual eXtensible Local Area Network
>> Flags: 0x0800, VXLAN Network ID (VNI)
>> Group Policy ID: 0
>> VXLAN Network Identifier (VNI): 656
>> Reserved: 0
>> Ethernet II, Src: 50:54:33:00:00:0f (50:54:33:00:00:0f), Dst: Broadcast
>> (ff:ff:ff:ff:ff:ff)
>> Address Resolution Protocol (request)
>>
>> My BGP EVPN configuration is pretty simple and I rely on auto RT:
>>
>> protocols {
>> bgp {
>> group evpn {
>> type internal;
>> multipath;
>> multihop;
>> family evpn signaling;
>> }
>> }
>> evpn {
>> encapsulation vxlan;
>> extended-vni-list all;
>> multicast-mode ingress-replication;
>> }
>> }
>>
>> switch-options {
>> vtep-source-interface lo0.0;
>> vrf-import EVPN-VRF-VXLAN;
>> vrf-target {
>> target:65000:1;
>> auto;
>> }
>> }
>> policy-options {
>> policy-statement EVPN-VRF-VXLAN {
>> then accept;
>> }
>> }
>> vlans {
>> vlan-client1-654 {
>> vlan-id 654;
>> vxlan {
>> vni 654;
>> ingress-node-replication;
>> }
>> }
>> vlan-client1-655 {
>> vlan-id 655;
>> vxlan {
>> vni 655;
>> ingress-node-replication;
>> }
>> }
>> vlan-client1-656 {
>> vlan-id 656;
>> vxlan {
>> vni 656;
>> ingress-node-replication;
>> }
>> }
>> }
>>
>> Do you observe the same "issue" on your setup? Is there a known way to
>> fix that?
>>
>> Thanks!
>> --
>> Soap and education are not as sudden as a massacre, but they are more
>> deadly in the long run.
>> -- Mark Twain
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
More information about the juniper-nsp
mailing list