[j-nsp] [c-nsp] LACP between router VMs (James Bensley)

[Chris Burton]

Glad the conversation helped.  The note about the scripts was a general 
FYI in case there are others looking to use the scripts as there have 
been many conversations about the use of LLDP and LACP with the vMX in 
the past, both on this thread as well as other chats, forums, etc... 
just included for completeness.

LACP and LLDP (and the like) are link local only protocols (would need 
to dig through the specifications to find the exact rules), they should 
not be forwarded by the bridge, otherwise you could break the protocols 
themselves, in the case of LLDP it may be a simple as creating 
troubleshooting nightmare in terms of bad information, but in the case 
of LACP it could cause real harm, that is why they are only transmitted 
on the local link and not forwarded by the soft bridge by default (I do 
not know of any hardware bridges that allow you to disabled this 
restriction, if you know of any I would be interested.  Disabling this 
behavior on soft switch breaks that rule, that being said in the 
virtualized network world there are many things that have changed, so my 
statement was more of a general warning to not implement unless you 
understand and are willing to accept the possible consequences.

I have been running both OVS and Linux Bridge (with modified Kernel) in 
various labs, both large and small for quite some time without any 
issues, so my guess is it would be fine with the above caveats and ample 
precautions, though in production I use physical NICs and SR-IOV based 
NICs so I have not attempted to implement this bypass in production and 
do not know all of the consequences for production deployment.

Out of curiosity what is your use case that you need to use LACP to 
communicate with VMs?

Cheers,

-C


On 11/12/2017 05:36 AM, adamv0025 at netconsultings.com wrote:
>> Chris Burton
>> Sent: Saturday, November 11, 2017 11:50 PM
>>
>> This has been discussed a few times on this list and other forums. That being
>> said, if you are looking to do this with Linux bridge you will need to modify
>> the net/br_private.h header library to remove the mask restriction placed on
>> using group_fwd_mask setting in /sys/class/net and recompile the kernel to
>> allow it forward LACP BPDU's.
>>
> My question regarding the Linux bridge was out of curiosity and to my surprise spawned a very fruitful discussion which I learned a lot from.
> So thank you everyone.
> I'm actually using OVS and the "sudo ovs-vsctl set bridge br-1 other-config:forward-bpdu=true" did the trick.
>
>> If you are looking to use Openvswitch you can do as Sergio mentioned and
>> set the OVS bridge to allow forwarding of LACP BPDU's after you manually
>> add each interface, but if you want to use the default vmx.sh script to bind
>> the interfaces with OVS you will need to modify the config/vmx-
>> junosdev.conf file and add in "use_ovs_transport: True" as a key, and
>> depending on what version of OS you are running (I presume Ubuntu 16.04)
>> you may also need to modify the sub-script that vmx.sh --bind-dev calls to
>> prevent it from checking for the "openvswitch-datapath-source" package. I
>> have only tested OVS 2.6 (Ubuntu cloud repo) with version 16.1 of VMX, but
>> unless the sub-scripts are different for earlier or later versions it should work
>> (but it is unlikely to be supported since I do not see mention of using OVS in
>> any of the official Juniper documents, and they likely will not support this
>> even if OVS is supported as it breaks standards, same goes for LB mod's).
>>
> I'm not using any juniper scripts at all, I need complete control about all aspects of the setup and there are VMs of different vendors so I'm using custom scripts and procedures to create XMLs, define and start VMs and to interconnect them in OVS.
>
>
>> It should be understood though that you are breaking standards
>> compatibility and could create massive problems on the network, so running
>> this type of configuration outside of a lab environment should fall under the
>> do not do it category. If you are going to do this production you should use
>> physical NICs and SR-IOV with VF’s which eliminates the issue as was
>> mentioned by James.
>>
> As I mentioned earlier, I don't need to talk LACP to PF just LACP between VFs.
> Can you elaborate on the "breaking standards compatibility" please?
> In my setup I just need hundreds of simple p2p links (basically emulating fibres) between VMs so I need the OVS (or other bridge) not to intervene.
>
>
> adam
>
>
>
>