[j-nsp] RR cluster

[Shane Amante]

On Feb 6, 2013, at 1:17 AM, Huan Pham <drie.huanpham at gmail.com> wrote:
> Aggree with Doug with one condition: RRs do not share cluster ID.
> 
> If the two RRs have the same Cluster ID, then one RR does not accept routes advertised by the other RR which it receives from its clients. It however DOES accept routes generated by the other RR itself.
> 
> As a best practice, keep the Cluster ID unique to maximise the redundancy. Although clients may receive redundant routing updates, no routing loop occurs, nor there is a loop of routing updates. 

You may wish to be careful offering this advice or, at the very least, encourage others to test this in a Lab environment before choosing one direction vs. the other.  Specifically, the downside with the recommendation above is a [potential] amplification of BGP UPDATEs from RRs down to RR-clients.  More specifically, what we have observed is that a widely-used vendor[1] views any difference in the _contents_ of the CLUSTER_LIST -- not the length of the CLUSTER_LIST -- as cause to transmit a new BGP UPDATE.  This is also compounded by BGP MRAI being set at 0 -- in other words, there's very little (read: no) statistical probability that a RR will coalesce the change into a single UPDATE message that is sent to RR-clients.  Where this is most readily apparent is in a topology similar to the following:

  +----- RR1 --- RR3 -----+
  |       |  \ /  |       |
PE-1      |   x   |      PE-2
  |       |  / \  |       |
  +----- RR2 --- RR4 -----+

The above diagram depicts iBGP sessions between devices.  At a minimum, when PE-1 generates an UPDATE and sends it to RR1 and RR2.  With Unique Cluster IDs, they will each send their own UPDATE to RR3 & RR4.  Keep in mind, that there is a slight timing difference on when the UPDATEs are xmit'ed and recv'd by RR3, and RR4, due to a variety of factors not least of which is propagation delay, CPU processing time, etc. on the PEs and RRs.  The end result is that, for example, RR3 views the two CLUSTER_LISTs as different, runs path selection each time *and* floods and UPDATE for each run to PE-2.  (The same behavior is observed on RR4).  So, the end result is PE-2 receives 4x the UPDATEs, instead of just 2x.

See slides 30 - 32 of here for more info: <http://www.nanog.org/meetings/nanog46/presentations/Monday/McPherson_BGP_Scala_N46.pptx>

In fairness, it's somewhat hard to fault the vendor for the aforementioned behavior.  After all, BGP is a distributed database and synchronization protocol and in a conservative implementation you'd want to lean to the side of ensuring any changes are synchronized throughout the whole network.  The downside is that you pay an increased tax in terms of flooding and processing many more BGP messages throughout the domain, which starts to get concerned when upper tiers of a RR hierarchy contain millions of paths.  

-shane

[1] Note, we did not observe this on JUNOS; but, we also did not *test* JUNOS for this behavior either, which is why I'm offering some caution.


> Huan
> 
> 
> On 06/02/2013, at 2:02 PM, Doug Hanks <dhanks at juniper.net> wrote:
> 
>> vanilla ibgp between the RRs would work
>> 
>> 
>> On 2/5/13 6:36 PM, "Ali Sumsam" <ali+junipernsp at eintellego.net> wrote:
>> 
>>> Hi All,
>>> I want to configure two RRs in my network.
>>> What should be the relation between two of them?
>>> I want them to send updates to each other and to the RR-Clients.
>>> 
>>> Regards,
>>> *Ali Sumsam CCIE*
>>> *Network Engineer - Level 3*
>>> eintellego Pty Ltd
>>> ali at eintellego.net ; www.eintellego.net
>>> 
>>> Phone: 1300 753 383 ; Fax: (+612) 8572 9954
>>> 
>>> Cell +61 (0)410 603 531
>>> 
>>> facebook.com/eintellego
>>> PO Box 7726, Baulkham Hills, NSW 1755 Australia
>>> 
>>> The Experts Who The Experts Call
>>> Juniper - Cisco ­ Brocade - IBM
>>> _______________________________________________
>>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>> 
>> 
>> 
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp