[j-nsp] BGP output queue priorities between RIBs/NLRIs

[Jeffrey Haas]

> On Nov 9, 2020, at 12:19 PM, Rob Foehl <rwf at loonybin.net> wrote:
> 
> [External Email. Be cautious of content]
> 
> 
> On Mon, 27 Jul 2020, Rob Foehl wrote:
> 
>> Anyone know the secret to getting BGP output queue priorities working across
>> multiple NLRIs?
> [...]
>> I've tried about a dozen combinations of options, and cannot get any other
>> result with inet/evpn routes in the same session -- inet.0 routes always
>> arrive ahead of *.evpn.0.
> 
> Following up on this for posterity:
> 
> That last part turns out to not be entirely true.  It appears that the
> output queue priorities do work as intended, but route generation walks
> through the RIBs in a static order, always starting with inet.0 -- so
> maybe the last ~1000 inet routes wind up in the output queues at the same
> time as evpn routes.
> 
> This was declared to be working as designed, and the issue is now stuck in
> ER hell; best estimate for a real solution is "maybe next year".  Route
> refresh for EVPN routes triggering a full walk of all RIBs was also
> confirmed, but remains unexplained.

As the source of this particular bit of difficulty, a bit of explanation for why it simply wasn't done when the initial feature was authored.

An immense amount of work in the BGP code is built around the need to not have to keep full state on EVERYTHING.  We're already one of the most stateful BGP implementations on the planet.  Many times that helps us, sometimes it doesn't.  

But as a result of such designs, for certain kinds of large work it is necessary to have a consistent work list and build a simple iterator on that.  One of the more common patterns that is impacted by this is the walk of the various routing tables.  As noted, we start roughly at inet.0 and go forward based on internal table order.

The primary challenge for populating the route queues in user desired orders is to move that code out of the pattern that is used for quite a few other things.  While you may want your evpn routes to go first, you likely don't want route resolution which is using earlier tables to be negatively impacted.  Decoupling the iterators for the overlapping table impacts is challenging, at best.  Once we're able to achieve that, the user configuration becomes a small thing.

I don't recall seeing the question about the route refreshes, but I can offer a small bit of commentary: The CLI for our route refresh isn't as fine-grained as it could be.  The BGP extension for route refresh permits per afi/safi refreshing and honestly, we should expose that to the user.  I know I flagged this for PLM at one point in the past.

-- Jeff