[c-nsp] FIB scale on ASR9001

[Lukas Tribus]

Hello,

On Thu, 11 Nov 2021 at 10:22, Saku Ytti <saku at ytti.fi> wrote:
>
> On Thu, 11 Nov 2021 at 10:19, Mark Tinka <mark at tinka.africa> wrote:
>
> > Thanks for the clue, Saku. Hopefully someone here has the energy to ask
> > Cisco to update their documentation, to make this a recommendation. I
> > can't be asked :-).
>
> I think it should just be a config error. You're not just cucking
> yourself, but your peers and customers. So it shouldn't be a choice
> you can make.
>
> We can also imagine improvements
>   1) by default keep all RPKI rejects, and have 'soft-inbound never'
> optionally to turn that off

When enabling ROV on the ASR9001, I set everything to
"soft-reconfiguration inbound always", because I couldn't imagine how
ROV would work in a reliable way otherwise. I didn't think people
would complain about periodic route-refreshes, I just didn't trust
that it would work reliably on huge internet exchanges with many peers
and questionable BGP implementations on the other side. I wanted my
EBGP behaviour to remain *exactly* the same, just as before-ROV.

For ROV to work reliably it needs to be able to reconsider previously
rejected invalids, so I would not recommend disabling
soft-reconfiguartion inbound, instead I'd suggest to set it to always.

Of course it would be better to store ROV-rejects separately, instead
of rejecting them. Better yet: add a new drop statement in RPL, which
makes the route not eligible for path selection, but doesn't remove it
from memory - this way the operator has full flexibility. I can't
believe I'm saying this, but a famous CPE vendor from Latvia actually
supports this (routing filter action: reject vs discard) [1], maybe
the XR BGP team could steal some ideas from them ;)

I don't like to depend on optional or not commonly used BGP features
and code paths in other people's networks for changes of any kind on
my end.

Memory consumption was negligible for my use-case, iirc it was 200 -
300 MB for 30 - 40 peers, one of which was transit (so full table -
this was about a year ago). I believe we had this conversation before,
and you mentioned that this could be a DoS vector, which is true but
all the solutions that we can possibly suggest simply implement a
"selective" soft-reconfig inbound always" anyway, so the DoS vector
needs to be addressed in a different way, in my opinion.


cheers,
lukas

[1] https://wiki.mikrotik.com/wiki/Manual:Routing/Routing_filters