[j-nsp] Managing large route-filter-lists

[Brian Rak]

Another downside here is disk usage:

 > show ephemeral-configuration irrdata | display set | count
Count: 216414 lines

which ends up being:

-rw-r-----  1 root  config     164M May 21 19:11 juniper.eph_irrdata

I don't think that this would be any smaller in the normal config 
database, but that alone is currently using up around 16% of the 
available space in /var/rundb

On 5/21/2018 2:51 PM, Brian Rak wrote:
> We switched this over to using ephemeral configs: 
> https://www.juniper.net/documentation/en_US/junos/topics/concept/ephemeral-configuration-database-overview.html
>
> This seems to have dramatically reduced configuration time (at the 
> expense of being slightly less clear).
>
> It also has the bonus that our IRR filters no longer show up in the 
> main configuration, and 'show | compare' is back to being fast again.
>
> The downside seems to be that these can blow up the router somehow... 
> there's a big warning about it in the py-ez code: 
> https://github.com/Juniper/py-junos-eznc/blob/master/lib/jnpr/junos/utils/config.py#L750
>
>
> On 5/21/2018 12:20 PM, Saku Ytti wrote:
>> Hey Brian.
>>
>>
>>
>> Yeah you should just expect long commits.  We're closer to 1M than
>> 500k right now.
>>
>> One thing to consider, if you are able to pull off received but not
>> accepted routes off the router BMP or just 'show route ... hidden',
>> and then offline create intersection of IRR data and received prefixes
>> and only upload the intersection, in our case this would mean
>> configuration size reduction of some 90%.
>>
>>
>>
>>
>> On 21 May 2018 at 18:46, Brian Rak <brak at gameservers.com> wrote:
>>> What is the best way to manage large numbers of large 
>>> route-filter-lists
>>> effectively?
>>>
>>> We've been generating per-peer route-filter-lists based on IRR data, 
>>> and
>>> loading them via netconf.  However, I'm noticing that commits take 
>>> longer
>>> and longer, and that we're hitting weird junos errors around the
>>> configuration database.
>>>
>>> Right now, we have a 200k+ line config, which ends up being around 8mb.
>>> This is on a QFX10008, so I would expect it to have sufficient CPU 
>>> power to
>>> handle this.
>>>
>>> We're already aggregating prefix lists down to the smallest possible 
>>> size
>>> (with heavy use of upto), so I can't really think of any reductions 
>>> there.
>>>
>>> Should I just expect commits to take multiple minutes here? Even 
>>> with a 60s
>>> timeout, we end up failing to commit some of these updates.
>>> _______________________________________________
>>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
>>
>