[c-nsp] Hierarchical FIB on Cisco 7600

[James Bensley]

On 14 May 2016 at 09:23, Gert Doering <gert at greenie.muc.de> wrote:
> Hi,
>
> On Sat, May 14, 2016 at 06:57:42AM +0000, Adam Vitkovsky wrote:
>> Thank you very much for posting this back to the list,
>> Wasn't aware the hierarchical lookup is most likely done via
>> recirculation on 7600 (what a nasty trick)
>
> Well, I find it an interesting option to give to users - if you need
> the faster convergence, but can live with half the lookup rate, it's
> nice to *have* the option.  (Our 6500s peak at well below 10Gbit/s
> throughput, so we could easily affort to half top pps if we wanted to
> do use PIC - but the increased memory usage is what keeps us from
> doing so)

Yes the memory issue is quite a problem. Both the memory and PPS rate
make it a no-go on the 7600 PEs we have, however even the 6500s in DCs
doing aggregation where the throughput is lower, still even here the
memory alone makes it a no-go.


On 14 May 2016 at 07:57, Adam Vitkovsky <Adam.Vitkovsky at gamma.co.uk> wrote:
> -does it affect only L3VPN lookups or lookups in global routing table as well please?
> I assume it does affect PIC Edge as well as Core for L3VPN prefixes.


For L3VPNs the PPS rate is halved due to the recirculation
(specifically, it requires a double lookup which is achieve through
re-circ). For vanilla IP to IP traffic there is no impact (TAC claim,
I haven't bothered to test because I'm interested in MPLS VPN traffic)
by using an internal fudge for IP to IP traffic using dual CEF
adjacencies that are load-balanced and can be substituted/replaced.

Not having the H-FIB enabled does effect PIC Edge convergence time
(since we are using PIC-Core on all other boxes, ME3x00, ASR920,
ASR9000s and PIC Edge, the 7600s are getting the backup advertisements
in BGP for PIC-Edge but the convergence is slower than other PE
models). We can pre-compute the best backup path and advertise it to a
7600 peer. The problem is that if say the primary PE-CE link dies and
we are receiving 10 prefixes from that CE next-hop address, we have
the backup paths via the secondary PE-CE link pre-computed but we have
to update the CEF tables for all 10 prefixes, without the prefix
indirection we still have to make multiple hardware programming
updates (but there is some benefit to at least having precompute the
backup paths and advertising them in MP-iBGP, not sure exactly what
though in terms of miliseconds/seconds). In the case of a PE-PE link
failing and we need to update 600k routes for the global routing
table, we’re fucked, the benefit is negligible.

I have a few notes on PIC Core and Edge on Cisco, including some 7600
specific stuff since it turned out to be a problem child platform for
PIC Core (compared to say ASR9000s, which have H-FIB enabled by
default and I not sure if you can turn it off):

https://null.53bits.co.uk/index.php?page=bgp-pic-edge#hfib

Cheers,
James.