[c-nsp] ASR-100x intro

Tue Feb 19 23:09:58 EST 2013

The shared RAM is kind of a misnomer.
On systems with an separate ESP and RP/RP2 there are two linux processors that control the cards.
On the combined systems there is only one processor, so it doesn't really share RAM there is just one processor.

The route limit on the RP/RP2 is kind of a misnomer as well.
All of the FIB space lives on the ESP portion of the board on shared systems and
Solely on the ESP in the separate cards.
The RAM for the QFP is separate from the control processor RAM.
The QFP has IRAM and DRAM, IRAM is instruction space, DRAM is data space.
In theory the IRAM can be used for FIB space (I am told that is bad).
Of course in theory the control processor can also do routing for punted packets.

How this thing handles running out of FIB DRAM (data ram) on the QFP is still an open question.

Having said all of that, the control processor ram does limit how many and how large of a BGP table
you can hold.  The FIB only holds one CEF adjacency for each route.  Each CEF adjacency can have up to
16 load sharing interfaces (in theory).

Final answer on how long 1M routes will work for is variable.
At least 18 months, possibly as long as 3 or 4 years.  Beyond that and there is going to be table pruning if
growth continues.

LR Mack McBride
Network Architect

-----Original Message-----
From: Charles Sprickman [mailto:spork at bway.net] 
Sent: Tuesday, February 19, 2013 9:03 PM
To: Lukasz Bromirski
Cc: Mack McBride; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] ASR-100x intro

On Feb 16, 2013, at 1:29 PM, Lukasz Bromirski wrote:

> 
> On Feb 16, 2013, at 10:34 AM, Charles Sprickman <spork at bway.net> wrote:
> 
>> On Feb 8, 2013, at 12:27 PM, Mack McBride wrote:
>> 
>>> One of the questions I haven't gotten a good answer to.
>>> The ESP actually has the hardware for the route table.
>>> The ESP20 and ESP40 handle 4 million routes.
>>> The others handle less (the 5G for examples handles 500k v4 or 125k v6).
>> 
>> And the ASR-1002-X with the "integrated" ESP-?? handles "1M IPv4 or 
>> 1M IPv6 routes" according to 
>> http://www.cisco.com/en/US/prod/collateral/routers/ps9343/data_sheet_
>> c78-450070.html
>> 
>> But the ESP-20 and ESP-40, which share many specs with the mystery embedded ESP in the 1002-X claims "4M IPv4 or 4M IPv6".
>> 
>> I don't know what the "OR" means, I can have 1M v4 OR 1M v6 but not some mix of the two?  The use of the word "or" there is strange.
> 
> It's either 1M for IPv4, 1M for IPv6 or some mix of it, depending on your requirements.

Thanks.  The wording is odd.

> 
>> And the RP side is clear as mud as well.  The RP2 also claims "4M IPv4 or 4M IPv6" with the 16GB RAM option, but then the 1002-X "embedded" RP2 is back at the "1M IPv4 or 1M IPv6" number even though it's possible to order the 1002-X with 16GB RAM.
> 
> For the RR role (when the entries are not downloaded to FIB) it'll be 
> around 22-24M depending on the config and the requirements with 16GB of RAM.

Specifically on the 1002-X or the RP2 when loaded in another chassis?

Seriously, it would be really helpful if Cisco could answer these questions - the rest of the line seems pretty clear, but the 1002-X due to the combined RP2/ESP seems to be a bit of an oddball.

Anyone on-list have an ASR-1002-X?

> 
>> Am I understanding the architecture of this correctly?  I mean, if my RP2 can hold 4M routes, which today would be what, about 9 full views, are ALL those routes shoved down to the forwarding plane, or just the "best" routes?  If so, why can't a lesser-spec'd ESP be limited to 1M routes even if the RP2 has 4M "possible" paths?
> 
> Only best entries are programmed into FIB (unless you enable BGP PIC, 
> or additional-paths extensions). That's for typical usage. For RR, you 
> use table-map to stop programming entries into FIB (typical for RR 
> scenario), and you can max-out the RAM with the entries.

With today's IPv4 table at around 450K, would anyone with multiple views like to share how many entries they end up with in the FIB?

In general, anyone feel like commenting on how long one could live with a FIB that maxes out at 1M routes?

> 
>>> What happens to the other routes?
>> Maybe we're asking the same question.  I hope so.
> 
> They fail to fit into available RAM, and process responsible for 
> 'getting them' will complain.
> 
>>> It seems they could get handled in software but the ESP is basically software anyway.
>>> So the situation is clearly opaque.
>> The 1002-X makes it even more opaque.  Someone said earlier in the thread that the 1002-X is essentially a fixed-config with an RP2 and an ESP-40.  But the specs don't match, at least on the number of routes.
> 
> They should, it's a question of what's supported today and what will be supported.
> For example, because of the way the ESP and RP are connected, and the 
> front-facing ports, it's rated at 36Gbit/s maximum - while it's still ESP40.

But with shared DRAM between the RP2 and ESP, which I have to guess is why we see a limit of 1M routes on the ESP and RP.

> 
> QuantumFlow is a matrix of CPUs - in this sense it's a "software", but 
> given the way those work, how the tasks are programmed and processed 
> on them it's treated as 'hardware forwarding'.

I assume the ASR series has not really been around long enough to get a feel for what features/specs may change based solely on software updates.

> 
>>> The MX80 from juniper for example has the same situation and is equally opaque.
>> We've had some very rough times getting similar information on the Juniper MX series.  There is some hint that the RP equivalent can have more routes than the FIB, but nothing definite and no hard numbers so far after putting in multiple requests with our Juniper salesperson.  We're also getting mixed answers about whether the integrated GigE ports on the MX are capable of hierarchical queueing when the chassis is fully "unlocked" as an MX-80 (this sounds incorrect based on what I've read, but some SE over there is claiming that's the case).
> 
> From the practical experience, they're not. But you can also check it 
> out in the official Juniper docs:
> 
> http://www.juniper.net/techpubs/en_US/release-independent/junos/topics
> /reference/general/mx80-features.html
> 
>> All I really want to know is if 3-5 years down the line, assuming these graphs (http://bgp.potaroo.net/v6/v6rpt.html) are still ramping up and we are looking at 4-5 full views of v4 and v6 will I be needing to retire the damn thing.  Traffic-wise, both boxes are fine for that long.  I also suspect (at least in the cisco case with the RP2) there is enough cpu power to last quite some time.
>> 
>> This is almost making going with a used 6500 bundle look appealing.
> 
> 
> If you have doubts, look at ASR9001. 

Or the 1004 with RP2 and ESP-20 or ESP-40 (which looks not too shabby used).

Thanks for the input,

Charles

> 
> -- 
> "There's no sense in being precise when |               Łukasz Bromirski
> you don't know what you're talking     |      jid:lbromirski at jabber.org
> about."               John von Neumann |    http://lukasz.bromirski.net