[c-nsp] ASR-100x intro

[Beck, Andre]

Hi Charles, all,

On Wed, Apr 10, 2013 at 03:44:34PM -0400, Charles Sprickman wrote:
> On Apr 10, 2013, at 3:53 AM, Beck, Andre wrote:
> > We've prodded Cisco via several routes about that problem, as I couldn't
> > believe the ASR1002-X would really max out at 1M FIB despite all the
> > similarities to the ESP40+RP2 combo. Turns out there was a Cisco internal
> > "Unidimensional Scaling Sheet" with way better numbers for the case of
> > equipping the box with 16GB RAM.
> 
> I think I posted that here as well, but I never found out if it was OK to officially quote the numbers as the document I saw had "CONFIDENTIAL" all over it.

That's the problem - I couldn't base a purchase decision on a rumour, so
we kept prodding through our SEs and the partner helpline. I wanted to
at least have an Ok to cite them here (and everywhere else), but now
that they even made them into the data sheet on CCO, I feel achievement ;)

> > From Table 8:
> > 
> > 500,000 IPv4 or 500,000 IPv6 routes with 4GB Memory
> > 1,000,000 IPv4 or 1,000,000 IPv6 routes with 8GB Memory
> > 3,500,000 IPv4 or 3,000,000 IPv6 routes with 16GB Memory
> 
> I saw that, but note that if you look at the RP2 datasheet, they still maintain the limit is 1M:
> 
> http://www.cisco.com/en/US/prod/collateral/routers/ps9343/data_sheet_c78-441072.html
> 
> Performance
> ? 1,000,000 IPv4 or 1,000,000 IPv6 routes

I think that's easy to explain: The RP data sheet wasn't touched. The
ESP data sheet was also just "hot fixed", note the table 8 still has 8GB
in the headline and the alternate values for routes were added to this
table anyway. Better here than nowhere public, I think.
 
> That's even more strange as the RP generally can carry more routes than the ESP - other datasheets claim that it can do at least 4M routes or more when acting as a route reflector.

The sharing of memory between the almost-but-not-quiet-unlike-RP2&ESP40
in the 1002-X will have some costs, it's probably what reduces the numbers
a little to 3M when compared to the bigger boxes with dedicated memory.
I've made sure that the 3M number is a true FIB limit, not just a BGP
route reflector number (which would be achieved by filtering the routes
away from RIB and thus FIB installation). I was assured they make it to
the FIB.

Anyway, if the 1002-X ESP data sheet claimes 3M+ routes in the FIB, they
must have a way to appear there, and the only way is through the RIB on
the RP. Thus the RP must have the capacity to deal with them. It all makes
sense to me, at least on paper.

When my 1002-X box with 16GB hits the lab, before anything else, I'll set
it up to push BGP tables into it using bgpsimple (I also tried exabgp but
it gets slow as molasses as soon as you try to push some 100k prefixes)
and see where it goes through the roof. I've hacked up a little bash
script to generate (IPv4) BGP tables of almost arbitrary sizes with a
somewhat natural-looking distribution (of shorter prefixes vs. longer
ones, and of shorter AS-paths vs. longer ones). The AS-path fanout is
not truly natural-looking (it's just 3 or 4 at every hop), but good
enough for this purpose I think. The Maennle paper deals with way better
data, but I couldn't find their RTG code anywhere, so some random made-up
stuff had to do. It craps out a NPE-400 (15.2(4)S) at 300k prefixes (first
the CEF table implodes[1], at 330k the BGP process starts to starve on RAM
and soon after, it fails to install routes in the RIB). Couldn't test
a NPE-G2 yet (doesn't work in dynamips and lab hardware is on the way),
let's see what the 1002-X is going to take.

[1] My prefix distribution is still unnatural compared to Maennle,
    which probably overtaxes the 8-8-8-8 mtrie compared to what
    happens in the real DFZ today. But I consider that a feature ;)

> Based on some quotes, it's something of a bargain as well.  If you step up to the ASR-1004 with an RP2 and an ESP that can do 4M routes, the cost is about double.  Perhaps that's one reason for this inaccuracy. :)

If it's "that box - or no sale at all" things look different, though. The
big chassis with ESP40+RP2 is completely oversized for the small ISP with
no need for SPAs at all - or just a feeling that maybe, in a year or three,
there might be some use for 10G but as of now, 6* 1G is enough for me - just
don't give me a hard limit on FIB size. Of course, when all the 1M FIB
devices out there will hit the ceiling, there is going to be a plateau
in the DFZ growth for a while. But I expect v4+v6 combined prefix size
to pass through the 1M mark within the next five years anyway, and
purchasing something with a life expectancy of max 5y is a no-no (our
7200s serve(d) us way longer, even though it required a constant influx
of new NPEs).
 
> Also finally heard back from Juniper, and they are claiming a 4M RIB and 1M FIB in the MX-80 series, so I suppose that puts them out of the running.  They offered that an MX240/480/960 scales "much higher", but didn't even hint at a list price.  I suspect it's well out of our price range (and out of anyone's price range looking to replace a G2 or augment some 6500 gear that can't scale beyond 1M routes in the FIB).

Interesting. Seems most vendors don't realize how such a deterrent reduces
their sales of medium sized equipment and there is no way in the world the
ISPs in question would be able to buy the forklift upgrade to really big
gear. So either you ignore that market and make the world a little worse
(by indirectly helping the shift to giant supercarrier monopoly and the
death of the classic small ISP) - or you build products that fit in there.
 
> Glad to hear someone else confirming the 1002-X numbers though?

Ideally through lab experiment ;)

Thanks & HTH,
Andre.
-- 
                    Cool .signatures are so 90s...

-> Andre Beck    +++ ABP-RIPE +++      IBH IT-Service GmbH, Dresden <-