[c-nsp] help with the correct choice of a cisco router

[Nick Hilliard]

On 05/03/2012 12:13, Riccardo Giuntoli wrote:
> Do you have any particular recommendation and/or link to some tests that
> you consider interesting? 

Unfortunately not.

Just in case it isn't clear from my previous email, the test results they
provide should be considered accurate in the context of the methodology
they used.  I'm not suggesting for a moment that the results aren't
repeatable and consequently an accurate portrayal of what they're testing.
 The problem is that vendors often suggest subtle configuration tweaks
which make a dramatic difference to the results.  E.g. two that come to
mind are the EANTC c6500 10G test:

> http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd800c958a.pdf

In this case, the choice of port used by the tester turned out to be
critical due to ASIC layout.

Also, the Miercom N5K vs Arista 7124:

> http://www.cisco.com/web/strategy/docs/finance/miercom_n5k_testrpt.pdf

... where it turned out that the entire test was designed in favour of the
N5K, even down to the exact choice of packet size used in the testing
configuration.

What's important to understand about both of these tests are that they are
completely repeatable and consequently completely accurate in terms of
providing a result about the particular methodology used.  However, you
cannot generalise the test results and say that in general a sup720 +
ws-x6704-10ge handles 10G at line rate or that in general an Arista 7124
drops packets all over the floor in situations where an N5k wouldn't.  This
is because the tests are loaded.

I haven't looked at the miercom ISR G2 tests and can't comment on them
other than to say that I'm sure they're repeatable.  But as with all vendor
tests, I remain sceptical about whether they provide a good overall view of
the ISR G2 products.

Nick