[c-nsp] 10Gb+ "Core" w/ Netflow

[Jeremy Bresley]

On 3/13/2015 11:14 PM, CiscoNSP List wrote:
>> if you want cisco kit and need more than 5x 1G ports, then you should take
>> a look at the asr9001 platform with 20x1G cards.  The cost per port is much
>> lower than trying to scale up asr1k boxes.
>
> Interesting - Thanks very much Nick.
>
> They do look very nice, and scale looks like exactly what we need....I know there are always "extra" costs with these, and having zero experience on the 9000 range, is it
>
> ASR-9001
>
>
> A9K-750W-AC x 2 (As I
> assume they don’t come with power supplies standard?)
>
>
> 20
> x 1 GE Modular Port Adapter
>
>
> 4 x 10 GE Modular Port Adapter (If we wanted more 10Gb)
> Licenses?  i.e. Are they similar to ME's where you need to purchase licenses to unlock MPLS, 10G ports etc?
> Anything else you need?
>

 From my relatively limited experience with them as a couple of Internet 
edge routers taking a couple of full feeds (in production for about a 
year and a half now)...

They're solid.  We've had no unplanned downtime on ours (moving 
datacenters and software upgrades are the only downtime so far) We're 
running XR 5.2.2 on them, waiting for at least 5.3.1 to upgrade to the 
next level.  From a pricing perspective, we needed at least 3 10G ports 
and a couple of 1G ports, and the ASR9001s were considerably less 
expensive than getting an ASR1004 with RP2 and 3 10G cards.  And you get 
the benefits of IOS-XR like sane BGP filtering and scalability that goes 
way beyond what we were ever able to get from our ASR1000s they 
replaced.  Boot time is long, plan on 10-15 minutes on a reboot, but BGP 
convergence is quick. Bouncing a full feed took 2-3 minutes for the 
first peer, and less than a minute for the second one.

The biggest thing to be careful of with licensing is the difference 
between the ASR9001 and the ASR9001S.  The ASR9001S only has 2 of the 4 
on-board ports enabled, and can only do 60Gbps instead of 120Gbps.  For 
the other licenses that are available, see Table 4:
http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r5-3/general/release/notes/reln_530a9k.html#reference_62DD9EA197C548DA9C702E711DC18534

The only licenses that show up in CCW for these besides the small 
handful in the table are the BNG, Lawful Intercept, and nV licenses.

You can upgrade from a 9001S to a normal 9001, but it's more cost 
effective to buy it with the higher performance if you are going to need 
it (we needed 3 10G ports day 1, so it didn't make sense to upgrade from 
the S model).  The on-board ports are SFP+, the add-in 10G cards are 
XFP.  If you have other ASR9Ks or GSRs, those optics might be reusable 
in the add-in cards.  If you're mainly dealing with enterprise optics, 
the XFP ones aren't common (except on the 10G card for the ASR1K which 
is a carryover from the GSRs).  The support on the ASR9Ks is a bit more 
limited than on the ASR1Ks and other enterprise class boxes.  The SFP+ 
ports support SR, LR, ER, and ZR as well as the BX40D-I BiDi one.  They 
do NOT support LRMs. Both the on-board and add-in cards support 
CWDM/DWDM optics.

The one pain point I still have with them is software upgrades.  If 
you're used to downloading an image from CCO, copying it to the router, 
changing a boot statement and typing reload, the XR way of doing things 
will be a MAJOR education.  To start, you download a tar file which 
contains all the different packages available for all 9Ks.  So there's 
files in there for nVs, ASR903s, etc that you don't need.  The bundle is 
about 1.5GB.  Once you strip out what you don't need in there (which 
requires a couple readthroughs of the release notes to see which .pie's 
include various features), you make a smaller copy of that tar file to 
download to the router.  Once you copy it, then you have to install and 
activate it.  Then once you reboot into the new version you commit it, 
then you can remove the old one.  The other major change from IOS is 
SMUs.  Think of these like hotfixes for routers, they're a fix for a 
specific bug which doesn't require a full software update to fix.  They 
will be marked as hitless (no impact), traffic loss (will impact traffic 
but not require a reload), and reboot required.

I'd be curious if anybody has used the CSM (Cisco Software Manager, boo 
to Cisco having two products with the same acronym for two different 
products) to push upgrades to 9Ks.  I've used it to grab the list of 
SMUs, but never gotten it to actually push the changes, but haven't 
tracked down if that's a firewall between me and the routers that is 
blocking it.

Overall, I've been pleased with the ASR9001s, they've been a very solid 
box that hasn't required us to do much with them other than let them sit 
and quietly move a lot of traffic for us.  If you have any specific 
questions about them, post to the list, these kinds of discussions are 
helpful to everybody.

Jeremy "TheBrez" Bresley
brez at brezworks.com