[c-nsp] DS-3 -> OC-3 upgrade

[Justin M. Streiner]

On Mon, 16 May 2005, Rick Ernst wrote:

> I've noticed with the DS-3s that CPU utilization is roughly 40% at 40Mbs of
> in/out traffic (80Mbs aggregate).  I've also seen that over 40% that
> latency increases through the router.  There is a small ACL (about 30
> entries), plus RPF checks on inbound traffic.

Depending on how you look at the CPU utilization, the real utilization may 
be quite a bit higher.  Doing the "5 minute average CPU utilization" 
polling via SNMP will get you just that, an average.  There are brief 
spikes that can go up much higher, but because those spikes are often very 
short in nature, they don't register much of a difference in the average 
utilization.

If your router is running code that supports it, take a look at the output 
of a "show proc cpu hist".

> Based upon this empirical evidence, I'm hesitant to go OC-3, or even 100Mbs
> through them.  Do my numbers seem way off, or am I missing something?
> Would upgrading to NPE-400 "surely be enough", or should I look at other
> ways of bringing in higher-speed links?

They could be accurate.  Unfortunately, the best answer I could give what 
what I read is "it depends" on packet sizes, whether CEF is being used or 
not (since you're doing RPF checks, I'll assume yes) what else the router 
is doing at the time, etc...  If you see a brief spike in latency about 
once every 60 seconds that lasts for a few seconds then goes away, then 
take a look at how much CPU your BGP Scanner and BGP Router processes are 
taking up.  Handling multiple full BGP feeds can also chew up a fair 
amount of CPU.

Also, keep in mind that if you're measuring latency by doing pings and/or 
traceroutes, those are not necessarily the best measures of router 
latency.  Traffic _to_ a router like this is normally process switched, as 
traffic _through_ such a router could be CEF switched.  Process switching 
is much more CPU intensive.  If you have a historical baseline of to-router
latency, even if it's done using pings and/or traceroutes, those 
measurements could provide a somewhat useful precursor to router latency 
problems to come - the proverbial canary in a mine shaft...

2 years ago, I upgraded one of the 720VXRs where I used to work from an 
NPE-300 to an NPE-G1, and I was not all that impressed with the 
difference.  CPU usage was quite a bit lower during normal times, but 
since at its heart it's still doing software-based forwarding, a 
relatively low packet-per-second DoS attack would still bring the router 
to its knees in short order.

If you're looking to get close to wire-speed on those OC3 interfaces, 
assuming a 'standard' mix of network traffic in terms of packet sizes, you 
may want to look at something like the smaller 7600 platforms, or a 
Juniper M5/M7i/M10i....

jms