[c-nsp] High latency/jitter at times on a NPE-G1?

Rodney Dunn rodunn at cisco.com
Thu Jun 1 09:11:51 EDT 2006


If you want to prove it put a sniffer/analyzer on both sides
of the box and time sync the traces.

That is just about the ONLY way you can 100% prove the box
is introducing latency/jitter in the forwarding stream.

There will always be some latency/jitter but it should be
so small it's irrelevant. 

What order of magnitude are you talking about?

TAC will have an extremely hard time solving this problem unless
you can prove with a trace when and how much latency the box is
really introducing.

I'd be very amazed if it's the 72xx because it's software forwarding
and once the packet goes in the switching vector it can't be
held up other than if there is congestion for the most part.

Just curious:

can you post a 'sh c7200' output.

Rodney

On Thu, Jun 01, 2006 at 09:52:26AM -0300, Sean Watkins wrote:
> Hi,
> 
> I've got a couple of recently upgraded 7206VXRs NPE-400 -> NPE-G1s, that are
> experiencing at times, great amounts of jitter/latency.
> 
> Reading over the lists for the past couple of years, there are a variety of
> sources of this; we've gone ahead and opened a tac case 603658197 on it, but
> that seems to be going nowhere.
> 
> We've examined the switching paths, they seem good. CEF is confirmed
> running; we've begun turning off almost everything not explicity required
> and still from time to time we see great amounts of jitter/latency.
> 
> Any ideas where to start looking? The only thing I see is the BGP
> Router/Scanner process running every minute or so, but our latency spikes
> don't correspond to its invocation?
> 
> All interfaces are running at peak, at about 60% utilization.
>  
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 16%/14%; one minute: 16%; five minutes:
> 17%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   30      195780    134972       1450  0.15%  0.08%  0.05%   0 Net
> Background   
>   64     5181684     53996      95964  1.59%  1.08%  1.05%   0 IP Background
> 
>  159         856       194       4412  0.15%  0.03%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 16%/14%; one minute: 16%; five minutes:
> 17%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   30      195780    134974       1450  0.15%  0.08%  0.05%   0 Net
> Background   
>   64     5181684     53996      95964  1.59%  1.08%  1.05%   0 IP Background
> 
>  159         864       199       4341  0.15%  0.03%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 16%/14%; one minute: 16%; five minutes:
> 17%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   30      195780    134974       1450  0.15%  0.08%  0.05%   0 Net
> Background   
>   64     5181684     53996      95964  1.59%  1.08%  1.05%   0 IP Background
> 
>  159         868       204       4254  0.15%  0.03%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 16%/14%; one minute: 16%; five minutes:
> 17%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   30      195780    134974       1450  0.15%  0.08%  0.05%   0 Net
> Background   
>   64     5181684     53996      95964  1.59%  1.08%  1.05%   0 IP Background
> 
>  159         872       209       4172  0.15%  0.03%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 16%/14%; one minute: 16%; five minutes:
> 17%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   30      195780    134974       1450  0.15%  0.08%  0.05%   0 Net
> Background   
>   64     5181684     53996      95964  1.59%  1.08%  1.05%   0 IP Background
> 
>  159         880       214       4112  0.15%  0.03%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 42%/14%; one minute: 18%; five minutes:
> 18%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   52     1249292  18854470         66  0.23%  0.13%  0.11%   0 IP Input
> 
>  154      642284   3472343        184  0.15%  0.05%  0.05%   0 BGP Router
> 
>  157    11663628     84936     137322 27.43%  3.39%  2.36%   0 BGP Scanner
> 
>  159         884       219       4036  0.39%  0.06%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 42%/14%; one minute: 18%; five minutes:
> 18%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   52     1249292  18854493         66  0.23%  0.13%  0.11%   0 IP Input
> 
>  154      642284   3472345        184  0.15%  0.05%  0.05%   0 BGP Router
> 
>  157    11663628     84936     137322 27.43%  3.39%  2.36%   0 BGP Scanner
> 
>  159         888       224       3964  0.39%  0.06%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 42%/14%; one minute: 18%; five minutes:
> 18%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   52     1249296  18854504         66  0.23%  0.13%  0.11%   0 IP Input
> 
>  154      642284   3472345        184  0.15%  0.05%  0.05%   0 BGP Router
> 
>  157    11663628     84936     137322 27.43%  3.39%  2.36%   0 BGP Scanner
> 
>  159         896       229       3912  0.39%  0.06%  0.09%   3 Virtual Exec
> 
> router>show proc cpu | exclude 0.00
> CPU utilization for five seconds: 42%/14%; one minute: 18%; five minutes:
> 18%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
>   52     1249300  18854523         66  0.23%  0.13%  0.11%   0 IP Input
> 
>  154      642284   3472359        184  0.15%  0.05%  0.05%   0 BGP Router
> 
>  157    11663628     84936     137322 27.43%  3.39%  2.36%   0 BGP Scanner
> 
>  159         900       234       3846  0.39%  0.06%  0.09%   3 Virtual Exec
> 
> router>
> 
> This router is running a full table;
> 
> Sean
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


More information about the cisco-nsp mailing list