[c-nsp] Troubleshooting high processing on router

Rodney Dunn rodunn at cisco.com
Sun Aug 7 09:01:28 EDT 2005


clear count
sh int stat

Figure out which interface is process switching
traffic at a higher rate.

Get:
sh ip int
sh cef int
sh run int

sh buffers input-interface <intfname> packet 

sh ip ro <dstip> and then look at the egress interface
that packet would go out of.


I don't know how the RPM works but most likely it's
punting due to a feature not supported in the CEF
path. One common one with VA's and PPP is compression that
gets negotiated with a user.

Rodney




On Sun, Aug 07, 2005 at 12:59:08PM +0300, Joost greene wrote:
> Hi,
> 
> I am trying to troubleshoot why a router has high cpu usage, from
> output shown below, it appears that IP Input is the beast,
> 
> I know that this box is pushing alot of different types of
> traffic(MPLS, BGP,...,almost 500 Virtual-Access/Virtual-Template and
> SVIs)
> 
>  Its an RPM (equivalent to 7204 router)
> 
> My question is how do i troubleshoot high IP input, i looked at the
> below output of 'sh ip traffic, cant make sense of the numbers, i am
> trying to look at number of broadcasts compared to normal packets
> numbers but i wouldnt know if its much or normal.
> 
> Cisco IOS Software, RPM Software (RPM-JS-M), Version 12.3(11)T5,
> RELEASE SOFTWARE (fc1)
> Technical Support: http://www.cisco.com/techsupport
> Copyright (c) 1986-2005 by Cisco Systems, Inc.
> Compiled Sat 02-Apr-05 14:57 by yiyan
> 
> ROM: System Bootstrap, Version 12.1(20001003:080040)
> [swtools-rommon400 102], DEVELOPMENT SOFTWARE
> BOOTLDR: RPM Software (RPM-BOOT-M), Version 12.2(4)T,  RELEASE SOFTWARE (fc1)
> 
>  uptime is 2 weeks, 2 days, 7 hours, 17 minutes
> System returned to ROM by reload at 04:33:28 CAI Fri Jul 22 2005
> System restarted at 04:34:26 CAI Fri Jul 22 2005
> Running default software
> Last reload reason: Reload command
> 
> 
> Cisco RPM-PR (NPE400) processor with 491520K/32768K bytes of memory.
> R7000 CPU at 350MHz, Implementation 39, Rev 3.2, 256KB L2, 4096KB L3 Cache
> Last reset from s/w peripheral
> 2 FastEthernet interfaces
> 1 ATM interface
> 125K bytes of NVRAM.
> 
> 
> 
> 
> CPU utilization for five seconds: 44%/30%; one minute: 45%; five minutes: 43%
>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
>   57   2226091361581548486        140 13.83% 13.00% 12.34%   0 IP Input
>   32     3442980   1431013       2405  0.15%  0.28%  0.25%   0 Per-Second Jobs
>   45      619476   2259111        274  0.07%  0.04%  0.02%   0 CEF process
>   40      366308   6764173         54  0.07%  0.01%  0.00%   0 ATM OAM Input
>  164     1044272   2393821        436  0.07%  0.06%  0.07%   0 RIP Send
>   38       99488    145992        681  0.07%  0.00%  0.00%   0 ATM Periodic
>   50     1026996    288610       3558  0.07%  0.06%  0.07%   0 Compute load avg
>  105      598792   8684878         68  0.07%  0.02%  0.01%   0 Tag Input
>    8           0         2          0  0.00%  0.00%  0.00%   0 ATM Idle Timer
>    9          40     23473          1  0.00%  0.00%  0.00%   0 IPC Dynamic Cach
>    7           0         2          0  0.00%  0.00%  0.00%   0 Serial Backgroun
>   12        4548   1406529          3  0.00%  0.00%  0.00%   0 IPC Deferred Por
>    6           0         2          0  0.00%  0.00%  0.00%   0 Timers
>   10       29520  43553511          0  0.00%  0.00%  0.00%   0 RADIUS
>   11        4436   1406526          3  0.00%  0.00%  0.00%   0 IPC Periodic Tim
>    5        1352      3142        430  0.00%  0.00%  0.00%   0 Pool Manager
>   17           0         1          0  0.00%  0.00%  0.00%   0 Crash writer
>   18       14304   1406522         10  0.00%  0.00%  0.00%   0 EnvMon
>   19           0         1          0  0.00%  0.00%  0.00%   0 OIR Handler
>   20         700     28788         24  0.00%  0.00%  0.00%   0 ARP Input
>   13         616     10194         60  0.00%  0.00%  0.00%   0 IPC Seat Manager
>   14           0         1          0  0.00%  0.00%  0.00%   0 IPC BackPressure
>    2         776    281645          2  0.00%  0.00%  0.00%   0 Load Meter
>   24           0         1          0  0.00%  0.00%  0.00%   0 SERIAL A'detect
>   15           4      4348          0  0.00%  0.00%  0.00%   0 AAA high-capacit
>   26           0         2          0  0.00%  0.00%  0.00%   0 Dialer event
>   16           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager
>   28           4         2       2000  0.00%  0.00%  0.00%   0 Critical Bkgnd
>    3           0        13          0  0.00%  0.00%  0.00%   0 OSPF Hello 99
>   30         864      3791        227  0.00%  0.00%  0.00%   0 Logger
>   31       15096   1406510         10  0.00%  0.00%  0.00%   0 TTY Background
>   21          40       927         43  0.00%  0.00%  0.00%   0 DDR Timers
>   33       11520   1403194          8  0.00%  0.00%  0.00%   0 RPM Hardware Dia
>   34        7244   1403194          5  0.00%  0.00%  0.00%   0 RPM Software Dia
>   35           0         1          0  0.00%  0.00%  0.00%   0 CSP Timer
>   36           0         1          0  0.00%  0.00%  0.00%   0 RPM TEST
>   37           0         1          0  0.00%  0.00%  0.00%   0 Coproc Event Pro
>    4     1987000    168132      11818  0.00%  0.09%  0.07%   0 Check heaps
>   39        6628     41483        159  0.00%  0.00%  0.00%   0 ATM ARP INPUT
>   22           0         2          0  0.00%  0.00%  0.00%   0 Entity MIB API
>   23        1056   1396612          0  0.00%  0.00%  0.00%   0 MPLS HC Counter
>   42        3192   1407251          2  0.00%  0.00%  0.00%   0 ATM PA Helper
> 
> RPM1-Maadi-I#sh ip traffic
> IP statistics:
>   Rcvd:  686065560 total, 10112838 local destination
>          10750 format errors, 0 checksum errors, 1271416 bad hop count
>          0 unknown protocol, 2330835 not a gateway
>          0 security failures, 0 bad options, 827 with options
>   Opts:  0 end, 0 nop, 0 basic security, 0 loose source route
>          0 timestamp, 0 extended security, 0 record route
>          0 stream ID, 0 strict source route, 827 alert, 0 cipso, 0 ump
>          0 other
>   Frags: 13710 reassembled, 102 timeouts, 0 couldn't reassemble
>          13554 fragmented, 5515 couldn't fragment
>   Bcast: 458932 received, 327605 sent
>   Mcast: 2633985 received, 15778165 sent
>   Sent:  14984677 generated, 664821480 forwarded
>   Drop:  3118222 encapsulation failed, 899 unresolved, 1 no adjacency
>          2185541 no route, 0 unicast RPF, 0 forced drop
>          0 options denied
>   Drop:  0 packets with source IP address zero
>   Drop:  0 packets with internal loop back IP address
> 
> ICMP statistics:
>   Rcvd: 4 format errors, 1 checksum errors, 0 redirects, 145 unreachable
>         699239 echo, 52036 echo reply, 0 mask requests, 0 mask replies, 0 quench
>         0 parameter, 0 timestamp, 0 info request, 0 other
>         0 irdp solicitations, 0 irdp advertisements
>   Sent: 21 redirects, 796819 unreachable, 53607 echo, 699239 echo reply
>         0 mask requests, 0 mask replies, 0 quench, 0 timestamp
>         0 info reply, 3516273 time exceeded, 0 parameter problem
>         0 irdp solicitations, 0 irdp advertisements
> 
> UDP statistics:
>   Rcvd: 2837629 total, 3 checksum errors, 2361837 no port
>   Sent: 11300834 total, 0 forwarded broadcasts
> 
> TCP statistics:
>   Rcvd: 6191695 total, 2030 checksum errors, 206 no port
>   Sent: 564078 total
> 
> BGP statistics:
>   Rcvd: 2030894 total, 34 opens, 0 notifications, 1593141 updates
>         437621 keepalives, 98 route-refresh, 0 unrecognized
>   Sent: 437768 total, 34 opens, 15 notifications, 0 updates
>         437668 keepalives, 51 route-refresh
> 
> OSPF statistics:
>   Rcvd: 328169 total, 0 checksum errors
>         286801 hello, 17 database desc, 1 link state req
>         39777 link state updates, 1573 link state acks
> 
>   Sent: 1858925 total
>         1974490 hello, 15 database desc, 6 link state req
>         3649 link state updates, 21436 link state acks
> 
> IP-EIGRP statistics:
>   Rcvd: 0 total
>   Sent: 0 total
> 
> PIMv2 statistics: Sent/Received
>   Total: 0/0, 0 checksum errors, 0 format errors
>   Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,  Hellos: 0/0
>   Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0
>   Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0
>   Queue drops: 0
>   State-Refresh: 0/0
> 
> IGMP statistics: Sent/Received
>   Total: 0/0, Format errors: 0/0, Checksum errors: 0/0
>   Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0
>   DVMRP: 0/0, PIM: 0/0
>   Queue drops: 0
> 
> ARP statistics:
>   Rcvd: 1529 requests, 385 replies, 0 reverse, 0 other
>   Sent: 8457 requests, 7841 replies (144 proxy), 0 reverse
> 
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


More information about the cisco-nsp mailing list