[c-nsp] cpu spike every minute

Holemans Wim wim.holemans at ua.ac.be
Tue Jan 11 07:51:19 EST 2011


We are seeing a cpu spike (and corresponding icmp respons latency) every minute on one of our 65XX.
It is a   6506-E with Sup32-8G running IOS version ipbasek9-vz.122-18.SXF6.
I checked al our mgmt processes (snmp requests, arp table copies,...) but found nothing that could lead to this behavior.
Normal icmp respons times (seen from our mgmt station) are a couple msec but every minute this rises till 1000 ms and more.

We also see a spike in cpu usage at the same time and I tried to determine what process uses all this cpu and got the following result :
Normal : cpu around 11%
ldus220#sh proc cpu | excl 0.0
CPU utilization for five seconds: 11%; one minute: 13%; five minutes: 14%
PID       5Sec    1Min     5Min Process
1         0.1%    2.7%     3.7% kernel
12312     7.6%    7.0%     7.4% ios-base
12329     1.8%    1.6%     1.2% tcp.proc
12330     0.6%    0.5%     0.5% udp.proc
12331     0.1%    0.2%     0.2% iprouting.iosproc
12332     0.1%    0.1%     0.1% cdp2.iosproc

Hit : cpu above 40%, caused by kernel process
ldus220#sh proc cpu | excl 0.0
CPU utilization for five seconds: 46%; one minute: 16%; five minutes: 14%
PID       5Sec    1Min     5Min Process
1        35.6%    6.3%     4.5% kernel
12312     7.1%    7.1%     7.4% ios-base
12329     1.0%    1.5%     1.2% tcp.proc
12330     0.3%    0.5%     0.5% udp.proc
12331     0.1%    0.2%     0.2% iprouting.iosproc
12332     0.1%    0.1%     0.1% cdp2.iosproc

Using cpu detail, I can see it is process id 17 that is hit but I don't have a clue what this process does. How can I find what this process does and if it is internal kernel housekeeping that is causing this or an external cause ?
ldus220#sh proc cpu detail | excl 0.0
CPU utilization for five seconds: 46%; one minute: 21%; five minutes: 18%
PID/TID   5Sec    1Min     5Min Process             Prio  STATE        CPU
1        33.9%    5.7%     4.3% kernel                                26d20h
       1  54.2%   79.5%    81.6% [idle thread]          0  Ready       1355d
     17  33.9%    5.7%     4.2%                       10  Running      6d07h
12312     7.3%    9.8%     9.1% ios-base                                106d

Any info or pointer to info would be appreciated.

Wim Holemans
Network Services
University of Antwerp



More information about the cisco-nsp mailing list