[c-nsp] cpu spike every minute

[Matlock, Kenneth L]

We had the same thing with the same version of Modular code. (Which is the reason why we no longer run Modular)
 
I'll have to dig up the bug ID, but basically it's a small memory leak that's triggered by 'show run' or 'write term'.
 
If you have dual supervisors then swap to the other one and the problem will go away for a while.
 
Do you use Rancid or the like to archive configs? That's what was triggering it on our side.
 
Either way, Cisco removed Safe Harbor certification from that code release, so I'd recomend upgrading that code anyway.
 
When I get into the office I'll find the particular bug ID.
 
Ken Matlock
matlockk at exempla.org
Network Analyst
Exempla Healthcare

________________________________

From: cisco-nsp-bounces at puck.nether.net on behalf of Holemans Wim
Sent: Tue 1/11/2011 5:51 AM
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] cpu spike every minute



We are seeing a cpu spike (and corresponding icmp respons latency) every minute on one of our 65XX.
It is a   6506-E with Sup32-8G running IOS version ipbasek9-vz.122-18.SXF6.
I checked al our mgmt processes (snmp requests, arp table copies,...) but found nothing that could lead to this behavior.
Normal icmp respons times (seen from our mgmt station) are a couple msec but every minute this rises till 1000 ms and more.

We also see a spike in cpu usage at the same time and I tried to determine what process uses all this cpu and got the following result :
Normal : cpu around 11%
ldus220#sh proc cpu | excl 0.0
CPU utilization for five seconds: 11%; one minute: 13%; five minutes: 14%
PID       5Sec    1Min     5Min Process
1         0.1%    2.7%     3.7% kernel
12312     7.6%    7.0%     7.4% ios-base
12329     1.8%    1.6%     1.2% tcp.proc
12330     0.6%    0.5%     0.5% udp.proc
12331     0.1%    0.2%     0.2% iprouting.iosproc
12332     0.1%    0.1%     0.1% cdp2.iosproc

Hit : cpu above 40%, caused by kernel process
ldus220#sh proc cpu | excl 0.0
CPU utilization for five seconds: 46%; one minute: 16%; five minutes: 14%
PID       5Sec    1Min     5Min Process
1        35.6%    6.3%     4.5% kernel
12312     7.1%    7.1%     7.4% ios-base
12329     1.0%    1.5%     1.2% tcp.proc
12330     0.3%    0.5%     0.5% udp.proc
12331     0.1%    0.2%     0.2% iprouting.iosproc
12332     0.1%    0.1%     0.1% cdp2.iosproc

Using cpu detail, I can see it is process id 17 that is hit but I don't have a clue what this process does. How can I find what this process does and if it is internal kernel housekeeping that is causing this or an external cause ?
ldus220#sh proc cpu detail | excl 0.0
CPU utilization for five seconds: 46%; one minute: 21%; five minutes: 18%
PID/TID   5Sec    1Min     5Min Process             Prio  STATE        CPU
1        33.9%    5.7%     4.3% kernel                                26d20h
       1  54.2%   79.5%    81.6% [idle thread]          0  Ready       1355d
     17  33.9%    5.7%     4.2%                       10  Running      6d07h
12312     7.3%    9.8%     9.1% ios-base                                106d

Any info or pointer to info would be appreciated.

Wim Holemans
Network Services
University of Antwerp

_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/