[c-nsp] GSR CPU Process is very HIGH 95%

bharath kondi bluffmaster4hearts at gmail.com
Wed Oct 7 08:32:12 EDT 2009


Hi All,

I am seeing a strange thing in our GSR 12000, CPU process reached 95%. Below
is the findings, can anyone help me ASAP.

The highest usage is Net Input. What is Net Input? Please help me guys.

Thanks in advance.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 66   1485974962884865916          0 17.91% 15.99% 16.04%   0 Net Input


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

GW-01-KLS-MY#show processes cpu

CPU utilization for five seconds: 95%/71%; one minute: 96%; five minutes:
95%

 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

   1         276      4999         55  0.00%  0.00%  0.00%   0 Chunk Manager


   2       17848   2601954          6  0.00%  0.00%  0.00%   0 Load Meter


   3     7930952   8730366        908  0.00%  0.13%  0.15%   0 OSPF-1 Hello


   4           0         1          0  0.00%  0.00%  0.00%   0 EE48 TCAM
Carve

   5   143111728   7403379      19330  0.00%  1.67%  1.29%   0 Check heaps


   6         592      1014        583  0.00%  0.00%  0.00%   0 Pool Manager


   7           0         2          0  0.00%  0.00%  0.00%   0 Timers


   8           0         2          0  0.00%  0.00%  0.00%   0 RP LC ATM
Common

   9           0         2          0  0.00%  0.00%  0.00%   0 Serial
Backgroun

  10           0         1          0  0.00%  0.00%  0.00%   0 LB  Daemon


  11         196     20433          9  0.00%  0.00%  0.00%   0 MBUS Flash


  12       45480  20345689          2  0.00%  0.00%  0.00%   0 MBUS System


  13       24252  13034266          1  0.00%  0.00%  0.00%   0 MBUS
interface S

  14       27968  64775693          0  0.00%  0.00%  0.00%   0 Test CPUHOG
proc

  15           0         2          0  0.00%  0.00%  0.00%   0 Fabric


  16           0         9          0  0.00%  0.00%  0.00%   0 GoodieMgr
Server

  17         288    218659          1  0.00%  0.00%  0.00%   0 IPC Dynamic
Cach

  18           0         1          0  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
0

  19       16924  13034266          1  0.00%  0.00%  0.00%   0 IPC Periodic
Tim

  20    52069932 306854514        169  0.47%  0.54%  0.53%   0 IPC Seat
Manager

  21       15728  13034286          1  0.00%  0.00%  0.00%   0 IPC Deferred
Por

  22        1044   1311179          0  0.00%  0.00%  0.00%   0 Compute SRP
rate

  23    12694176  64786441        195  0.00%  0.09%  0.10%   0 ARP Input


  24       14104  12942815          1  0.00%  0.00%  0.00%   0 PFE
statistics C

  25       17696   1300822         13  0.00%  0.00%  0.00%   0 HC Counter
Timer

  26           0         2          0  0.00%  0.00%  0.00%   0 DDR Timers


  27          20        27        740  0.00%  0.00%  0.00%   0 Entity MIB
API

  28           0         1          0  0.00%  0.00%  0.00%   0 SERIAL
A'detect

  29           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone
Manager

  30     1432648 122155691         11  0.00%  0.07%  0.07%   0 Mbus Agent
Manag

  31        6784      5707       1188  0.00%  0.00%  0.00%   0 Mbus Agent
Downl

  32       23844    172499        138  0.00%  0.00%  0.00%   0 Board
Management

  33          28      2291         12  0.00%  0.00%  0.00%   0 Logger MBUS
tx

  34         132      2553         51  0.00%  0.00%  0.00%   0 Logger IPC tx


  35     2044120  39245664         52  0.07%  0.02%  0.00%   0 RP MBUS
heartbea

  36          44        45        977  0.00%  0.00%  0.00%   0 RP
Arbitration

  37           0         1          0  0.00%  0.00%  0.00%   0 User LED
Message

  38           0         1          0  0.00%  0.00%  0.00%   0 Critical
Bkgnd

  39      701828   2834530        247  0.00%  0.01%  0.00%   0 Net
Background

  40           0         3          0  0.00%  0.00%  0.00%   0 IDB Work


  41         692     23956         28  0.00%  0.00%  0.00%   0 Logger


  42       26256  12942800          2  0.00%  0.00%  0.00%   0 TTY
Background

  43       97096  13850111          7  0.00%  0.00%  0.00%   0 Per-Second
Jobs

  44           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl0

  45           0         4          0  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
1

  46           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl1

 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

  47           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl1

  48           0         1          0  0.00%  0.00%  0.00%   0 Field Diag
Autob

  49           0         1          0  0.00%  0.00%  0.00%   0 Field Diag
msg p

  50           0         1          0  0.00%  0.00%  0.00%   0 Field Diag
Misc

  51       11260  13034300          0  0.00%  0.00%  0.00%   0 ICC Slave
Reques

  52           0         1          0  0.00%  0.00%  0.00%   0 ICC Async
mcast

  53         520        37      14054  0.00%  0.00%  0.00%   0 OIR Sync
Process

  54       32340  13034274          2  0.00%  0.00%  0.00%   0 FIA Poll


  55      173236 283385374          0  0.00%  0.04%  0.02%   0 Fabric Ping


  56           0         1          0  0.00%  0.00%  0.00%   0 Inode Table
Dest

  57           0         1          0  0.00%  0.00%  0.00%   0 QoS HA CHKPT


  58      197120   6538296         30  0.00%  0.00%  0.00%   0 rf task


  59           0         2          0  0.00%  0.00%  0.00%   0 IPC Apps Task


  60           0         1          0  0.00%  0.00%  0.00%   0 IPC RTTYC
Messag

  61      112880    435694        259  0.00%  0.00%  0.00%   0 RP Standby


  62           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl0

  63      412512   5265597         78  0.00%  0.00%  0.00%   0 GSR RP CMD
Event

  64          36       298        120  0.00%  0.00%  0.00%   0 RTTYS Process


  65           8        26        307  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
2

 * 66   1485974962884865916          0 17.91% 15.99% 16.04%   0 Net Input
     *

  67      339752   4475894         75  0.00%  0.04%  0.02%   0 Compute load
avg

  68     7998564    307368      26022  0.00%  0.03%  0.05%   0 Per-minute
Jobs

  69           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl2

  70           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl2

  71        2356     31202         75  0.00%  0.00%  0.00%   0 Board Mgr sl
2

  72          56       217        258  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
3

  73           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl3

  74           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl3

  75        2732     31196         87  0.00%  0.00%  0.00%   0 Board Mgr sl
3

  76           4        17        235  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
6

  77           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl6

  78           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl6

  79        4256     29356        144  0.00%  0.00%  0.00%   0 Board Mgr sl
6

  80           0        16          0  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
8

  81           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl8

  82           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl8

  83        2612     29355         88  0.00%  0.00%  0.00%   0 Board Mgr sl
8

  84       23380  12943077          1  0.00%  0.00%  0.00%   0 MBUS Config


  85      532660  71175882          7  0.00%  0.07%  0.06%   0 Env Mon


  86       80600  12943006          6  0.00%  0.00%  0.00%   0 RP Ethernet


  87           0         1          0  0.00%  0.00%  0.00%   0 GSR RLC
Manager

  88           0         2          0  0.00%  0.00%  0.00%   0 RLC RVI HA


  89           0         1          0  0.00%  0.00%  0.00%   0 FPD
Management P

  90           0         1          0  0.00%  0.00%  0.00%   0 FPD Action
Proce

  91           0         1          0  0.00%  0.00%  0.00%   0 Memory
Scanner

  92           0         1          0  0.00%  0.00%  0.00%   0 BMA Microcode
Lo

  93           0         1          0  0.00%  0.00%  0.00%   0 GLC POST
Loader

  94           0        21          0  0.00%  0.00%  0.00%   0
bfr_fab_bw_info_

 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

  95           0         1          0  0.00%  0.00%  0.00%   0 BITS Clock
Mgr p

  96           4         2       2000  0.00%  0.00%  0.00%   0 MIP Mailbox


  97           0         1          0  0.00%  0.00%  0.00%   0 vcq_proc


  98           0        22          0  0.00%  0.00%  0.00%   0 TurboACL


  99           0         2          0  0.00%  0.00%  0.00%   0 TurboACL
chunk

 100           0         2          0  0.00%  0.00%  0.00%   0 MCAST HA


 101           4        16        250  0.00%  0.00%  0.00%   0 GSR COS init


 102           0         1          0  0.00%  0.00%  0.00%   0 GSR DTS init


 103        1304      4380        297  0.00%  0.00%  0.00%   0 LC adj
updater

 104           0         6          0  0.00%  0.00%  0.00%   0 GSR SSRP
queuein

 105       17436   2601502          6  0.00%  0.00%  0.00%   0 Distributed
Mult

 106    28727104 109128095        263  0.31%  0.45%  0.43%   0 IP Input


 107           4         1       4000  0.00%  0.00%  0.00%   0 ICMP event
handl

 108     3836880   3228486       1188  0.00%  0.04%  0.05%   0 CDP Protocol


 109           0        22          0  0.00%  0.00%  0.00%   0 QoS Manager


 110           0         1          0  0.00%  0.00%  0.00%   0 PPP IP Add
Route

 111           0         1          0  0.00%  0.00%  0.00%   0 LSP Tunnel
FRR

 112           0         2          0  0.00%  0.00%  0.00%   0 MPLS
Auto-Tunnel

 113           0         5          0  0.00%  0.00%  0.00%   0 dMATM RP msg
han

 114           0         2          0  0.00%  0.00%  0.00%   0 HDLC HA


 115       23260   2034726         11  0.00%  0.00%  0.00%   0 ARP HA


 116        1376      2475        555  0.00%  0.00%  0.00%   0 EM FD Syslog


 117           0         2          0  0.00%  0.00%  0.00%   0 EM FD SNMP


 118           0         1          0  0.00%  0.00%  0.00%   0 AC Switch


 119           0         1          0  0.00%  0.00%  0.00%   0
ac_atm_state_eve

 120           0         1          0  0.00%  0.00%  0.00%   0 SSS Manager


 121           0         1          0  0.00%  0.00%  0.00%   0 SSS Feature
Mana

 122      100152  50892410          1  0.00%  0.00%  0.00%   0 SSS Feature
Time

 123          72      1169         61  0.00%  0.00%  0.00%   0 SSM
connection m

 124           0         2          0  0.00%  0.00%  0.00%   0 SSCOP Input


 125           0         2          0  0.00%  0.00%  0.00%   0 SSCOP Output


 126         496    217137          2  0.00%  0.00%  0.00%   0 SSCOP Timer


 127    20922632    722654      28952  0.00%  0.13%  0.13%   0 IP Background


 128           0         2          0  0.00%  0.00%  0.00%   0 ILMI Input


 129           0         2          0  0.00%  0.00%  0.00%   0 SNMP Timers


 130           0         2          0  0.00%  0.00%  0.00%   0 ILMI Request


 131           0         2          0  0.00%  0.00%  0.00%   0 ILMI Response


 132           0         1          0  0.00%  0.00%  0.00%   0 ILMI Timer
Proce

 133           4         2       2000  0.00%  0.00%  0.00%   0 ATM PVC
Discover

 134           0         2          0  0.00%  0.00%  0.00%   0 ATMSIG
DRIVERAPI

 135      300676    682768        440  0.00%  0.00%  0.00%   0 Adj Manager


 136     1952840  28999991         67  0.00%  0.11%  0.12%   0 CEF process


 137           0        16          0  0.00%  0.00%  0.00%   0 Routemap RP
IPC

 138      219300 123720446          1  0.00%  0.02%  0.00%   0 MDFS RP
process

 139           0         1          0  0.00%  0.00%  0.00%   0 CHKPT EXAMPLE


 140           0         1          0  0.00%  0.00%  0.00%   0 CHKPT DevTest


 141      180332  10527491         17  0.00%  0.01%  0.00%   0 TCP Timer


 142       96060    430337        223  0.00%  0.00%  0.00%   0 TCP Protocols


 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

 143           0         1          0  0.00%  0.00%  0.00%   0 RARP Input


 145        3780    216804         17  0.00%  0.00%  0.00%   0 IP Cache Ager


 146           0         1          0  0.00%  0.00%  0.00%   0 Uidb Mgr


 147          20     10880          1  0.00%  0.00%  0.00%   0 Feature
Manager

 148        8196   4332167          1  0.00%  0.00%  0.00%   0 ISE MQC
Download

 149        4620    433616         10  0.00%  0.00%  0.00%   0 ISE MQC DRR
quan

 150           0         2          0  0.00%  0.00%  0.00%   0 PIM mdt
tunnel a

 151         492    217137          2  0.00%  0.00%  0.00%   0 TCP Intercept
Ti

 152           0         1          0  0.00%  0.00%  0.00%   0 Socket Timers


 153           0         2          0  0.00%  0.00%  0.00%   0 TC-ATM Proc


 154           0         2          0  0.00%  0.00%  0.00%   0 Tag Input


 155           4        15        266  0.00%  0.00%  0.00%   0 LFIB FRR IOS


 156           0         4          0  0.00%  0.00%  0.00%   0 TSPTUN HA


 157           0         1          0  0.00%  0.00%  0.00%   0 L2X Socket
proce

 158           0         1          0  0.00%  0.00%  0.00%   0 L2TP mgmt
daemon

 159           0         2          0  0.00%  0.00%  0.00%   0 FR HA


 160           0         3          0  0.00%  0.00%  0.00%   0 IPv6 CEF
process

 161       20612    216805         95  0.00%  0.00%  0.00%   0 6PE Scanner


 162           0         1          0  0.00%  0.00%  0.00%   0 6PE fibidb
Scann

 163           0         2          0  0.00%  0.00%  0.00%   0 PPP HA


 164           0         2          0  0.00%  0.00%  0.00%   0 ATM HA


 165           0         1          0  0.00%  0.00%  0.00%   0 L2X Data
Daemon

 166           0         1          0  0.00%  0.00%  0.00%   0 Key chain
liveke

 167          16        30        533  0.00%  0.00%  0.00%   0 NonBlk Xmt Sl
4

 168           0        25          0  0.00%  0.00%  0.00%   0 MPLS Auto
Mesh P

 169           0         1          0  0.00%  0.00%  0.00%   0 Encrypt Proc


 170     4578332     30717     149053  0.00%  0.00%  0.00%   0 Key Proc


 171           0         1          0  0.00%  0.00%  0.00%   0 Crypto
Support

 172          36         3      12000  0.00%  0.00%  0.00%   0 Crypto CA


 173           0         2          0  0.00%  0.00%  0.00%   0 Lawful
Intercept

 174           0         2          0  0.00%  0.00%  0.00%   0 LI messaging


 175        4416    220934         19  0.00%  0.00%  0.00%   0 SPA ENTITY
Proce

 176           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl4

 177           0         1          0  0.00%  0.00%  0.00%   0 SNMP
ConfCopyPro

 178           0         1          0  0.00%  0.00%  0.00%   0 Syslog Traps


 179           0         1          0  0.00%  0.00%  0.00%   0 SONET Traps


 180           0         1          0  0.00%  0.00%  0.00%   0 IP Source
Tracke

 181           0         1          0  0.00%  0.00%  0.00%   0 DATA Transfer
Pr

 182           0         1          0  0.00%  0.00%  0.00%   0 DATA
Collector

 183           0         4          0  0.00%  0.00%  0.00%   0 EM Server


 184           0         1          0  0.00%  0.00%  0.00%   0 MAC ACL RP
msg h

 185           0         2          0  0.00%  0.00%  0.00%   0 EM Applet
Direct

 186      191300    715241        267  0.00%  0.00%  0.00%   0 CEF Scanner


 187     1036816  51181537         20  0.00%  0.01%  0.03%   0 CEF RP IPC
Backg

 188        2944    180951         16  0.00%  0.00%  0.00%   0 SSH Event
handle

 189    23567044  74381747        316  0.55%  0.33%  0.35%   0 IP SNMP


 190     7748276  37082413        208  0.31%  0.14%  0.12%   0 PDU
DISPATCHER

 191    44648072  37364151       1194  1.11%  0.67%  0.67%   0 SNMP ENGINE


 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

 192        3144      5310        592  0.00%  0.00%  0.00%   0 SNMP Traps


 193        1784      2583        690  0.00%  0.00%  0.00%   0 Mbus Agent
Upgra

 194          16       133        120  0.00%  0.00%  0.00%   0 TCP Listener


 195       44848  13302324          3  0.00%  0.00%  0.00%   0 NTP


 196           0        11          0  0.00%  0.00%  0.00%   0 Delayed
function

 197        3060       634       4826  0.00%  0.00%  0.00%   0 Async i/o
server

 198        2176       423       5144  0.00%  0.00%  0.00%   0 Async i/o
server

 199      802932  14893657         53  0.00%  0.00%  0.00%   0 OSPF-1 Router


 200    25683912  63061115        407  0.23%  0.23%  0.19%   0 BGP Router


 201     2313128  15901418        145  0.00%  0.05%  0.01%   0 BGP I/O


 202  1270749548 477837637       2659  2.07%  2.04%  2.29%   0 BGP Scanner


 203       21580   1300806         16  0.00%  0.00%  0.00%   0 OBFL Envmon


 204           0         2          0  0.00%  0.00%  0.00%   0
cpf_process_msg_

 205           0         1          0  0.00%  0.00%  0.00%   0 NonBlkResp
Sl4

 206       54924   4082881         13  0.00%  0.00%  0.00%   0
cpf_process_ipcQ

 207        1160      1910        607  0.00%  0.00%  0.00%   0 Board Mgr sl
4

 208        4832      5458        885  0.31%  1.43%  0.58%   3 SSH Process


GW-01-KLS-MY#

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Thanks and Regards,

Bharath Kondi.


More information about the cisco-nsp mailing list