[c-nsp] High CPU utilization caused by the TPLUS process

[Jeffrey Ollie]

Several times in the past month we have had high CPU utilization
caused by the TPLUS process, which I think is caused by stuck CLI
users.  As far as I can tell the only way to remedy the problem is to
reload the switch and/or router (this affects many different systems
with very different software versions and hardware).  "clear line X"
and "disconnect ssh X" have no effect.  Is there some way to recover
from this situation other than reloading the switch?  This output is
typical of a switch/router that has the problem:

ankeny-b2-sw1#who
    Line       User       Host(s)              Idle       Location
*  1 vty 0     jcollie    idle                 00:00:00 161.210.45.126
   2 vty 1     rancid     idle                    3d08h 161.210.221.251

  Interface      User        Mode                     Idle     Peer Address

ankeny-b2-sw1#clear line vty 1
[confirm]
 [OK]
ankeny-b2-sw1#who
    Line       User       Host(s)              Idle       Location
*  1 vty 0     jcollie    idle                 00:00:00 161.210.45.126
   2 vty 1     rancid     idle                    3d08h 161.210.221.251

  Interface      User        Mode                     Idle     Peer Address

ankeny-b2-sw1#disconnect ssh vty 1
ankeny-b2-sw1#who
    Line       User       Host(s)              Idle       Location
*  1 vty 0     jcollie    idle                 00:00:00 161.210.45.126
   2 vty 1     rancid     idle                    3d08h 161.210.221.251

  Interface      User        Mode                     Idle     Peer Address

ankeny-b2-sw1#show proc cpu sorted
CPU utilization for five seconds: 69%/1%; one minute: 70%; five minutes: 71%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 247   192022762 147173137       1304 55.91% 57.62% 58.09%   0 TPLUS
 187    11656575  55566492        209  4.47%  3.43%  3.35%   0 IP Input
 129    11183422  27874303        401  1.59%  1.33%  1.34%   0 Hulc LED Process
 181      958624   2471735        387  0.31%  0.09%  0.06%   0 CDP Protocol
 232     7068788   2597774       2721  0.31%  0.51%  0.48%   0 Marvell wk-a Pow
  58     2491351  62102425         40  0.15%  0.21%  0.21%   0 Fifo Error Detec
 108        2095      2987        701  0.15%  0.12%  0.06%   1 SSH Process
  72      400916  34843451         11  0.15%  0.05%  0.05%   0 HLFM address lea
  95     2556520   2458104       1040  0.15%  0.19%  0.20%   0 hpm counter proc
  31      320900   3003540        106  0.15%  0.01%  0.00%   0 Net Background
 137      514826    237111       2171  0.15%  0.04%  0.04%   0 HQM Stack Proces
  41      847968  29029606         29  0.15%  0.21%  0.22%   0 Transport Port A
 193     3316744   3842208        863  0.15%  0.14%  0.15%   0 Spanning Tree
 259      117317  12823251          9  0.15%  0.03%  0.01%   0 PM Callback
  14          57         7       8142  0.00%  0.00%  0.00%   0 Entity MIB API
  13           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager
  12           0         2          0  0.00%  0.00%  0.00%   0 AAA high-capacit
  11         132      7673         17  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT
  10           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API
  15           9       648         13  0.00%  0.00%  0.00%   0 EEM ED Syslog
   9      134263    239511        560  0.00%  0.01%  0.00%   0 ARP Input

-- 
Jeff Ollie

"You know, I used to think it was awful that life was so unfair. Then
I thought, wouldn't it be much worse if life were fair, and all the
terrible things that happen to us come because we actually deserve
them? So, now I take great comfort in the general hostility and
unfairness of the universe."

	-- Marcus to Franklin in Babylon 5: "A Late Delivery from Avalon"