[c-nsp] Router down - What is "Net Background"?

[Justin Shore]

I just had a 3660 terminating about 1500 PVCs for DSL die this AM.  The 
3660 is running 12.3(22) w/ 256MB of RAM.  I'm at the remote POP now, 
consoled in and looking at the box.  The console hangs for 45-60s before 
I get a chance to run a quick command or two.  Then it goes back to the 
hung state about 5 seconds later.  The box keeps dropping my 2 OC3s and 
bouncing my IS-IS processes.  Syslog didn't appear to capture anything 
interesting leading up to this crash.  I had the box power cycled while 
I was driving down here.

Overall the CPU is low and neither CPU nor memory were being taxed over 
the past number of months.  The Net Background process is using a fair 
bit of the processor.  Actually, that's an understatement.  At times 
it's only using 9% or so.  At other times it's up to 97% as was the case 
here:

3660-2.brd#sh proc cpu sort 5m
CPU utilization for five seconds: 1%/0%; one minute: 17%; five minutes: 19%
  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
   28     4715940       611    7731049 97.09% 13.74%  9.23%   0 Net 
Background
   31      238608       865     275847  2.65%  8.58%  8.48%   0 
Per-Second Jobs
    5        8336       684      12187  0.00%  0.56%  0.68%   0 Check 
heaps
   60        7764      7776        998  0.01%  0.27%  0.43%   0 IP Input 

   18        3856      8741        441  0.00%  0.42%  0.33%   0 ARP 
Input
  167        2556      3046        839  0.02%  0.12%  0.13%   0 ISIS Upd 

  140        2732        15     182133  0.00%  0.00%  0.12%   0 Key Proc 

  179        1680      1676       1002  0.00%  0.13%  0.11%   0 OSPF 
Router
   47        1912       152      12578  0.00%  0.12%  0.11%   0 
Per-minute Jobs
   82        1792       597       3001  0.01%  0.15%  0.11%   0 IP 
Background
    3        1828       238       7680  0.00%  0.11%  0.09%   0 OSPF 
Hello
  100         644      1217        529  0.00%  0.13%  0.08%   0 DHCPD 
Receive
   51        1700       376       4521  0.00%  0.04%  0.05%   0 ATM 
Periodic
   83         700       417       1678  0.00%  0.01%  0.02%   0 IP RIB 
Update
  108         500       871        574  0.00%  0.06%  0.02%   0 CEF 
process
  124         172       471        365  0.00%  0.04%  0.02%   0 Exec 

  166         620      1908        324  0.00%  0.03%  0.02%   0 ISIS Adj 

  106         420       284       1478  0.00%  0.03%  0.00%   0 Adj 
Manager
    6         436      1406        310  0.00%  0.02%  0.00%   0 Pool 
Manager
   91         188       556        338  0.00%  0.01%  0.00%   0 ILMI 
Timer Proce
  163          92      1795         51  0.00%  0.00%  0.00%   0 CLNS 
Input


Per-Second Jobs is also using quite a bit of CPU.  The log has CPUHOG 
entries in it pointing at the Net Background process too:

000428: .Apr  4 08:36:31 CDT: %SYS-3-CPUHOG: Task ran for 50332 msec 
(10/0), process = Net Background, PC = 60439FFC.
-Traceback= 6043A004

What does the Net Background process do?  I haven't been able to find 
any answers on Cisco.com.


We're in the middle of SmartNet renewals and negotiations and 
unfortunately this box's contract has expired.  I can have our AM bless 
a TAC case for us if need be, assuming I can locate him; I think he's in 
training this week back East.

Any ideas?  I'm going to go back and scrutinize the log for anything 
useful.  As of right now the box is intermittently pingable, 
corresponding directly to the brief instances that I can use the 
console.  Customer packets seem to flow about as well.

Thanks
  Justin