[c-nsp] debugging stack corruption

[bill fumerola]

anyone see anything like this. i assume only a reload will fix this:

rtr1#sh proc cpu | e 0.0
CPU utilization for five seconds: 33%/8%; one minute: 37%; five minutes:
35%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
    3    528125122320274973         22 23.35% 20.79% 20.97%   0 Exec            
   70   3616544001417549298        255  0.15%  0.11%  0.12%   0 IP Input        
  115      4851843096833738          0  0.15%  0.14%  0.15%   0 HQF Shaper Backg
rtr1#

nobody else is logged on, little to no amount of traffic is running
through the aux/cons ports, but this is interesting:

rtr1#show stacks
Minimum process stacks:
 Free/Size   Name
 5676/6000   CDP BLOB
 8640/9000   EM ED RF
11052/12000  Router Init
 8676/9000   cdp init process
 8348/12000  Init
 5304/6000   RADIUS INITCONFIG
 3616/6000   BGP Open
 2264/3000   Rom Random Update Process
 5616/6000   URPF stats
 5316/6000   BGP Accepter
 9248/12000  Exec
 7176/12000  SSH Process
 4264/6000   TFTP Read Process
 4204/6000   MSDP Open
34540/36000  TCP Command
 5236/7200   TTY Daemon
 8496/9000   IP-EIGRP Router
 3360/6000
d^\ytd^[^P^Ld^\zTd^[`Dd^[I$d^\^[Td^[T^Dd^\y^Dd^\^P<d^[@Td^[p<d^\^B,d^\^Sld^\^@dd^\dtd^\f4d^\eDd^\g^Dd^\eTd^\c^Td^\f$d^\ddd^\g^Td^[e\d^[NTd^[)$d^[pLd^[|^\d^\
,d^[mdd^\^Nld^\
dd^[	4d^[Q<d^[^V^\d^\1dd^[1<d^[O4d^[|Dd^\^P<d^[^Ydd^[e\d^[)$d^[NTd^[
4d^[1^Dd^[`Td^[{td^[^E^\d^[m<d^[^Z^Ld^[8<d^[}^Dd^[j^\d^\^Q|d^[x^\d^[u^\d^\
,d^\^ALd^[jTd^[pLd^[|^\d^[~td^[^D,d^[RDd^ld^[x$d^[^^Dd^[ptd^[^Bld^[^QLd^[^Q\d^[
ld^[zdd^\,$d^[ttd^[^Vdd^[iLd^[^X\d^[)4d^\34d^[v$d^[^VTd^\^Ptd^^\d^[{Dd^[R|d^\^Q^\d^[`^Ld^[]^Ld^\
,d^[^R^Dd^[^Fld^[\d^[b^Td^[^LDd^\^P^Dd^[^B4d^[^NLd^[^Y,d^[^Kdd^\
^\d^\^CDd^[s^Td^[^A^\d^[U,d^[j,d^[~^Dd^\^QDd^[Jtd^[~Ld^[|^Td^[,Dd^^\d^[rld^[R|d^[{Dd^[
\d^[^Add^[^Q\d^[^QLd^[
ld^[ttd^[zdd^\,$d^[^Vdd^[)4d^\34d^[wLd^[m,d^[^Z|d^[\,d^[g|d^[y|d^[^D<d^[x$d^[^^Dd^[
ld^[^Bld^[RDd^[ptd^[^Q$d^[v4d^\^Ptd^[^VTd^[7$d^\1td^[P$d^[uTd^[^VTd^[zdd^[7$d^[z,d^[z^\d^[ytd^[@Td^[<^Dd^\,$d^\+Dd^\,4d^[^D<d^\`^Dd^[^VTd^[k4d^[P^Td^[a$d^[$d^[^V^\d^[^Utd^[mdd^[^Ytd^[|^Ld^[^L^Ld^\^ALd^[#^Dd^[e\d^[f^Dd^\^FTd^[^Pld^[^B|d^[n^\d^[d4d^[H|d^[^Rtd^[^N^Td^Td^[^Td^[{,d^[+dd^[`Td^[.^Dd^[s\d^[^ETd^[^Z^Ld^[
$d^[YTd^\^L^Dd^[1^Dd^[^O^\d^[^PDd^[^L^\d^\
dd^[
Ld^[)$d^[#td^[1<d^[^E|d^[^_Ld^[KTd^[
4d^[^BDd^[yLd^[+,d^[^E^\d^\^S^Dd^[
4d^[y^Td^[^WDd^[l\d^[Y|d^\1^Dd^\0$d^\/Dd^\1dd^[{^Dd^[^SDd^[^LTd^[|^\d^[H4d^[pLd^[M<d^[.,d^[]ld^[Q<d^[U^\d^[~td^[l$d^[8<d^[6^Ld^[^F4d^[^Odd^\^O$d^[^K<d^[^N<d^[^K^Dd^[^W4d^[_,d^[p^Dd^[+^\d^[N,d^[$Td^[~^\d^[eLd^[NTd^[
,d^[xTd^[r4d^[u^\d^[n^Ld^[rDd^[p^Td^[{td^[~<d^\
,d^[}$d^[}^Dd^[P\d^[w|d^[mtd^[O4d^[{ld^[x\d^[?<d^[m<d^[
<d^[o4d^[w<d^[y<d^[*<d^\^P<d^[u|d^[^Ydd^\^Pdd^[^Y<d^[D|d^\^P,d^[.td^\^Nld^\^N4d^[|Dd^[$^\d^[jTd^[q,d^[j^\d^[\Td^\^Q|d^[f,d^[^D,d^[gDd^[x^\d^[]4d^\&Dd^[w^Ld^[bLd^[L\d^[
Dd^[dld^[.<d^[Lld^\	td^\4d^\ld^^Td^\d^\	<d^\
^Dd^Ld^$d^[,d^[dd^[^\d^[Td^\
 6856/9000
d^\^[Td^[T^Dd^\y^Dd^\^P<d^[@Td^[p<d^\^B,d^\^Sld^\^@dd^\dtd^\f4d^\eDd^\g^Dd^\eTd^\c^Td^\f$d^\ddd^\g^Td^[e\d^[NTd^[)$d^[pLd^[|^\d^\
,d^[mdd^\^Nld^\
dd^[	4d^[Q<d^[^V^\d^\1dd^[1<d^[O4d^[|Dd^\^P<d^[^Ydd^[e\d^[)$d^[NTd^[
4d^[1^Dd^[`Td^[{td^[^E^\d^[m<d^[^Z^Ld^[8<d^[}^Dd^[j^\d^\^Q|d^[x^\d^[u^\d^\
,d^\^ALd^[jTd^[pLd^[|^\d^[~td^[^D,d^[RDd^ld^[x$d^[^^Dd^[ptd^[^Bld^[^QLd^[^Q\d^[
ld^[zdd^\,$d^[ttd^[^Vdd^[iLd^[^X\d^[)4d^\34d^[v$d^[^VTd^\^Ptd^^\d^[{Dd^[R|d^\^Q^\d^[`^Ld^[]^Ld^\
Minimum process stacks:
 Free/Size   Name
,d^[^R^Dd^[^Fld^[\d^[b^Td^[^LDd^\^P^Dd^[^B4d^[^NLd^[^Y,d^[^Kdd^\
^\d^\^CDd^[s^Td^[^A^\d^[U,d^[j,d^[~^Dd^\^QDd^[Jtd^[~Ld^[|^Td^[,Dd^^\d^[rld^[R|d^[{Dd^[
\d^[^Add^[^Q\d^[^QLd^[
ld^[ttd^[zdd^\,$d^[^Vdd^[)4d^\34d^[wLd^[m,d^[^Z|d^[\,d^[g|d^[y|d^[^D<d^[x$d^[^^Dd^[
ld^[^Bld^[RDd^[ptd^[^Q$d^[v4d^\^Ptd^[^VTd^[7$d^\1td^[P$d^[uTd^[^VTd^[zdd^[7$d^[z,d^[z^\d^[ytd^[@Td^[<^Dd^\,$d^\+Dd^\,4d^[^D<d^\`^Dd^[^VTd^[k4d^[P^Td^[a$d^[$d^[^V^\d^[^Utd^[mdd^[^Ytd^[|^Ld^[^L^Ld^\^ALd^[#^Dd^[e\d^[f^Dd^\^FTd^[^Pld^[^B|d^[n^\d^[d4d^[H|d^[^Rtd^[^N^Td^Td^[^Td^[{,d^[+dd^[`Td^[.^Dd^[s\d^[^ETd^[^Z^Ld^[
$d^[YTd^\^L^Dd^[1^Dd^[^O^\d^[^PDd^[^L^\d^\
dd^[
Ld^[)$d^[#td^[1<d^[^E|d^[^_Ld^[KTd^[
4d^[^BDd^[yLd^[+,d^[^E^\d^\^S^Dd^[
4d^[y^Td^[^WDd^[l\d^[Y|d^\1^Dd^\0$d^\/Dd^\1dd^[{^Dd^[^SDd^[^LTd^[|^\d^[H4d^[pLd^[M<d^[.,d^[]ld^[Q<d^[U^\d^[~td^[l$d^[8<d^[6^Ld^[^F4d^[^Odd^\^O$d^[^K<d^[^N<d^[^K^Dd^[^W4d^[_,d^[p^Dd^[+^\d^[N,d^[$Td^[~^\d^[eLd^[NTd^[
,d^[xTd^[r4d^[u^\d^[n^Ld^[rDd^[p^Td^[{td^[~<d^\
,d^[}$d^[}^Dd^[P\d^[w|d^[mtd^[O4d^[{ld^[x\d^[?<d^[m<d^[
<d^[o4d^[w<d^[y<d^[*<d^\^P<d^[u|d^[^Ydd^\^Pdd^[^Y<d^[D|d^\^P,d^[.td^\^Nld^\^N4d^[|Dd^[$^\d^[jTd^[q,d^[j^\d^[\Td^\^Q|d^[f,d^[^D,d^[gDd^[x^\d^[]4d^\&Dd^[w^Ld^[bLd^[L\d^[
Dd^[dld^[.<d^[Lld^\	td^\4d^\ld^^Td^\d^\	<d^\
^Dd^Ld^$d^[,d^[dd^[^\d^[Td^\
10468/12000  HSRP (Standby)

Interrupt level stacks:
Level    Called Unused/Size  Name
  1  2648551315   6280/9000  Network interfaces
  2           0   9000/9000  DMA/Timer Interrupt
  3      185107   7472/9000  PA Management Int Handler
  4  1715750501   8444/9000  Console Uart
  5           0   9000/9000  OIR/Error Interrupt
  7  3207930022   8532/9000  NMI Interrupt Handler

Spurious interrupts: 233
rtr1#

and on a different router:

rtr1.chi#sh stacks
Minimum process stacks:
 Free/Size   Name
[....]
 3500/6000  
 7160/9000    5,<$/jDSw_h 5,< 5,< 5,< 5,< 5,< d(X d(X 5,< 5,< 5,< 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< d'X 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,< 5,<V7;@ 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<Vo^BT 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,< 5,< 5,< 5,< 5,< 5,<Q6mT 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,<#^Qz|#^Qy|#^Qy| 5,<#^Qx|#^Qx| 5,<%Dtx%Dtx%Dtx%Dtx%Dsx%Dsx%Dsx%Dsx 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,< 5,<%Dsx 5,< 5,< 5,<%Drx 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,<#^Qw|#^Qw|#^Qv| 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,<S3^R@ 5,< 5,< 5,<Vo^BT d'X d&X d&X d%X 5,< 5,< 5,<&S<<!A`t 5,< 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<#W:x#W9x
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<Vo^AT-4,4 5,<PN^P^T 5,<
5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,< 5,<
5,< 5,<Vo^ATVo
 5316/6000   BGP Accepter
10176/12000  Exec

although that router doesn't display the same CPU symptoms.

first router is running:
Cisco IOS Software, 7301 Software (C7301-K91P-M), Version 12.2(31)SB11, RELEASE SOFTWARE (fc3)
ROM: System Bootstrap, Version 12.3(4r)T4, RELEASE SOFTWARE (fc1)
BOOTLDR: 7301 Software (C7301-BOOT-M), Version 12.3(26), RELEASE SOFTWARE (fc2)

second router is running:
Cisco IOS Software, 7301 Software (C7301-K91P-M), Version 12.2(31)SB12,
RELEASE SOFTWARE (fc3)


-- bill