[c-nsp] Simultaneous SAA Freak Out

[Crist Clark]

Had a problem last night we're having some fun trying to figure out.
The TAC doesn't have any immediate answers. But hope this may sound
familiar to someone out there.

Within two or three minutes of each other, we had a bunch of routers
distributed all over our network go a little whacky with the SAA. We
caught the messages on our syslog server. They look something like,

Jan 12 18:12:44 xxxx-2621-01.xxxx.com 5153: -Process= "IP RTR Probe MaxName", ipl= 6, pid= 28
Jan 12 18:12:44 xxxx-2621-01.xxxx.com 5154: -Traceback= 803DF7D4 803DFC28 80474524 80476CF8 804EF8F0 804F4CFC 809DB97C 809D3530 803F3B7C
Jan 12 18:13:44 xxxx-2621-01.xxxx.com 5155: Jan 13 02:13:29 UTC: %SCHED-3-STUCKTMR: Sleep with expired timer 81194624, time 0x4F45551C0 (00:01:00 ago).

What's really weird is the timing. These routers are widely distributed
in a physical (North America, Europe, South America) and logical (they
are mostly EIGRP speakers but not all in the same AS and not all are
even participating in EIGRP; they don't pass the same transit traffic)
sense. About the only thing they have in common is reporting to the same
network managment system (HPOV), which all of this noise almost killed.

Again, they all started this within a two or three minutes of each
other. We're not even sure where to start looking for problems. Anyone
offer ideas?
-- 
Crist J. Clark                               crist.clark at globalstar.com
Globalstar Communications                                (408) 933-4387