[c-nsp] Weird config changes on C2621XM with AIM-VPN/BPII

[Jared Mauch]

On Thu, Sep 17, 2015 at 01:47:46PM +0000, Nick Nauwelaerts wrote:
> i would guess to join our nexus fex's in the pub, the also like to go missing in between rancid checkups.
> 
> in our case it seems to be a wonky nx-os revision in combination with datacenter manager which seems to cause quite some load with its checkups.
> 
> anything in the router's logs during the disappeances?

	I've seen similar issues before with various hardware.  Usually it's a software bug
where two people are talking to the microcontroller at the same time
and there's no concurrency checking.  We've exposed a lot of bugs by having two scripts
do the same thing at the same time.  Often a cisco device doesn't expect
concurrent memory/device access.

	Recommendation: 

	Figure out how to make it happen, either in a tight loop, or having 3 windows
open doing while [1 == 1]: do clogin -x /tmp/rancid-commands hostname done

it might be as simple as finding the show controller or show inventory commands and running those
in a loop.  make sure cisco knows how you login and they reproduce it the same way themselves with
these critical variables in mind:

1) via SSH
2) via IPv6
3) where SSH uses specific terminal types
4) where the SSH client offers keys

	We had issues where optics would report odd things for a year or so and filled a lot
of rancid logs.  This was because Cisco wasn't expecting a certain older flavor of their own
optic and their EEPROM validation code wasn't perfect.

	- Jared

-- 
Jared Mauch  | pgp key available via finger from jared at puck.nether.net
clue++;      | http://puck.nether.net/~jared/  My statements are only mine.