[c-nsp] Disappearing Interfaces in config

Mon Oct 18 15:56:40 EDT 2004

On Mon, 18 Oct 2004, Church, Chuck wrote:

> It's my understanding that the router config is 'created' by the router
> parsing all the data structures for interfaces, protocols etc.  Sounds
> like a bug has caused it to stop seeing all these things, even though
> they're working properly.

That sounds about right.  I don't understand the whole issue of
"fragmented memory in IOS", but watching the log while issuing a "wr mem"
gave me this:

Oct 18 15:50:30 l3-router-lo-0 3010779: Oct 18 15:50:27.257 EDT:
%SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from
0x6060D264, alignment 0
Oct 18 15:50:30 l3-router-lo-0 3010781: Alternate Pool: None  Free: 0
Cause: No Alternate pool
Oct 18 15:50:30 l3-router-lo-0 3010783: -Process= "SSH Process", ipl= 0,
pid= 154
Oct 18 15:50:30 l3-router-lo-0 3010780: Pool: Processor  Free: 735912
Cause: Memory fragmentation
Oct 18 15:50:30 l3-router-lo-0 3010782:
Oct 18 15:50:30 l3-router-lo-0 3010784: -Traceback= 606115F0 60613D24
6060D26C 6058972C 60586E94 60586B40 605F0F28 605F11B4 605F1D9C 6058718C
60564A04 60564C84 60598F7C 60589540 6059BD98 61AC34EC

Now it doesn't say anything there about the write/build operation, just
"SSH Process", but that does coincide exactly with me issuing the command.

For fun, here's the top of "sh mem":

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)
Largest(b)Processor   62E90820   186054624   183144856     2909768
651152      918136      I/O   20000000    33554432      391936    33162496
32407296    32967580    I/O-2    E000000    33554432     5630232
27924200    26284208    2792415

>  I wouldn't reboot it, because you'll lose all those ones that stopped
> making it into the config.

Yeah.  Luckily I do have the config in cvs via rancid, so I think I can
tftp that back up to "startup-config", but I'm not overjoyed about being
the first one to do a reload on a router that's been up for almost two
years. :)

Also, the only reason I'm using "service compress-config" is that with all
these interfaces for dsl, the config is too large to fit in nvram.  Is it
possible to direct the router to save directly to one of the pcmcia cards?
I would wager that it wouldn't tank on memory if it wasn't trying to
compress the config.  Or not...

Thanks,

Charles

>  I'm sure a later IOS fixes it, but it might take some work to get a
> working config again. I'd grab a few previous configs, and compare those
> to a current 'sh int' and see if you can piece together a config with
> all interfaces.  Then reload it with what ever IOS seems to fix the
> problem, and paste in what's missing.  Is this router low on memory,
> possibly affecting the real-time creation of the config?  On the other
> hand, I'd be reluctant to put a different IOS on a router that's been up
> for over a year, but that's just me ;)
>
>
> Chuck Church
> Lead Design Engineer
> CCIE #8776, MCNE, MCSE
> Netco Government Services - Design & Implementation
> 1210 N. Parker Rd.
> Greenville, SC 29609
> Home office: 864-335-9473
> Cell: 703-819-3495
> cchurch at netcogov.com
> PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4371A48D
>
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Charles
> Sprickman
> Sent: Monday, October 18, 2004 2:30 PM
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] Disappearing Interfaces in config
>
> I'd also like to add that the interfaces DO NOT show in in "sh run"
> either, "service compress-config" is on, and the interfaces are up and
> passing traffic...  Looking at old cvs logs it seems I've lost about 100
> interfaces, including a few loopbacks.
>
> Charles
>
> On Mon, 18 Oct 2004, Charles Sprickman wrote:
>
> > I've got a stumper here...
> >
> > We've got a 7206 running 12.2(8)T5 (cco expired) and I noticed that
> rancid
> > has started showing me about 5 interfaces that appear and then
> disappear
> > on each run.
> >
> > I took note of one and found that I can see the interface is
> configured:
> >
> > router.bway.net#sh in atm 4/0.2491
> > ATM4/0.2491 is up, line protocol is up
> >   Hardware is ENHANCED ATM PA
> >   Interface is unnumbered. Using address of Loopback8 (216.220.x.1)
> >   MTU 4470 bytes, BW 40704 Kbit, DLY 190 usec,
> >      reliability 255/255, txload 207/255, rxload 73/255
> >   Encapsulation ATM
> >   60187 packets input, 4705441 bytes
> >   62612 packets output, 4946027 bytes
> >   1 OAM cells input, 1 OAM cells output
> >   AAL5 CRC errors : 0
> >   AAL5 SAR Timeouts : 0
> >   AAL5 Oversized SDUs : 0
> >
> > And after doing a "wr mem" I see no sign of it in the config:
> >
> > router.bway.net#sh conf | inc 2491
> > ip route 216.220.x.91 255.255.255.255 ATM4/0.2491
> >
> > The route to the pvc is there, but no matching config for the
> interface.
> >
> > I've got 1721 pvcs configured:
> >
> > 4095 maximum active VCs, 1721 current VCCs
> >
> > Any ideas?  This just came out of nowhere.  No reboot in over a year,
> and
> > all of our pvc configs are done in advance, so we have not added any
> > interfaces since maybe 5 months ago.
> >
> > Thanks,
> >
> > Charles
> >
> > --
> > Charles Sprickman
> > spork at inch.com
> >
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>