[c-nsp] Disappearing Interfaces in config

Charles Sprickman spork at inch.com
Mon Oct 18 17:20:06 EDT 2004 

On Mon, 18 Oct 2004, Church, Chuck wrote:

> But it also fails on 'sh run', which in that case it's not trying to
> compress the config.

Right...  thanks.

> There was a thread on storing the config on other devices just a couple
> days ago.  See if that helps.  Otherwise, try freeing up some memory,
> and then wri mem.  Maybe try temporarily disabling your routing
> protocol, and then clearing all IP routes.  Might free up enough ram to
> build it correctly, without resorting to old info. But I think that
> version you're running is susceptible to some vulnerabilies, including
> the interface wedge one.  Not a bad idea to fix that anyway.

Yeah, I have some wonky "workarounds" in place to try and protect it from
the wedge.  I did apparently download c7200-jk9o3s-mz.122-17a.bin before
my CCO login was yanked, so perhaps that fixes this.  Although my guess is
that as Steve mentioned it's a memory leak.  A slow one that took two
years to impact everything, but a leak nonetheless.

I'm going to try to get someone onsite to babysit.  My POTS line order has
been in for a year and Level3 still can't quite find it and throw it in
our cabinets...

Charles

>
> Chuck Church
> Lead Design Engineer
> CCIE #8776, MCNE, MCSE
> Netco Government Services - Design & Implementation
> 1210 N. Parker Rd.
> Greenville, SC 29609
> Home office: 864-335-9473
> Cell: 703-819-3495
> cchurch at netcogov.com
> PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4371A48D
>
>
> -----Original Message-----
> From: Charles Sprickman [mailto:spork at inch.com]
> Sent: Monday, October 18, 2004 3:57 PM
> To: Church, Chuck
> Cc: cisco-nsp at puck.nether.net
> Subject: RE: [c-nsp] Disappearing Interfaces in config
>
> On Mon, 18 Oct 2004, Church, Chuck wrote:
>
> > It's my understanding that the router config is 'created' by the
> router
> > parsing all the data structures for interfaces, protocols etc.  Sounds
> > like a bug has caused it to stop seeing all these things, even though
> > they're working properly.
>
> That sounds about right.  I don't understand the whole issue of
> "fragmented memory in IOS", but watching the log while issuing a "wr
> mem"
> gave me this:
>
> Oct 18 15:50:30 l3-router-lo-0 3010779: Oct 18 15:50:27.257 EDT:
> %SYS-2-MALLOCFAIL: Memory allocation of 20000 bytes failed from
> 0x6060D264, alignment 0
> Oct 18 15:50:30 l3-router-lo-0 3010781: Alternate Pool: None  Free: 0
> Cause: No Alternate pool
> Oct 18 15:50:30 l3-router-lo-0 3010783: -Process= "SSH Process", ipl= 0,
> pid= 154
> Oct 18 15:50:30 l3-router-lo-0 3010780: Pool: Processor  Free: 735912
> Cause: Memory fragmentation
> Oct 18 15:50:30 l3-router-lo-0 3010782:
> Oct 18 15:50:30 l3-router-lo-0 3010784: -Traceback= 606115F0 60613D24
> 6060D26C 6058972C 60586E94 60586B40 605F0F28 605F11B4 605F1D9C 6058718C
> 60564A04 60564C84 60598F7C 60589540 6059BD98 61AC34EC
>
> Now it doesn't say anything there about the write/build operation, just
> "SSH Process", but that does coincide exactly with me issuing the
> command.
>
> For fun, here's the top of "sh mem":
>
>                 Head    Total(b)     Used(b)     Free(b)   Lowest(b)
> Largest(b)Processor   62E90820   186054624   183144856     2909768
> 651152      918136      I/O   20000000    33554432      391936
> 33162496
> 32407296    32967580    I/O-2    E000000    33554432     5630232
> 27924200    26284208    2792415
>
> >  I wouldn't reboot it, because you'll lose all those ones that stopped
> > making it into the config.
>
> Yeah.  Luckily I do have the config in cvs via rancid, so I think I can
> tftp that back up to "startup-config", but I'm not overjoyed about being
> the first one to do a reload on a router that's been up for almost two
> years. :)
>
> Also, the only reason I'm using "service compress-config" is that with
> all
> these interfaces for dsl, the config is too large to fit in nvram.  Is
> it
> possible to direct the router to save directly to one of the pcmcia
> cards?
> I would wager that it wouldn't tank on memory if it wasn't trying to
> compress the config.  Or not...
>
> Thanks,
>
> Charles
>
> >  I'm sure a later IOS fixes it, but it might take some work to get a
> > working config again. I'd grab a few previous configs, and compare
> those
> > to a current 'sh int' and see if you can piece together a config with
> > all interfaces.  Then reload it with what ever IOS seems to fix the
> > problem, and paste in what's missing.  Is this router low on memory,
> > possibly affecting the real-time creation of the config?  On the other
> > hand, I'd be reluctant to put a different IOS on a router that's been
> up
> > for over a year, but that's just me ;)
> >
> >
> > Chuck Church
> > Lead Design Engineer
> > CCIE #8776, MCNE, MCSE
> > Netco Government Services - Design & Implementation
> > 1210 N. Parker Rd.
> > Greenville, SC 29609
> > Home office: 864-335-9473
> > Cell: 703-819-3495
> > cchurch at netcogov.com
> > PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4371A48D
> >
> >
> > -----Original Message-----
> > From: cisco-nsp-bounces at puck.nether.net
> > [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Charles
> > Sprickman
> > Sent: Monday, October 18, 2004 2:30 PM
> > To: cisco-nsp at puck.nether.net
> > Subject: Re: [c-nsp] Disappearing Interfaces in config
> >
> > I'd also like to add that the interfaces DO NOT show in in "sh run"
> > either, "service compress-config" is on, and the interfaces are up and
> > passing traffic...  Looking at old cvs logs it seems I've lost about
> 100
> > interfaces, including a few loopbacks.
> >
> > Charles
> >
> > On Mon, 18 Oct 2004, Charles Sprickman wrote:
> >
> > > I've got a stumper here...
> > >
> > > We've got a 7206 running 12.2(8)T5 (cco expired) and I noticed that
> > rancid
> > > has started showing me about 5 interfaces that appear and then
> > disappear
> > > on each run.
> > >
> > > I took note of one and found that I can see the interface is
> > configured:
> > >
> > > router.bway.net#sh in atm 4/0.2491
> > > ATM4/0.2491 is up, line protocol is up
> > >   Hardware is ENHANCED ATM PA
> > >   Interface is unnumbered. Using address of Loopback8 (216.220.x.1)
> > >   MTU 4470 bytes, BW 40704 Kbit, DLY 190 usec,
> > >      reliability 255/255, txload 207/255, rxload 73/255
> > >   Encapsulation ATM
> > >   60187 packets input, 4705441 bytes
> > >   62612 packets output, 4946027 bytes
> > >   1 OAM cells input, 1 OAM cells output
> > >   AAL5 CRC errors : 0
> > >   AAL5 SAR Timeouts : 0
> > >   AAL5 Oversized SDUs : 0
> > >
> > > And after doing a "wr mem" I see no sign of it in the config:
> > >
> > > router.bway.net#sh conf | inc 2491
> > > ip route 216.220.x.91 255.255.255.255 ATM4/0.2491
> > >
> > > The route to the pvc is there, but no matching config for the
> > interface.
> > >
> > > I've got 1721 pvcs configured:
> > >
> > > 4095 maximum active VCs, 1721 current VCCs
> > >
> > > Any ideas?  This just came out of nowhere.  No reboot in over a
> year,
> > and
> > > all of our pvc configs are done in advance, so we have not added any
> > > interfaces since maybe 5 months ago.
> > >
> > > Thanks,
> > >
> > > Charles
> > >
> > > --
> > > Charles Sprickman
> > > spork at inch.com
> > >
> > > _______________________________________________
> > > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > > archive at http://puck.nether.net/pipermail/cisco-nsp/
> > >
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
>

More information about the cisco-nsp mailing list