[c-nsp] 6504-E IOS SSH/memory issues

[Harold 'Buz' Dale]

We had a similar problem with a 7609 with a supe720.  TAC diagnosed that
we were using too much memory for our BGP tables.
We would get a login prompt but it would fail even with the correct
password. The box seems to be passing packets although we get some BGP
peering issues occasionally. This box is scheduled for replacement soon so
we are just dealing with it for the next week or so.
Good Luck,
Buz

----------
buz.dale at usg.edu
Network Support Specialist University System of GA -IT Services.
706-583-2052 or (Toll Free in GA) 888-875-3697
 






On 3/24/14, 9:16 AM, "Patrick M. Hausen" <hausen at punkt.de> wrote:

>Hi, all,
>
>in Saturday our Rancid started to complain that it could not log on to one
>of our core/uplink routers, anymore. Yet the system is generally alive and
>happily pushing packets - Nagios did not ring me about any link or service
>failing, so this came as a bit of a surprise.
>
>Turns out, SSH logins are not possible, anymore. Telnet and rsh work just
>fine. For each faile SSH login there is a line like this in the log:
>
>Mar 20 12:30:09.415: %AAA-3-ACCT_LOW_MEM_UID_FAIL: AAA unable to create
>UID for incoming calls due to insufficient processor memory
>
>Ah ... OK ... if it's failing in AAA, why does telnet still work? And the
>free memory
>doesn't look too bad, either:
>
>                Head    Total(b)     Used(b)     Free(b)   Lowest(b)
>Largest(b)
>Processor   477267E0   881661984   860385044    21276940    18235288
>20933772
>      I/O    8000000    67108864    21605604    45503260    45451176
>45501532
>
>          Processor memory
>
>Alloc PC        Size     Blocks      Bytes    What
>
>0x4014A218 0000000024 0000000001 0000000024    XDR: mfib pltf group
>0x4014A218 0000000028 0000000001 0000000028    XDR: mfib pltf group
>0x4014A218 0000000032 0000000001 0000000032    XDR: mfib pltf group
>0x401567F4 0000003808 0000000001 0000003808    Init
>0x4016D4BC 0000000024 0000000001 0000000024    Init
>...
>
>In the thousands of lines that follow, there are precisely 256 memory
>blocks
>allocated to the "SSH process". Is this a single process holding all that
>memory
>or are there 256 SSH processes, that are somewhat stuck/zombie because
>they are not terminated when the connection is closed?
>
>I admit that I rarely log off, but rather just close the window running
>my SSH connection.
>Bad admin. ;-) But any sane OS should timeout the TCP connection
>eventually and
>then terminate the process waiting on that socket.
>
>IOS version is 15.1(2)SY1 advanced enterprise.
>
>How can I proceed finding and eliminating the root cause? Rebooting the
>box to clean
>up is an option if planned ahead, but not a suitable permanent fix (i.e.
>rebooting regularly
>is out of the question).
>
>Thanks for any hints,
>Patrick
>-- 
>punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe
>Tel. 0721 9109 0 * Fax 0721 9109 100
>info at punkt.de       http://www.punkt.de
>Gf: Jürgen Egeling      AG Mannheim 108285
>
>
>
>
>_______________________________________________
>cisco-nsp mailing list  cisco-nsp at puck.nether.net
>https://puck.nether.net/mailman/listinfo/cisco-nsp
>archive at http://puck.nether.net/pipermail/cisco-nsp/