[c-nsp] con0 and XRemote problem which ends with serious memory issues

[Antonio Soares]

Hello Team,

I found this old question since I am getting exactly the same problem. The
6500 is running 122-33.SXH8b.

Apart from the reboot or possibly from the Supervisor Switchover, anyone
knows who to solve this ?

It was the first time I heard about Cisco IOS running XRemote. I was able to
reproduce this in the lab with 12.2.18SFX17b.





Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
amsoares at netcabo.pt
http://www.ccie18473.net



-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Andy B.
Sent: quinta-feira, 12 de Maio de 2011 14:39
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] con0 and XRemote problem which ends with serious memory
issues

Hi,

I'm facing an issue with a 6500 running SXI5 that eventually ends up eating
all memory and reload is the only way to solve it:

#who
    Line       User       Host(s)              Idle       Location
   0 con 0                XRemote: 24 clients  01:54:29


This box has been up for 12 hours, and the number of XRemote increases over
the day, while no console is attached to con 0.

I tried to clear line con 0 to no avail and then I also tried to identify
the TCB by using sh tcp brief and then clearing the TCB.

Here is an example:

5B09CF34  x.x.189.1.8000          x.218.199.147.2055          CLOSED

#sh tcp tcb 5B09CF34
Connection state is CLOSED, I/O status: 8, unread input bytes: 9 Mininum
incoming TTL 0, Outgoing TTL 255 Local host: x.x.189.1, Local port: 8000
Foreign host: x.218.199.147, Foreign port: 2055

Enqueued packets for retransmit: 0, input: 1  mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x289A8DC):
Timer          Starts    Wakeups            Next
Retrans             1          0             0x0
TimeWait            0          0             0x0
AckHold             1          1             0x0
SendWnd             0          0             0x0
KeepAlive           0          0             0x0
GiveUp              0          0             0x0
PmtuAger            0          0             0x0
DeadWait            1          0       0x291DA40

iss: 2341161453  snduna: 2341161454  sndnxt: 2341161454     sndwnd:  65535
irs: 1925376677  rcvnxt: 1925376687  rcvwnd:       4119  delrcvwnd:      0

SRTT: 52 ms, RTTO: 1968 ms, RTV: 1916 ms, KRTT: 0 ms
minRTT: 416 ms, maxRTT: 416 ms, ACK hold: 200 ms
Flags: passive open, higher precedence, retransmission timeout
  path mtu capable

Datagrams (max data segment is 1460 bytes):
Rcvd: 4 (out of order: 0), with data: 1, total data bytes: 9
Sent: 4 (retransmit: 0), with data: 0, total data bytes: 0


(Note that all those XRemote sessions seem to be on port 8000, but I cannot
explain why)

#clear tcp tcb 5B09CF34
[confirm]
 [OK]


It does not disappear from the list and I tried to clear it numerous times,
and eventually it disappeared.



Furthermore, while this goes on, this is spamming my logs:

May 12 15:32:05.660 CEST: %SYS-2-GETBUF: Bad getbuffer, bytes= -10 -Process=
"Exec", ipl= 0, pid= 119 -Traceback= 42377590 422007D0 422013A4 42200A60
42203984 4052E5B0 4233FCAC 4055C178 41392EC8 41392EB4 May 12 15:32:05.928
CEST: %SYS-2-GETBUF: Bad getbuffer, bytes= -10 -Process= "Exec", ipl= 0,
pid= 119 -Traceback= 42377590 422007D0 422013A4 42200A60 42203984 4052E5B0
4233FCAC 4055C178 41392EC8 41392EB4 May 12 15:32:06.180 CEST: %SYS-2-GETBUF:
Bad getbuffer, bytes= -10 -Process= "Exec", ipl= 0, pid= 119 -Traceback=
42377590 422007D0 422013A4 42200A60 42203984 4052E5B0 4233FCAC 4055C178
41392EC8 41392EB4 May 12 15:32:06.432 CEST: %SYS-2-GETBUF: Bad getbuffer,
bytes= -10 -Process= "Exec", ipl= 0, pid= 119 -Traceback= 42377590 422007D0
422013A4 42200A60 42203984 4052E5B0 4233FCAC 4055C178 41392EC8 41392EB4 May
12 15:32:06.684 CEST: %SYS-2-GETBUF: Bad getbuffer, bytes= -10 -Process=
"Exec", ipl= 0, pid= 119 -Traceback= 42377590 422007D0 422013A4 42200A60
42203984 4052E5B0 4233FCAC 4055C178 41392EC8 41392EB4 May 12 15:32:06.936
CEST: %SYS-2-GETBUF: Bad getbuffer, bytes= -10 -Process= "Exec", ipl= 0,
pid= 119 -Traceback= 42377590 422007D0 422013A4 42200A60 42203984 4052E5B0
4233FCAC 4055C178 41392EC8 41392EB4 May 12 15:32:07.200 CEST: %SYS-2-GETBUF:
Bad getbuffer, bytes= -10 -Process= "Exec", ipl= 0, pid= 119 -Traceback=
42377590 422007D0 422013A4 42200A60 42203984 4052E5B0 4233FCAC 4055C178
41392EC8 41392EB4



The main problem is that the number of XRemote sessions is going up to
128 and it is slowly eating up all available memory until it is all used up
and you are forced to reload. Last time this happened, memory was full in
roughly 6 weeks.

I have no service and no ACL using port 8000.

When I reload the box it's good for a while, and then it starts over again.


Has anyone seen this behaviour? How can this be solved without reloading
every once and a while?

Thanks.

Andy
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/