[c-nsp] Memory leak
Javier Castillo Alcibar
Javier.Castillo at alhambra-eidos.com
Thu Oct 20 02:34:38 EDT 2005
Hello,
I've been following Rodney's recommendations, and here is the guilty proccess: "*Dead* Process". It is holding more and more memory. When I'd reboot the router yesterday, it had 350Kbytes, and now, it has 5Mbytes.
The "show memory dead" shows a lot of lines, but no one is very clear for me... :)
Any ideas??
Regards,
Javier
router#show memory dead
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 64A12400 43965440 27493036 16472404 15900456 15029852
I/O 7400000 12582912 1806644 10776268 10281408 10444060
Processor memory
Address Bytes Prev Next Ref PrevF NextF Alloc PC what
64A12400 0000065540 00000000 64A22434 001 -------- -------- 6053BB7C MallocLite
64A22434 0000065540 64A12400 64A32468 001 -------- -------- 6053BB7C MallocLite
64A32468 0000065540 64A22434 64A4249C 001 -------- -------- 6053BB7C MallocLite
64A4249C 0000065540 64A32468 64A524D0 001 -------- -------- 6053BB7C MallocLite
64A524D0 0000065540 64A4249C 64A62504 001 -------- -------- 6053BB7C MallocLite
64A62504 0000065540 64A524D0 64A72538 001 -------- -------- 6053BB7C MallocLite
64A72538 0000130088 64A62504 64A92190 001 -------- -------- 6053BB7C MallocLite
64D5457C 0000000048 64D544B8 64D545DC 001 -------- -------- 6040B070 NameDB String
656158D4 0000000044 656156F8 65615930 001 -------- -------- 60C6603C Clock buffer
65615930 0000000056 656158D4 65615998 001 -------- -------- 60C6603C Clock buffer
6574EC2C 0000000028 6574EBC0 6574EC78 001 -------- -------- 60398854 AAA MI SG NAME
65AFBCD0 0000000844 65AFBA20 65AFC04C 001 -------- -------- 60486780 *Packet Header*
65B2D71C 0000000084 65B2D680 65B2D7A0 001 -------- -------- 60DB0F28 SWIDB_SB: NETBIOS Info
65B2D7A0 0000000084 65B2D71C 65B2D824 001 -------- -------- 60C6603C Clock buffer
65B383A0 0000000084 65B37FEC 65B38424 001 -------- -------- 60DB0F28 SWIDB_SB: NETBIOS Info
65C0A344 0000008196 65C0A0E8 65C0C378 001 -------- -------- 6284EB9C SSH Process
65C0CED4 0000001760 65C0C6A0 65C0D5E4 001 -------- -------- 62934648 IPv6 routing table
65C10330 0000002628 65C0D5E4 65C10DA4 001 -------- -------- 622CB1F4 IPv6 backup
65C884B8 0000088400 65C78484 65C9DE38 001 -------- -------- 6053BB7C MallocLite
65CCAC00 0000100392 65CBABCC 65CE3458 001 -------- -------- 6053BB7C MallocLite
65D0A7B0 0000105444 65D0277C 65D243C4 001 -------- -------- 6053BB7C MallocLite
65D53100 0000084236 65D4B0CC 65D67A3C 001 -------- -------- 6053BB7C MallocLite
65D7C8C4 0000088396 65D6C890 65D92240 001 -------- -------- 6053BB7C MallocLite
65DD7BC8 0000000052 65DD7B60 65DD7C2C 001 -------- -------- 608A03E0 IFINDEX hw subblock
65DD8070 0000000048 65DD7FFC 65DD80D0 001 -------- -------- 60417020 TTY timer block
65E06180 0000090340 65E0132C 65E1C294 001 -------- -------- 6053BB7C MallocLite
65E7FFB8 0000013448 65E7B164 65E83470 001 -------- -------- 622CB1C4 IPv6 adj
65E834C0 0000004012 65E83470 65E8449C 001 -------- -------- 60416FC4 TTY timers array
65E8704C 0000000080 65E86FA0 65E870CC 001 -------- -------- 60C6603C Clock buffer
65E9787C 0000069856 65E87848 65EA898C 001 -------- -------- 6053BB7C MallocLite
65EB74B8 0000002004 65EB6C84 65EB7CBC 001 -------- -------- 62934648 IPv6 Static RIB
65ECB580 0000086832 65EBB54C 65EE08E0 001 -------- -------- 6053BB7C MallocLite
65EFE998 0000102664 65EEE964 65F17AD0 001 -------- -------- 6053BB7C MallocLite
65F5F070 0000000084 65F5F004 65F5F0F4 001 -------- -------- 60DB0F28 SWIDB_SB: NETBIOS Info
65F5F0F4 0000000028 65F5F070 65F5F140 001 -------- -------- 608A03E0 IFINDEX hw subblock
65F5FAA4 0000000084 65F5F954 65F5FB28 001 -------- -------- 6260A83C crypto_handle_table
65F60064 0000000104 65F5FFCC 65F600FC 001 -------- -------- 60417020 TTY timer block
660088B4 0000000028 66003A60 66008900 001 -------- -------- 608A03E0 IFINDEX hw subblock
66008900 0000020004 660088B4 6600D754 001 -------- -------- 62934648 IPv6 routing table
66054E78 0000002388 66054E04 660557FC 001 -------- -------- 622DB8F0 IPv6 PDB
66055E80 0000000048 660557FC 66055EE0 001 -------- -------- 60417020 TTY timer block
660E368C 0000000120 660E2A80 660E3734 001 -------- -------- 60DB0F28 SWIDB_SB: NETBIOS Info
660E886C 0000028868 660E3A18 660EF960 001 -------- -------- 622CB194 IPv6 route
6612F8F4 0000073284 6611F8C0 66141768 001 -------- -------- 6053BB7C MallocLite
66163354 0000076888 66153320 66175FDC 001 -------- -------- 6053BB7C MallocLite
6619603C 0000082292 6618E008 661AA1E0 001 -------- -------- 6053BB7C MallocLite
6620BC08 0000000048 66208A18 6620BC68 001 -------- -------- 60417020 TTY timer block
6620BC68 0000000048 6620BC08 6620BCC8 001 -------- -------- 60417020 TTY timer block
6620BD14 0000000104 6620BCC8 6620BDAC 001 -------- -------- 60417020 TTY timer block
66229938 0000000312 6622324C 66229AA0 001 -------- -------- 62934648 IPv6 Static RIB
66242C50 0000008204 66242BF8 66244C8C 001 -------- -------- 622CB060 IPv6 Static
662452E8 0000000084 66245148 6624536C 001 -------- -------- 60DB0F28 SWIDB_SB: NETBIOS Info
66245470 0000002388 66245418 66245DF4 001 -------- -------- 622DB8F0 IPv6 PDB
66246128 0000000048 66246060 66246188 001 -------- -------- 60417020 TTY timer block
66269EE0 0000065540 66261EAC 66279F14 001 -------- -------- 6053BB7C MallocLite
66279F14 0000065540 66269EE0 66289F48 001 -------- -------- 6053BB7C MallocLite
66289F48 0000065540 66279F14 66299F7C 001 -------- -------- 6053BB7C MallocLite
66299F7C 0000065540 66289F48 662A9FB0 001 -------- -------- 6053BB7C MallocLite
662A9FB0 0000065540 66299F7C 662B9FE4 001 -------- -------- 6053BB7C MallocLite
662B9FE4 0000065540 662A9FB0 662CA018 001 -------- -------- 6053BB7C MallocLite
662CA018 0000065540 662B9FE4 662DA04C 001 -------- -------- 6053BB7C MallocLite
662DA04C 0000065540 662CA018 662EA080 001 -------- -------- 6053BB7C MallocLite
662EA080 0000065540 662DA04C 662FA0B4 001 -------- -------- 6053BB7C MallocLite
662FA0B4 0000065540 662EA080 6630A0E8 001 -------- -------- 6053BB7C MallocLite
6630A0E8 0000065540 662FA0B4 6631A11C 001 -------- -------- 6053BB7C MallocLite
6631A11C 0000065540 6630A0E8 6632A150 001 -------- -------- 6053BB7C MallocLite
6632A150 0000065540 6631A11C 6633A184 001 -------- -------- 6053BB7C MallocLite
6633A184 0000065540 6632A150 6634A1B8 001 -------- -------- 6053BB7C MallocLite
6634A1B8 0000065540 6633A184 6635A1EC 001 -------- -------- 6053BB7C MallocLite
6635A1EC 0000065540 6634A1B8 6636A220 001 -------- -------- 6053BB7C MallocLite
6636A220 0000065540 6635A1EC 6637A254 001 -------- -------- 6053BB7C MallocLite
6637A254 0000065540 6636A220 6638A288 001 -------- -------- 6053BB7C MallocLite
6638A288 0000065540 6637A254 6639A2BC 001 -------- -------- 6053BB7C MallocLite
6639A2BC 0000065540 6638A288 663AA2F0 001 -------- -------- 6053BB7C MallocLite
663AA2F0 0000065540 6639A2BC 663BA324 001 -------- -------- 6053BB7C MallocLite
663BA324 0000065540 663AA2F0 663CA358 001 -------- -------- 6053BB7C MallocLite
663CA358 0000065540 663BA324 663DA38C 001 -------- -------- 6053BB7C MallocLite
663DA38C 0000065540 663CA358 663EA3C0 001 -------- -------- 6053BB7C MallocLite
663EA3C0 0000065540 663DA38C 663FA3F4 001 -------- -------- 6053BB7C MallocLite
663FA3F4 0000065540 663EA3C0 6640A428 001 -------- -------- 6053BB7C MallocLite
6640A428 0000065540 663FA3F4 6641A45C 001 -------- -------- 6053BB7C MallocLite
6641A45C 0000065540 6640A428 6642A490 001 -------- -------- 6053BB7C MallocLite
6642A490 0000065540 6641A45C 6643A4C4 001 -------- -------- 6053BB7C MallocLite
6643A4C4 0000065540 6642A490 6644A4F8 001 -------- -------- 6053BB7C MallocLite
6644A4F8 0000065540 6643A4C4 6645A52C 001 -------- -------- 6053BB7C MallocLite
6645A52C 0000065540 6644A4F8 6646A560 001 -------- -------- 6053BB7C MallocLite
6646A560 0000065540 6645A52C 6647A594 001 -------- -------- 6053BB7C MallocLite
6647A594 0000065540 6646A560 6648A5C8 001 -------- -------- 6053BB7C MallocLite
6648A5C8 0000065540 6647A594 6649A5FC 001 -------- -------- 6053BB7C MallocLite
6649A5FC 0000065540 6648A5C8 664AA630 001 -------- -------- 6053BB7C MallocLite
664AA630 0000065540 6649A5FC 664BA664 001 -------- -------- 6053BB7C MallocLite
664BA664 0000065540 664AA630 664CA698 001 -------- -------- 6053BB7C MallocLite
664CA698 0000065540 664BA664 664DA6CC 001 -------- -------- 6053BB7C MallocLite
664DA6CC 0000065540 664CA698 664EA700 001 -------- -------- 6053BB7C MallocLite
664EA700 0000065540 664DA6CC 664FA734 001 -------- -------- 6053BB7C MallocLite
664FA734 0000065540 664EA700 6650A768 001 -------- -------- 6053BB7C MallocLite
6650A768 0000065540 664FA734 6651A79C 001 -------- -------- 6053BB7C MallocLite
6651A79C 0000065540 6650A768 6652A7D0 001 -------- -------- 6053BB7C MallocLite
6652A7D0 0000065540 6651A79C 6653A804 001 -------- -------- 6053BB7C MallocLite
6653A804 0000065540 6652A7D0 6654A838 001 -------- -------- 6053BB7C MallocLite
6654A838 0000065540 6653A804 6655A86C 001 -------- -------- 6053BB7C MallocLite
6655A86C 0000065540 6654A838 6656A8A0 001 -------- -------- 6053BB7C MallocLite
6656A8A0 0000065540 6655A86C 6657A8D4 001 -------- -------- 6053BB7C MallocLite
6657A8D4 0000065540 6656A8A0 6658A908 001 -------- -------- 6053BB7C MallocLite
6658A908 0000065540 6657A8D4 6659A93C 001 -------- -------- 6053BB7C MallocLite
6659A93C 0000065540 6658A908 665AA970 001 -------- -------- 6053BB7C MallocLite
------
5082012
I/O memory
Address Bytes Prev Next Ref PrevF NextF Alloc PC what
075CFDB0 0000000784 075CFA70 075D00F0 001 -------- -------- 604867CC *Packet Data*
------
832
======
5082844
Javier Castillo Alcíbar (2404)
+34 91 787 23 00 alhambra-eidos.com
-----Mensaje original-----
De: Rodney Dunn [mailto:rodunn at cisco.com]
Enviado el: lunes, 17 de octubre de 2005 19:25
Para: Javier Castillo Alcibar
CC: cisco-nsp at puck.nether.net
Asunto: Re: [c-nsp] Memory leak
On Mon, Oct 17, 2005 at 07:11:37PM +0200, Javier Castillo Alcibar wrote:
> This process, "TCP Protocols" is increasing, over the time, its
> Allocated memory, but nerver frees memory.......How can I know which
> process is deallocating those blocks?
You can't.
>
> I did some searchs in the bug toolkit, and I found some bugs related
> with memory leaks, but none of them seem to be related with my
> problem.....
You need to get the information I suggested to narrow it down.
Specifically the 'sh proc mem <PID>' over time.
>
> BTW, the "Memory Leak Detector" feature, can help me in this case?
You could try it:
http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps5207/products_feature_guide09186a008029cafe.html
Usually in the field we can figur it out just with the 'sh proc mem <PID>'
over time.
If you know it's TCP then try and focus on what TCP sessions are connecting and try to help narrow down the trigger. That will help TAC verify if and what bug it is.
>
> Javier.
>
>
>
>
>
>
> -----Mensaje original-----
> De: Rodney Dunn [mailto:rodunn at cisco.com] Enviado el: lunes, 17 de
> octubre de 2005 19:00
> Para: Javier Castillo Alcibar
> CC: cisco-nsp at puck.nether.net
> Asunto: Re: [c-nsp] Memory leak
>
> Just because a process allocates a block doesn't mean it is the only
> one that can free the same block.
>
> Monitor 'sh proc mem' over time and if the free amount keeps
> decreasing them compare 'sh process memory' to find the one that is
> holding more and more memory.
>
> Then monitor: sh proc mem <PID> for that process and open a TAC case
> for it so they can decode the functions for those PC's.
>
> Rodney
>
> On Mon, Oct 17, 2005 at 06:49:31PM +0200, Javier Castillo Alcibar wrote:
> > Hello list,
> >
> > I have two routers, 2610 and a 2651XM, with the same IOS: the latest
> > and wonderful 12.4(3a). Both routers have the same problem: they
> > have a memory leak, and I guess that the guilty process is one
> > called "TCP
> > Protocols":
> >
> > router>show proc memory | include TCP Protocols
> > 98 0 31173056 0 15120 7 7 TCP
> > Protocols
> > router>
> >
> > Notice how big is the allocated memory number, 31173056, and how
> > small
>
> > is the freed memory number, 0 :)
> >
> > Do you know what the hell is doing this process??
> >
> > Thx a lot.
> > regards,
> > Javier.
> >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list