[f-nsp] "Low pool memory, Pool(2-Global) below 5%" on Brocade MLX box
Chris Hellkvist
chris.hellkvist at googlemail.com
Tue Mar 8 12:17:48 EST 2016
Sure:
SSH at CRBR13-F> sh cam-partition brief
CAM partitioning profile: multi-service-2
Slot 1 XPP20SP 0:
# of CAM device = 3
Total CAM Size = 524288 entries (36Mbits)
IP: Raw Size 458752, User Size 458752(0 reserved)
Subpartition 0: Raw Size 2048, User Size 2048, (0 reserved)
Subpartition 1: Raw Size 411289, User Size 411289, (0 reserved)
Subpartition 2: Raw Size 37892, User Size 37892, (0 reserved)
Subpartition 3: Raw Size 5741, User Size 5741, (0 reserved)
Subpartition 4: Raw Size 1112, User Size 1112, (0 reserved)
IPv6: Raw Size 32768, User Size 16384(0 reserved)
Subpartition 0: Raw Size 2048, User Size 1024, (0 reserved)
Subpartition 1: Raw Size 27204, User Size 13602, (0 reserved)
Subpartition 2: Raw Size 2332, User Size 1166, (0 reserved)
Subpartition 3: Raw Size 512, User Size 256, (0 reserved)
Subpartition 4: Raw Size 384, User Size 192, (0 reserved)
SSH at CRBR13-F>sh default values
sys log buffers:50 mac age time:300 sec telnet sessions:5
ip arp age:10 min bootp relay max hops:4 ip ttl:64 hops
ip addr per intf:24
when multicast enabled :
igmp group memb.: 260 sec igmp query: 125 sec
when ospf enabled :
ospf dead:40 sec ospf hello:10 sec ospf retrans:5 sec
ospf transit delay:1 sec
when bgp enabled :
bgp local pref.:100 bgp keep alive:60 sec bgp hold:180 sec
bgp metric:10 bgp local as:1 bgp cluster id:0
bgp ext. distance:20 bgp int. distance:200 bgp local distance:200
when IS-IS enabled :
isis hello interval:10 sec isis hello multiplier:3
isis port metric:10 isis priority:64
isis csnp-interval:10 sec isis default-metric:10
isis distance:115 isis lsp-gen-interval:10 sec
isis lsp-interval:33 msec isis lsp-refresh-interval:900 sec
isis max-lsp-lifetime:1200 sec isis maximum-paths:4
isis retransmit-interval:5 sec isis spf-interval:5 sec
filter change update delay:10 sec
System Parameters Default Maximum Current Actual
Bootup Revertible
mac 32768 1048576 32768 32768
32768 Yes
vlan 512 4095 1024 1024
1024 No
spanning-tree 32 128 32 32
32 No
rstp 32 128 32 32
32 No
ip-arp 8192 65536 40000 40000
40000 No
ip-cache 102400 524288 440000 440000
440000 Yes
ip-route 102400 524288 440000 440000
440000 Yes
ip-subnet-port 24 128 24 24
24 No
virtual-interface 255 4095 800 800
800 No
vpls-mac 2048 262144 2048 2048
2048 Yes
vpls-num 512 4096 512 512
512 No
session-limit 8192 40960 8192 8192
8192 Yes
ip-filter-sys 4096 102400 4096 4096
4096 No
mgmt-port-acl-size 20 100 20 20
20 No
l2-acl-table-entries 64 256 64 64
64 No
ipv6-cache 32768 114688 16000 16000
16000 Yes
ipv6-route 32768 114688 16000 16000
16000 Yes
ip-vrf-route 5120 262144 5120 5120
5120 Yes
receive-cam 1024 16384 1024 1024
1024 No
ip-tunnels 256 8192 256 256
256 No
lsp-out-acl-cam 0 16384 0 0 0
No
trunk-num 128 256 128 128
128 No
config-file-size 8388608 32505856 8388608 8388608
8388608 No
ifl-cam 8192 81920 8192 8192
8192 No
ip-source-guard-cam 0 131072 0 0 0
No
ipv4-mcast-cam 4096 32768 4096 4096
4096 No
ipv6-mcast-cam 1024 8192 1024 1024
1024 No
mcast-vpls-cam 0 2048 0 0 0
Yes
ip-vrf 400 400 400 400
400 Yes
ipv6-vrf-route 128 16384 128 128
128 Yes
subnet-broadcast-acl-cam 0 4096 0 0 0
No
trunk-num-100g 8 64 8 8 8
No
openflow-flow-entries 0 65536 0 0 0
Yes
openflow-pvlan-entries 0 2048 0 0 0
Yes
openflow-unprotectedvlan 0 4096 0 0 0
Yes
ipv6-receive-cam 0 8192 0 0 0
No
ecmp-pram-block-size 32 32 32 32
32 Yes
tvf-lag-lb-fid-pool 0 2048 0 0 0
Yes
tvf-lag-lb-fid-group 4 16 4 4 4
Yes
openflow-group-select-bu 8 64 8 8 8
Yes
Openflow Slot Parameters slot Default
Maximum Actual Revertible
np-openflow-flow-entries : layer2or3 1 0
32768 0 No
np-openflow-flow-entries : layer2or3 2 0
32768 0 No
np-openflow-flow-entries : layer2or3 3 0
32768 0 No
np-openflow-flow-entries : layer2or3 4 0
32768 0 No
np-openflow-flow-entries : layer2or3 5 0
32768 0 No
np-openflow-flow-entries : layer2or3 6 0
32768 0 No
np-openflow-flow-entries : layer2or3 7 0
32768 0 No
np-openflow-flow-entries : layer2or3 8 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 1 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 2 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 3 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 4 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 5 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 6 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 7 0
32768 0 No
np-openflow-flow-entries : layer23ipv4 8 0
32768 0 No
2016-03-08 18:00 GMT+01:00 Eldon Koyle <ekoyle+puck.nether.net at gmail.com>:
> Could you also provide the output of 'show default values'?
>
> You may need to change your cam-partition profile.
>
> --
> Eldon
>
> On Tue, Mar 8, 2016 at 8:13 AM, Chris Hellkvist via foundry-nsp
> <foundry-nsp at puck.nether.net> wrote:
>> Dear all,
>>
>> recently we connected another upstream to a Brocade MLX box.
>>
>> We noticed that the MLX was not learning all routers from this BGP
>> peer both on the IPv4 and IPv6 session. Looking for the root cause we
>> found the following entries in the log of the box:
>> Mar 7 11:46:59:W:System: Low pool memory, Pool(2-Global) below 5%,
>> total memory(1073741824), available memory(53534720) on MP
>> Mar 7 11:46:58:W:System: Low physical memory, Pool(2-Global) below
>> 5%, total memory(1073741824), available memory(53235712) on MP
>>
>> I'm a bit worried because the system has not that many BGP sessions
>> active (around 8 while 4 peers send a full table that is reduces to
>> around 390k routes in the RIB/FIB using prefix lists).
>>
>> After removing the BGP peers the free memory went to arund 7%, however
>> prior to the configuration we have never had any warning regarding the
>> usual 10% threshold).
>>
>> Looking at the memory histogram i can see that most memory is consumed
>> by the following processes:
>> bgp 1221 106342047
>> bgp_io 3925 473596101
>>
>> Any idea how to reduce the memory footprint of bgp_io or to free up
>> more space by adjusting some system-max variables? Or any other
>> options than replacing the box with some new gear (luckily this will
>> happen end of 2016).
>>
>> Thanks,
>> Chris
>>
>> For the recordds, the modules we have in place and a sh ve of the system:
>> SSH at CRBR13-F#sh module
>> Module Status
>> Ports Starting MAC
>> M1 (left ):NI-MLX-MR Management Module Active
>> M2 (right):NI-MLX-MR Management Module Standby(Ready State)
>> F1: NI-X-SF Switch Fabric Module Active
>> F2: NI-X-SF Switch Fabric Module Active
>> F3: NI-X-SF Switch Fabric Module Active
>> S1: NI-MLX-10Gx4 4-port 10GbE Module
>> CARD_STATE_UP 4 001b.ed17.d000
>> S2: NI-MLX-10Gx4 4-port 10GbE Module
>> CARD_STATE_UP 4 001b.ed17.d030
>> S3: NI-MLX-10Gx4 4-port 10GbE Module
>> CARD_STATE_UP 4 001b.ed17.d060
>> S4: NI-MLX-10Gx4 4-port 10GbE Module
>> CARD_STATE_UP 4 001b.ed17.d090
>> S5: NI-MLX-10Gx4 4-port 10GbE Module
>> CARD_STATE_UP 4 001b.ed17.d0c0
>>
>> SSH at CRBR13-F#sh ve
>> System Mode: MLX
>> Chassis: NetIron 8-slot (Serial #: A60532F06T, Part #: 35549-000C)
>> NI-X-SF Switch Fabric Module 1 (Serial #: SA38070273, Part #: 35523-300C)
>> FE 1: Type fe200, Version 2
>> FE 3: Type fe200, Version 2
>> Switch Fabric Module 1 Up Time is 286 days 23 hours 8 minutes 55 seconds
>> NI-X-SF Switch Fabric Module 2 (Serial #: SA13070172, Part #: 35523-300C)
>> FE 1: Type fe200, Version 2
>> FE 3: Type fe200, Version 2
>> Switch Fabric Module 2 Up Time is 286 days 23 hours 8 minutes 55 seconds
>> NI-X-SF Switch Fabric Module 3 (Serial #: SA13080287, Part #: 35523-301A)
>> FE 1: Type fe200, Version 2
>> FE 3: Type fe200, Version 2
>> Switch Fabric Module 3 Up Time is 286 days 23 hours 8 minutes 55 seconds
>> ==========================================================================
>> SL M1: NI-MLX-MR Management Module Active (Serial #: S10526F06N, Part
>> #: 35524-103J):
>> Boot : Version 5.6.0T165 Copyright (c) 1996-2014 Brocade
>> Communications Systems, Inc.
>> Compiled on Sep 20 2013 at 16:42:38 labeled as xmprm05600
>> (516258 bytes) from boot flash
>> Monitor : Version 5.7.0T165 Copyright (c) 1996-2014 Brocade
>> Communications Systems, Inc.
>> Compiled on May 5 2014 at 10:41:24 labeled as xmb05700
>> (537647 bytes) from code flash
>> IronWare : Version 5.7.0cT163 Copyright (c) 1996-2014 Brocade
>> Communications Systems, Inc.
>> Compiled on Feb 25 2015 at 20:44:00 labeled as xmr05700c
>> (9651771 bytes) from Primary
>> Board ID : 00 MBRIDGE Revision : 37
>> 916 MHz Power PC processor 7447A (version 8003/0101) 166 MHz bus
>> 512 KB Boot Flash (AM29LV040B), 32 MB Code Flash (MT28F128J3)
>> 1024 MB DRAM INSTALLED
>> 1024 MB DRAM ADDRESSABLE
>> Active Management uptime is 286 days 23 hours 8 minutes 55 seconds
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp at puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
More information about the foundry-nsp
mailing list