[c-nsp] ASR9K - IPv6_leaf_P - Incorrect value
Alex
stellaresx at gmail.com
Fri Feb 14 02:21:18 EST 2020
Hello @ll,
Recently i encountered a drop of all BGP sessions on ASR9K due to low memory.
All of the neighbors (around 40) status changed to OOM
RP/0/RSP0/CPU0:#sh bgp ipv4 unicast summary
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 115785187 115785187 115785187 115785187 115785187 115785187
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
x.x.x.x 0 12345 4374395 186421 0 0 0 00:42:43 Idle (OOM)
The log stated:
RP/0/RSP0/CPU0:Jan 9 18:33:19.245 : wdsysmon[451]: %HA-HA_WD-4-TOP_MEMORY_USER_WARNING : 0: Process Name: bgp[1052], pid: 581965, Heap usage 810544 Kbytes, Virtual Shared memor
y usage: 56144 Kbytes.
RP/0/RSP0/CPU0:Jan 9 18:33:19.245 : wdsysmon[451]: %HA-HA_WD-4-TOP_MEMORY_USER_WARNING : 1: Process Name: mibd_entity[318], pid: 2167021, Heap usage 611188 Kbytes, Virtual Shar
ed memory usage: 26132 Kbytes.
RP/0/RSP0/CPU0:Jan 9 18:33:19.245 : wdsysmon[451]: %HA-HA_WD-4-TOP_MEMORY_USER_WARNING : 2: Process Name: ipv4_rib[1149], pid: 577855, Heap usage 312280 Kbytes, Virtual Shared
memory usage: 28516 Kbytes.
RP/0/RSP0/CPU0:Jan 9 18:33:19.245 : wdsysmon[451]: %HA-HA_WD-4-TOP_MEMORY_USER_WARNING : 3: Process Name: ipv6_rib[1150], pid: 577858, Heap usage 53604 Kbytes, Virtual Shared m
emory usage: 28584 Kbytes.
RP/0/RSP0/CPU0:Jan 9 18:33:19.245 : wdsysmon[451]: %HA-HA_WD-4-TOP_MEMORY_USER_WARNING : 4: Process Name: eth_server[60], pid: 53292, Heap usage 33420 Kbytes, Virtual Shared me
mory usage: 32568 Kbytes.
RP/0/RSP0/CPU0:Jan 9 18:33:20.251 : bgp[1052]: %ROUTING-BGP-5-LOWMEM_RESET : neighbor x.x.x.x reset due to SEVERE low memory condition - use 'clear bgp x.x.x.x' to
re-enable
RP/0/RSP0/CPU0:Jan 9 18:33:20.251 : bgp[1052]: %ROUTING-BGP-5-ADJCHANGE : neighbor x.x.x.x Down - Shutdown during SEVERE low memory condition (CEASE notification sent - o
ut of resource) (VRF: default) (AS: xxx)
RP/0/RSP1/CPU0:Jan 9 18:33:20.252 : bgp[1052]: %ROUTING-BGP-5-NBR_NSR_DISABLED_STANDBY : NSR disabled on neighbor x.x.x.x on standby RP due to Shutdown during SEVERE low
memory condition (VRF: default)
RP/0/RSP0/CPU0:Jan 9 18:33:34.226 : fib_mgr[221]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of th
e system memory
LC/0/2/CPU0:Jan 9 18:39:39.714 : fib_mgr[177]: %PLATFORM-PLAT_FIB-4-WARN_STR_PRM_OOR : The utilization of the object IPv4_LEAF_P drops below 83% on all NPs
LC/0/0/CPU0:Jan 9 18:45:10.210 : fib_mgr[177]: %PLATFORM-PLAT_FIB-4-WARN_STR_PRM_OOR : The utilization of the object IPv6_LEAF_P drops below 83% on all NPs
RP/0/RSP0/CPU0:(admin)#sh memory summary
Thu Jan 9 19:21:03.837 CEST
Physical Memory: 4096M total (565M available)
Application Memory : 3897M (565M available)
Image: 69M (bootram: 69M)
Reserved: 128M, IOMem: 2028M, flashfsys: 0
Total shared window: 191M
The IOS-XR installed is 5.3.4-SP10-1.0.0
While investigating we changed profile from L3 to L3XL inorder to be able to install more routes:
RP/0/RSP0/CPU0:#show cef platform resource summary loc 0/0/cpu0
Thu Jan 16 13:45:23.052 CEST
OBJECT USED MAX AVAILABLE
RPF_STRICT 0 65536 65536
IPv4_LEAF_P 780806 1314000 533194
IPv6_LEAF_P 389903 656500 266597
LEAF 826182 1460000 633818
TX_ADJ 826182 1460000 633818
NR_LDI 826182 1460000 633818
TE_NH_ADJ 0 32768 32768
RX_ADJ 28 32768 32740
R_LDI 13266 65536 52270
L2VPN_LDI 0 2048 2048
EXT_LSPA 0 0 0
IPv6_LL_LEAF_P 0 262144 262144
The problem is that we are unable to explain the value of 389903 concerning IPv6_LEAF_P. We tried dropping all IPv6 session but the value remained unchanged.
We decided to remove all BGP sessions:
RP/0/RSP0/CPU0:#sh cef platform resource summary location 0/0/CPU0
Fri Feb 14 07:25:27.917 CEST
OBJECT USED MAX AVAILABLE
RPF_STRICT 0 65536 65536
IPv4_LEAF_P 800 1314000 1313200
IPv6_LEAF_P 21 656500 656479 <—— Used=21. Value is correct
LEAF 40761 1460000 1419239
TX_ADJ 40761 1460000 1419239
NR_LDI 40761 1460000 1419239
TE_NH_ADJ 0 32768 32768
RX_ADJ 28 32768 32740
R_LDI 13265 65536 52271
L2VPN_LDI 0 2048 2048
EXT_LSPA 0 0 0
IPv6_LL_LEAF_P 0 262144 262144
RP/0/RSP0/CPU0:PA7#show cef platform resource location 0/0/CPU0
IPV6_LEAF_P usage is same on all NPs
NP: 0 struct 24: IPV6_LEAF_P (maps to ucode stru = 55)
Used Entries: 21 Max Entries: 656500
Adding one BGP IPv4 session:
RP/0/RSP0/CPU0:#sh cef platform resource summary location 0/0/CPU0
Fri Feb 14 07:34:57.943 CEST
OBJECT USED MAX AVAILABLE
RPF_STRICT 0 65536 65536
IPv4_LEAF_P 807653 1314000 506310
IPv6_LEAF_P 403345 656500 253155
LEAF 847658 1460000 612342
TX_ADJ 847658 1460000 612342
NR_LDI 847658 1460000 612342
TE_NH_ADJ 0 32768 32768
RX_ADJ 28 32768 32740
R_LDI 13266 65536 52270
L2VPN_LDI 0 2048 2048
EXT_LSPA 0 0 0
IPv6_LL_LEAF_P 0 262144 262144
But the following output states completely a different value:
RP/0/RSP0/CPU0:#show cef platform resource location 0/0/CPU0
Fri Feb 14 07:35:36.556 CEST
Node: 0/0/CPU0
----------------------------------------------------------------
RPF_STRICT usage is same on all NPs
NP: 0 struct 9: RPF_STRICT (maps to ucode stru = 15)
Used Entries: 0 Max Entries: 65536
-------------------------------------------------------------
IPV4_LEAF_P usage is same on all NPs
NP: 0 struct 23: IPV4_LEAF_P (maps to ucode stru = 54)
Used Entries: 807653 Max Entries: 1314000
-------------------------------------------------------------
IPV6_LEAF_P usage is same on all NPs
NP: 0 struct 24: IPV6_LEAF_P (maps to ucode stru = 55)
Used Entries: 21 Max Entries: 656500
-------------------------------------------------------------
Can someone help on this please?
We have another ASR9K with an older version of IOS-XR (4.2.1) which is actually configured with profile L3 only and have the same number of routes of the router version 5.3.4 but it never encountered any problem.
Thanks.
Regards,
Alex
More information about the cisco-nsp
mailing list