[c-nsp] ASR9001 BGP scaling and memory shortage

[Alexandr Gurbo]

Hello Vladimir,

What version IOS XR are you using?
We are doesn't have problems with FIB inconsistency. IOS XR 6.6.3.

#show processes memory detail location 0/RSP0/CPU0
Wed May 20 09:09:23.240 MSK
JID    Text       Data       Stack      Dynamic    Dyn-Limit  Shm-Tot    Phy-Tot    Process
------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- -------
1087           1M        10M       624K       818M      1658M       218M       829M  bgp

#show memory summary location 0/0/CPU0
Wed May 20 09:10:34.206 MSK
node:      node0_0_CPU0
--------------------------------------
Physical Memory: 8192M total
 Application Memory : 7985M (4258M available)
 Image: 78M (bootram: 78M)
 Reserved: 128M, IOMem: 0, flashfsys: 0
 Total shared window: 495M


On Tue, 19 May 2020 20:09:52 +0500
Vladimir Troitskiy <ruthenate at gmail.com> wrote:

> Hello everyone,
> 
> ASR9001 has some memory usage limits:
> - 1658M for a BGP process on a RSP
> - 1536M for a shared memory window on a LC
> Those limits seems to be unconfigurable.
> 
> Has anybody experienced any issues with these limits on high-loaded ASR9001
> boxes?
> We have a surprisingly high memory usage while the typical router setup is
> pretty lightweight - 4-5 full feeds (couple of upstreams and RRs). The only
> probably uncommon thing is we use "Internet in a VRF" approach.
> 
> #show processes memory detail location 0/RSP0/CPU0
> > Tue May 19 19:39:12.592 Ural
> > JID    Text       Data       Stack      Dynamic    Dyn-Limit  Shm-Tot
> >  Phy-Tot    Process
> > ------ ---------- ---------- ---------- ---------- ---------- ----------
> > ---------- -------
> > 1054           1M         5M       516K      1485M      1658M        76M
> >    1491M  bgp
> >
> 
> #show memory summary location 0/0/CPU0
> >
> node:      node0_0_CPU0
> > ------------------------------------------------------------------
> > Physical Memory: 8192M total
> >  Application Memory : 7988M (3811M available)
> >  Image: 75M (bootram: 75M)
> >  Reserved: 128M, IOMem: 0, flashfsys: 0
> >  Total shared window: 1327M
> >
> 
> We have already had FIB inconsistency issues due to SHMWIN exhaustion
> despite the fact the total prefix amount was far from the platform limit
> (4M):
> 
> > fib_mgr[184]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to allocate
> > new arena from the server : 'SHMWIN_SVR' detected the 'fatal' condition 'VM
> > is exhausted or totally fragmented'
> > fib_mgr[184]: %ROUTING-FIB-3-ASSERT_RL : FIB internal inconsistency
> > detected
> > fib_mgr[184]: %ROUTING-FIB-3-PD_FAIL : FIB platform error: fib_leaf_insert
> > 5204 Cannot insert leaf
> >
> 
> What are practical limits for BGP scaling on ASR9001 boxes? Could anyone
> share a memory usage stats?
> -- 
> Best regards,
> Vladimir Troitsky
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


-- 
Alexandr Gurbo <gurbo at golas.ru>