[c-nsp] cisco-nsp Digest, Vol 210, Issue 10
Catherine Trebnick
ctrebnick at gmail.com
Wed May 27 10:22:30 EDT 2020
On Wed, May 27, 2020 at 6:44 AM <cisco-nsp-request at puck.nether.net> wrote:
> Send cisco-nsp mailing list submissions to
> cisco-nsp at puck.nether.net
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> or, via email, send a message with subject or body 'help' to
> cisco-nsp-request at puck.nether.net
>
> You can reach the person managing the list at
> cisco-nsp-owner at puck.nether.net
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of cisco-nsp digest..."
>
>
> Today's Topics:
>
> 1. BGP router process using way more memory on one system
> (Drew Weaver)
> 2. Re: BGP router process using way more memory on one system
> (Nick Hilliard)
> 3. Re: ASR9001 BGP scaling and memory shortage (Vladimir Troitskiy)
> 4. asr-903 + policy-map control (Sean Watkins)
> 5. ASR1001 netflow 32 bits ASN (Alarig Le Lay)
> 6. Re: ASR1001 netflow 32 bits ASN (Alarig Le Lay)
> 7. IOS-XR IS-IS authentication (Eric Van Tol)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 24 May 2020 18:20:50 +0000
> From: Drew Weaver <drew.weaver at thenap.com>
> To: "'cisco-nsp at puck.nether.net'" <cisco-nsp at puck.nether.net>
> Subject: [c-nsp] BGP router process using way more memory on one
> system
> Message-ID: <7038e9f13a004ff3957d81ae60cf7d0a at thenap.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Hello,
>
> We have two routers that have a mirrored configuration. Peers, BGP
> configuration, everything. Exactly the same [except for IP addresses]
>
> One of the routers BGP router process is holding 617576024. The other is
> holding 577596716.
>
> The one that is holding more appears to be suffering from an out of memory
> condition.
>
> I am planning on rebooting it but before I do is there any known way of
> freeing up enough memory to allow basic virtual exec processes to execute?
>
> I've tried basic things like shutting down BGP peers, etc but even though
> the total memory that BGP says it's using goes down.. it still won't free
> up the memory.
>
> Thanks in advance.
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 25 May 2020 08:50:31 +0100
> From: Nick Hilliard <nick at foobar.org>
> To: Drew Weaver <drew.weaver at thenap.com>
> Cc: "'cisco-nsp at puck.nether.net'" <cisco-nsp at puck.nether.net>
> Subject: Re: [c-nsp] BGP router process using way more memory on one
> system
> Message-ID: <a702ad00-c617-ce0f-c5a2-414f420628b5 at foobar.org>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Drew Weaver wrote on 24/05/2020 19:20:
> > We have two routers that have a mirrored configuration. Peers, BGP
> > configuration, everything. Exactly the same [except for IP
> > addresses]
> >
> > One of the routers BGP router process is holding 617576024. The other
> > is holding 577596716.
> >
> > The one that is holding more appears to be suffering from an out of
> > memory condition.
>
> There were a couple of releases where the ipv4_rib process had a
> persistent memory leak. Try this:
>
> Router# admin process restart ipv4_rib
>
> This is non service affecting - restarting the process temporarily stops
> FIB reprogramming, then does a full RIB reload from all RIB sources,
> then does a FIB check across the device. I.e. it's safer to do this than
> to hobble along with OOM errors.
>
> Nick
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 25 May 2020 23:00:13 +0500
> From: Vladimir Troitskiy <ruthenate at gmail.com>
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] ASR9001 BGP scaling and memory shortage
> Message-ID:
> <CAOq6j68n=zFFkY0+v=o+Rd85AGjeou=
> 2rN_1yOnkXDRWg1sF2A at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Hello everyone,
>
> Other list members have a significantly lower memory usage for a BGP
> process and a shmwin on ASR9001 routers with more sessions/routes in GRT.
>
> Saku Ytti has suggested me some useful notes which I would like to mention
> as a summary for this thread:
> - one could use 'hw-module profile scale l3xl' in admin mode to increase an
> RLIMIT for a BGP process, even on Typhoon-based platforms (not only on
> Trident-based ones as I thought);
> - a shmwin shortage is probably caused by per-prefix label mode, the per-ce
> mode will be much more scalable. We use the per-prefix mode because of BGP
> PIC limitations, but maybe it's time to reconsider the feature-set used.
>
> ??, 19 ??? 2020 ?. ? 20:09, Vladimir Troitskiy <ruthenate at gmail.com>:
>
> > Hello everyone,
> >
> > ASR9001 has some memory usage limits:
> > - 1658M for a BGP process on a RSP
> > - 1536M for a shared memory window on a LC
> > Those limits seems to be unconfigurable.
> >
> > Has anybody experienced any issues with these limits on high-loaded
> > ASR9001 boxes?
> > We have a surprisingly high memory usage while the typical router setup
> is
> > pretty lightweight - 4-5 full feeds (couple of upstreams and RRs). The
> only
> > probably uncommon thing is we use "Internet in a VRF" approach.
> >
> > #show processes memory detail location 0/RSP0/CPU0
> >> Tue May 19 19:39:12.592 Ural
> >> JID Text Data Stack Dynamic Dyn-Limit Shm-Tot
> >> Phy-Tot Process
> >> ------ ---------- ---------- ---------- ---------- ---------- ----------
> >> ---------- -------
> >> 1054 1M 5M 516K 1485M 1658M 76M
> >> 1491M bgp
> >>
> >
> > #show memory summary location 0/0/CPU0
> >>
> > node: node0_0_CPU0
> >> ------------------------------------------------------------------
> >> Physical Memory: 8192M total
> >> Application Memory : 7988M (3811M available)
> >> Image: 75M (bootram: 75M)
> >> Reserved: 128M, IOMem: 0, flashfsys: 0
> >> Total shared window: 1327M
> >>
> >
> > We have already had FIB inconsistency issues due to SHMWIN exhaustion
> > despite the fact the total prefix amount was far from the platform limit
> > (4M):
> >
> >> fib_mgr[184]: %OS-SHMWIN-3-ALLOC_ARENA_FAILED : SHMWIN: Failed to
> >> allocate new arena from the server : 'SHMWIN_SVR' detected the 'fatal'
> >> condition 'VM is exhausted or totally fragmented'
> >> fib_mgr[184]: %ROUTING-FIB-3-ASSERT_RL : FIB internal inconsistency
> >> detected
> >> fib_mgr[184]: %ROUTING-FIB-3-PD_FAIL : FIB platform error:
> >> fib_leaf_insert 5204 Cannot insert leaf
> >>
> >
> > What are practical limits for BGP scaling on ASR9001 boxes? Could anyone
> > share a memory usage stats?
> > --
> > Best regards,
> > Vladimir Troitsky
> >
>
> --
> Best regards,
> Vladimir Troitsky
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 26 May 2020 09:30:43 -0600
> From: Sean Watkins <sean.watkins at gmail.com>
> To: cisco-nsp at puck.nether.net
> Subject: [c-nsp] asr-903 + policy-map control
> Message-ID:
> <
> CAKwiYyoMwAH53GikJdpg1E8YpP9dgA1XFkkY0MYHWVeCEj8tOw at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Has anyone here got a asr-903 running, and has policy-map type control
> going? Curious if it supports it.
>
> I've been experimenting with ISG (like everyone else :) -- and it
> seems like ASR-903 has most of the ISG features, but seems to be
> lacking the control type of policy-maps? Feature navigator on CCO is
> so broken I can't seem todo any research now.
>
>
> Ie:
> ASR-903(config)#policy-map ?
> WORD policy-map name
>
> ASR-903(config)#policy-map
>
>
> this is on
>
> Cisco IOS XE Software, Version 03.16.02a.S - Extended Support Release
> Cisco IOS Software, ASR900 Software
> (PPC_LINUX_IOSD-UNIVERSALK9_NPE-M), Version 15.5(3)S2a, RELEASE
> SOFTWARE (fc1)
> Technical Support: http://www.cisco.com/techsupport
> Copyright (c) 1986-2016 by Cisco Systems, Inc.
> Compiled Thu 18-Feb-16 23:52 by mcpre
>
>
>
> --
> --
> Sean Watkins
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 26 May 2020 17:54:49 +0200
> From: Alarig Le Lay <alarig at grifon.fr>
> To: cisco-nsp at puck.nether.net
> Subject: [c-nsp] ASR1001 netflow 32 bits ASN
> Message-ID: <20200526155449.7f6jdcflkoxeztt5 at mew.swordarmor.fr>
> Content-Type: text/plain; charset=utf-8
>
> Hi,
>
> I?m trying to setup flowspec export to an AS-Stats from an ASR1001
> running IOS XE 03.16.06.S
>
> If I?m using original-input template I get AS23456 instead of the real
> ASN, e.g.
>
> Flow 4
> ipv6FlowLabel: 74969
> IPv6 Extension Headers: 0x00000000
> SrcAddr: 2a03:7220:8083:a600::1
> DstAddr: 2a00:5884:8218::1
> Protocol: UDP (17)
> IP ToS: 0x00
> SrcPort: 43805 (43805)
> DstPort: 53 (53)
> TCP Flags: 0x00
> 00.. .... = Reserved: 0x0
> ..0. .... = URG: Not used
> ...0 .... = ACK: Not used
> .... 0... = PSH: Not used
> .... .0.. = RST: Not used
> .... ..0. = SYN: Not used
> .... ...0 = FIN: Not used
> SrcAS: 23456
> SrcMask: 32
> InputInt: 8
> DstAS: 0
> NextHop: 2a00:5884:0:6::8
> DstMask: 48
> OutputInt: 11
> Direction: Ingress (0)
> SamplerID: 0
> Octets: 103
> Packets: 1
> [Duration: 0.000000000 seconds (switched)]
> StartTime: 2608346.732000000 seconds
> EndTime: 2608346.732000000 seconds
>
> I tried to set my own template (the same as original-input without the
> ASN info) with this config:
>
> asbr01#sh run | sec NETFLOW
> flow record FR-NETFLOW-ASSTATS-IPv4
> match ipv4 tos
> match ipv4 protocol
> match ipv4 source address
> match ipv4 destination address
> match transport source-port
> match transport destination-port
> match interface input
> match flow sampler
> collect routing next-hop address ipv4
> collect ipv4 source mask
> collect ipv4 destination mask
> collect transport tcp flags
> collect interface output
> collect counter bytes
> collect counter packets
> collect timestamp sys-uptime first
> collect timestamp sys-uptime last
> flow exporter FE-NETFLOW-ASSTATS
> destination 89.234.186.43
> source GigabitEthernet0/0/1.33
> transport udp 9000
> template data timeout 300
> flow monitor FM-NETFLOW-ASSTATS-IPv4
> exporter FE-NETFLOW-ASSTATS
> cache timeout active 30
> record FR-NETFLOW-ASSTATS-IPv4
>
> But I had the following error message when I added `record
> FR-NETFLOW-ASSTATS-IPv4` (even before applying it to the interface).
>
> %FMFP-3-OBJ_DWNLD_TO_CPP_FAILED: SIP0:
> fman_fp_image: [FNF Object] type:MON_FDEF_BIND
> name:FM-NETFLOW-ASSTATS-IPv4-0-FR-NETFLOW-ASSTATS-IPv4-1197725476
> fnf-id:2000012 real-id:12 info:mon-id:2000007 flow-id:2000012 download
> to CPP failed
>
> Since then, even the original-input template isn?t working for IPv4.
> I didn?t test my personnal templatre on IPv6 and original-input is
> working on it for now.
>
> I only found something about QoS for FMFP-3-OBJ_DWNLD_TO_CPP_FAILED.
>
> Is it something known?
>
> Regards,
> --
> Alarig Le Lay
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 26 May 2020 18:25:25 +0200
> From: Alarig Le Lay <alarig at grifon.fr>
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] ASR1001 netflow 32 bits ASN
> Message-ID: <20200526162525.wykzbyqm6p4axxcr at mew.swordarmor.fr>
> Content-Type: text/plain; charset=utf-8
>
> I forgot to say it in my previous mail, but I also tried to add the
> 4-octet option, but I also have an error:
>
> %FMANRP_NETFLOW-3-INVALIDFLOWDEFCPP: CPP Flow definition can not be
> created 49
> -Traceback= 1#315780af4aa185802629fb38078844ee :7FA612E86000+F81236B
> :7FA612E86000+F811077 fnf_config:7FA5EA211000+1D534
> %FMFP-3-OBJ_DWNLD_TO_CPP_FAILED: SIP0: fman_fp_image: [FNF Object]
> type:MON_FDEF_BIND
> name:FM-NETFLOW-ASSTATS-IPv4-0-FR-NETFLOW-ASSTATS-IPv4-1197725476
> fnf-id:2000012 real-id:12 info:mon-id:2000007 flow-id:2000012 download to
> CPP failed
>
> Regards,
> --
> Alarig
>
>
> ------------------------------
>
> Message: 7
> Date: Wed, 27 May 2020 11:43:26 +0000
> From: Eric Van Tol <eric at atlantech.net>
> To: "cisco-nsp at puck.nether.net" <cisco-nsp at puck.nether.net>
> Subject: [c-nsp] IOS-XR IS-IS authentication
> Message-ID: <8515D231-FD7D-40AB-9462-4BDC045D727C at atlantech.net>
> Content-Type: text/plain; charset="utf-8"
>
> Sorry if this is a duplicate ? Outlook chose the ?bounces? address as the
> one to send to and I didn?t notice.
>
> Hi all,
> I?m testing out an NCS540 for use in our network and this is my first
> foray into IOS-XR. We have a mix of Juniper and Cisco IOS/IOS-XE devices
> that the NCS needs to interoperate with. I?m having some minor trouble with
> IS-IS authentication and it?s kind of driving me nuts because I can?t get
> IS-IS to come up when authentication is configured. I keep getting this
> error:
>
> BAD P2P IIH rcvd from TenGigE0/0/0/19 SNPA 5c5e.abde.1e00: dropped because
> cryptographic password mismatch
>
> Seems pretty obvious, but my keychain key password is configured and
> verified to match on both sides:
>
> key chain isis-chain
> key 1
> accept-lifetime 00:00:00 january 01 1993 infinite
> key-string password <password>
> send-lifetime 00:00:00 january 01 1993 infinite
> cryptographic-algorithm HMAC-MD5
> !
> accept-tolerance infinite
>
> I?ve tried both MD5 and HMAC-MD5, neither works. Here is my IS-IS config
> on the NCS540:
>
> router isis rtr1
> set-overload-bit on-startup wait-for-bgp
> is-type level-2-only
> net 49.0001.1071.3820.2192.00
> log adjacency changes
> lsp-mtu 1497
> lsp-password keychain isis-chain
> address-family ipv4 unicast
> metric-style wide level 2
> !
> address-family ipv6 unicast
> metric-style wide level 2
> single-topology
> !
> interface Loopback1
> passive
> address-family ipv4 unicast
> !
> address-family ipv6 unicast
> !
> !
> interface TenGigE0/0/0/19
> circuit-type level-2-only
> point-to-point
> hello-password keychain isis-chain
> address-family ipv4 unicast
> metric 3500
> !
> address-family ipv6 unicast
> metric 3500
> !
> !
>
> traceoptions on the Juniper shows something similar:
>
> ERROR: IIH from 1071.3820.2192 on xe-0/0/0.0 failed authentication
>
> Here?s the Juniper key config and isis stanza:
>
> authentication-key-chains {
> key-chain isis-chain {
> key 1 {
> secret "<password>"; ## SECRET-DATA
> start-time "1993-1-1.00:00:00 +0000";
> algorithm md5;
> }
> }
> }
> protocols {
> isis {
> level 1 disable;
> level 2 {
> authentication-key-chain isis-chain;
> wide-metrics-only;
> }
> interface xe-0/0/0.0 {
> point-to-point;
> level 2 {
> metric 3500;
> hello-authentication-key-chain isis-chain;
> }
> level 1 disable;
> }
> }
>
> I know it?s got to be something simple, but it?s not clicking for me
> today. It seems like any step forward I take with IOS-XR, I end up taking
> two steps back on the next thing that ?just works? everywhere else.
>
> -evt
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> cisco-nsp mailing list
> cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
>
>
> ------------------------------
>
> End of cisco-nsp Digest, Vol 210, Issue 10
> ******************************************
>
--
Best Regards,
Catharine Trebnick
(M) 612.419.1686
http://www.linkedin.com/in/trebnick
Follow me on twitter @ctrebnick
More information about the cisco-nsp
mailing list