[f-nsp] Max BGP peers BigIron RX4

Sat Mar 19 09:08:46 EDT 2011

Hi John,

I have found a work-around for it from Martijn and Johan who read these lists as well. They told me that when we change the max-vlans and the session-limit setting to a lower value it will create more "room" for the BGP sessions. I have tested this and it works properly. Keep in mind that in the newest release the max-vlan setting does not define the highest vlan number that is possible, it merely defines the maximum amount of vlans.

One thing I do wonder is what is the use for the session-limit, what does it actually "limit" and how can I read out how many session's the router is currently really needing/using.

And ofcourse it would be nice to be able to read out what now my maximum amount of BGP sessions is :)

Thanks for thinking with me. And I thought I should post this for others who search for this issue.

Pieter Taks

Van: John J <jazzyjpz at gmail.com<mailto:jazzyjpz at gmail.com>>
Datum: Thu, 17 Mar 2011 17:06:29 -0500
Aan: Pieter Taks <p.taks at nforce.nl<mailto:p.taks at nforce.nl>>
CC: "foundry-nsp at puck.nether.net<mailto:foundry-nsp at puck.nether.net>" <foundry-nsp at puck.nether.net<mailto:foundry-nsp at puck.nether.net>>
Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hello,

Hmmm, nice find w/the 'Maximum Peer Index Number' under the variables output. I checked some of our Brocade routers and it appears this number is dynamic and not static, but it's set ABOVE what our peer count is currently at(1 router w/less peers was showing 40 above...a router with a lot more peers had this at over 200 above current neighbor count). Being that you have maxed this out, I'm guessing either the platform variably alters that number UP TO a max of 253 or it has hit some sort of memory maximum from somewhere else which is not allowing it to move beyond 253. I can't tell you what is happening, but it's quite strange.

Let us know if you figure anything more out with this.

Sincerely.

On Wed, Mar 9, 2011 at 1:15 PM, Pieter Taks <p.taks at nforce.nl<mailto:p.taks at nforce.nl>> wrote:
Hi,

Thanks for reading with me and thinking what it could be. Even with all the soft-reconfigurations disabled I still got the same issue. Please see the debug outputs below, perhaps anyone else knows why this is limited to 252 BGP peers on an RX4?

In the sh ip bgp debug variables output it shows Maximum Peer Index Number:253. Is there a way to change this?
sh ip bgp debug variables
safi:0, &bgp:08fe0aac, enabled:1, operational:1, &dbg_mem=08fe1c4c, curr_afi:0
io_process_running:0, io_process_next_peer_number=0
in_long_loops 0, clear_all 0, timer 00000000, count 0
timer_enabled:1, timer_next_peer_number:1, 1s timer 1, short timer 1
scheduler id:5:2, ip:77.247.181.xxx/28, time=92219
bgp_tcb:08fe0b74 (0xf0090001, 0), tick_cnt=12, seconds=6717786
bgp_tcb6:f00a0001 (0x00000000, 0x00000019)
*peer:08fe0bb0, *peer_group:08fe1b5c, RIB_in_root_node:24a35960
Maximum Peer Index Number:253, check_nexthops:0 0
router_id:85.159.239.xx, configured:0, cluster_id:0.0.0.13, configured:1
route_is_router_reflector:0, client_to_client_reflection:1
networks:x08fe401c, aggregate:x08feb0b8
default_metric:4294967294, local_preference:100, keep_alive:60, hold_time:180
originate_default:0, originated:0
distance:20 200 200, fast_external_fallover=0
nexthop recur0, en_def:0, readvertise:1, auto_sum:0, synch:0
always_compare_med:1, compare_med_with_empty_aspath: 0, redistribute_ibgp:150911764, local_network_check_time_count:6
nexthop_cache_hit_count:159342982, nexthop_cache_miss_count:5142
system memory:536870912, total_allocated:104422684, bgp_defined_quota:2147483648

sh ip bgp debug
BGP Debug Information
Pid Size  Address   Total   Used    Free    NoMem  Errors  #_pools p_unit
0   8     08fe3d38  43688   31090   12598   0      0       5       2000
1   16    08fe3d64  52426   37260   15166   0      0       5       2000
2   24    08fe3d90  224692  131923  92769   0      0       10      2000
3   32    08fe3dbc  116502  77986   38516   0      0       12      800
4   48    08fe3de8  20163   12144   8019    0      0       7       400
5   64    08fe3e14  3851    981     2870    0      0       5       200
6   96    08fe3e40  2617    397     2220    0      0       6       80
7   128   08fe3e6c  2480    1081    1399    0      0       8       40
8   256   08fe3e98  755     478     277     0      0       6       20
9   512   08fe3ec4  0       0       0       0      0       0       10
10  4096  08fe3ef0  0       0       0       0      0       0       10
11  48    08fe3f1c  362952  349541  13411   0      0       20      4000
12  36    08fe3f48  183498  160937  22561   0      0       8       8000
13  56    08fe3f74  699040  674509  24531   0      0       42      4000
14  84    08fe3fa0  250214  169516  80698   0      0       22      4000
15  96    08fe3fcc  0       0       0       0      0       0       4000
  Total Memory Use for Route and Attributes Tables   : 104328584
  Memory Block Not Available Count   : 0
  Bad Memory Pool ID Count           : 0
  TCP buffers                        : 4096 0 0 0
  BGP Tx Parameters                  : 5 30 3
  BGP route update count             : 5 (4) last:17h54m45s
                   event             : (5:2) 77.247.181.xxx/28
  BGP io semaphore take 54915185, yield 20300844, 14291686
  Max timer process: l-5131 s-80740 (463), io: 88163 7431 us
  io_rx_yield_time 0x08fe0acc, 1
  1 sec timer value: 53364030, 21197655 TB
  MP active: 1, standby up 0
  Graceful_restart: enable 1, restart time 15, stale-routee 360, purge 600
    Restarted 0, fwd 0, restart_up_time_count[0] 0

sh ip bgp debug memory
BGP_CLASS: 198964, BGP_PEER_CLASS: 120071, BGP_CONFIGURATION_CLASS: 3534
BGP_AS_PATH_ENTRY: 84, BGP_IPV6_AS_PATH_ENTRY: 96, BGP_AS_PATH_SEGMENTS: 14
BGP_NLRI_ENTRY: 56, BGP_PATRICIA_KEY_ADDRESS: 16, BGP_PATRICIA_NODE: 48
BGP_RIB_OUT_NLRI_ENTRY: 36, BGP_RIB_OUT_HOLDER: 28, BGP_WITHDRAWN_ROUTE_ENTRY: 34
BGP_NEXTHOP_ADDRESS: 16, BGP_NEXTHOP_ENTRY: 180, BGP_DAMPING_NLRI_ENTRY: 36
BGP_ROUTE_DAMPING_REUSE_LIST: 2052, BGP_ROUTE_DAMPING_BLOCK: 3522, BGP_DAMPENING: 37374
CU_BGP_ROUTE_MAP_SET_T: 234, CU_BGP_ROUTE_MAP_MATCH_T: 1222
BGP_MATCH_TAG: 65, BGP_SET_AS_PATH: 40, BGP_SET_COMMUNITY: 138
BGP_MATCH_UNION: 207, BGP_SET_UNION: 138
ROUTE_FILTER_MEMORY_ENTRY: 220, BGP_MATCH_CRITERIA: 216, BGP_SET_COMMAND: 147

sh ip bgp debug route-table
There are 674476 NLRIs in BGP Route Table, time 323 ms

Best regards,

Pieter Taks

Datum: Fri, 4 Mar 2011 17:03:03 -0600

Onderwerp: Re: [f-nsp] Max BGP peers BigIron RX4

Hi,

That's interesting.

Does the RX series support the 'show ip bgp debug' cmd list? It's hidden up to the point of 'debug'. If you can get that far, take a look around in there, especially 'show ip bgp debug memory' & just the plain 'show ip bgp debug'. Maybe something will stick out.

I want to say the router dynamically allocates memory for certain fields up to a certain dynamic max related to other parts of BGP like # of routes. Do you have soft-reconfig turned on? Maybe try turning that off and see if it frees up any more BGP peers. Brocade software can be a little dodgy at times and not well documented.

Also, I wouldn't be surprised if it was just some sort of bug. We've had stranger things happen when it comes to a bug on the Brocades.

Sincerely.

On Tue, Feb 22, 2011 at 8:23 AM, Pieter Taks <p.taks at nforce.nl<mailto:p.taks at nforce.nl>> wrote:
Hi all,

While configuring a new peer on an BigIron RX4 we got the following error:
Error! BGP4 cannot allocate memory for peer 253

A PDF claims the following (http://www.terabitsystems.com/foundry-docs/Foundry-BigIron-RX-Series-Datasheet.pdf):
BGPv4: Scalable to 4 million routes, 500 peers and 14,000 attributes with MR2 management module

Is there any reason why it wouldn’t want to configure more peering sessions?

-          I do not see any system-max setting for this

-          As well as I do not see any full memory yet

Hope someone knows how to resolve this or what is causing this error.

sh memory
====================================================================
BigIron RX active MP slot 33:
Total SDRAM:           2147483648 bytes
Available Memory:      1646874624 bytes
Free Physical Pages:       401374 pages

Malloc statistics: total 500462813
os_malloc count: 24059060, fail: 3;  os_free count: 24046183, fail 0, diff: 12877
====================================================================
BigIron RX LP SL 1:
Total SDRAM:            536870912 bytes
Available Memory:        53039104 bytes
====================================================================
BigIron RX LP SL 3:
Total SDRAM:            536870912 bytes
Available Memory:        72671232 bytes
====================================================================
BigIron RX LP SL 4:
Total SDRAM:            536870912 bytes
Available Memory:        72654848 bytes

Any help is appreciated, thanks!

Pieter Taks

_______________________________________________
foundry-nsp mailing list
foundry-nsp at puck.nether.net<mailto:foundry-nsp at puck.nether.net>
http://puck.nether.net/mailman/listinfo/foundry-nsp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/foundry-nsp/attachments/20110319/66a1ff55/attachment.html>