[rbak-nsp] CGNAT performance issues

[Marcin Kuczera]

On 2020-01-27 12:54, Mariusz K. Grzeca wrote:
> Hi,
>
> We are currently experiencing some major peformance issues with one of our SEs.
>
> Our platform is SE1200 with 2xXCRP4 and 9 10ge-4-port cards, SEOS-12.1.1.12p15-Release. 7 BGP4 peers with a total of ~40Gbps throughput during the evening traffic peaks. Around 30k active CLIPS subscribers of which ~25k have a NAT policy attached (enhanced NAT with logging and paired mode).
>
> Initially we had 2 line cards intended for BGP sessions only, 5 cards for CLIPS sessions and 2 cards reserved for other low throughput purposes. The first symptoms we experienced were reduced bandwidths for subscribers with 1Gbps service plans (500-600 Mbps instead of the usual 941 Mbps). The problem only affected NAT clients. A couple of weeks later BGP sessions started flapping. At first it seemed like we were hitting the 20Gbps per card limit and rising tail drop counters on the BGP cards seemed to confirm it.
>
> So we fiddled with the wires and came out with a different setup - 7 cards with at most 2 ports connected, one for BGP peer and the other for CLIPS sessions. And it actually made the situation worse - rising tail drop counters on each card and even lower bandwidths during the evening traffic peaks and BGP sessions kept flapping. In order to prevent flapping BGP sessions we moved most of the BGP peers to another SmartEdge router and added 2 more 10ge-4-port cards for CLIPS sessions. Result - no more flapping but nothing has changed in terms of either bandwidth or tail drop counters.
>
> I would be grateful for any suggestion as to the possible causes of this situation.

First of all, Smart Edge is not the best solution for BGP today, as you
will not get any more software updates most probably, and this is what
is required to keep BGP router healthy.
Best case is to have 2 BGP routers, from different vendors. Like MX204
and ASR9001 (as we have, however ASR has less throughput than MX).

This will offload PPAs on linecards, as whatever happens, they will keep
a lot of routing entries to keep traffic flow properly.

Second, NAT. Smart Edge was bestseller when most of telcos did not use
NAT. So it might have some influence on performance, that was never
tested before, while it was supported by ericsson...
So moving out NAT to some software machine (linux based x86) might be
good test.

We use SmartEdge with similar traffic (but less cards), however it does
only BRAS funtionality without NAT. Smooth as never.

Marcin


>
>
> Thanks.
>
> _______________________________________________
> redback-nsp mailing list
> redback-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/redback-nsp


-- 

Marcin Kuczera / Wiceprezes Zarządu / CTO
+48 32 440 80 71/ marcin.kuczera at leon.pl <mailto:marcin.kuczera at leon.pl>

Leon Sp. z o.o.
ul. Kilińskiego 33d, 44-200 Rybnik
http://www.leon.pl/

INTERNET | TELEWIZJA | TELEFON

KRS 0000223101 Sąd Rejonowy w Gliwicach
Kapitał zakładowy 576.700 zł
NIP: 6332068698

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/redback-nsp/attachments/20200128/c0c948a4/attachment.htm>