[c-nsp] ASR9k Bundle QoS in 6.0.1

[Mark Tinka]

On 12/May/16 19:58, Saku Ytti wrote:

> I've not used it, curious to hear why it does not work? Of course if
> you don't have any entropy, then there is nothing you can do, if there
> is just single fat flow which needs more than single member has
> capacity, no flows, no flow balancing. What should work, is if in
> addition to fat flows you have others.

In our case, the traffic was EoMPLS traffic. In the direction exiting
the router toward a switch over a LAG. At that point, MPLS headers
should be getting stripped and the native Ethernet traffic forwarded on
downstream.

I'd have thought that the Trio will look deeper into the Ethernet frames
and extract IP- and TCP/UDP-level entropy to use as a hashing mechanism
to spray bits across the member links of the LAG, but no joy. We tried
several combinations of the hash keys (including the
"gtp-tunnel-endpoint-identifier", just for giggles), all under
"adaptive" mode, and nothing worked.

Per-packet mode was the only solution, as single flows on LAG's lead to
two problems:

  * The main reason for the LAG (increase port bandwidth cheaply) is
    negated.

  * Policers programmed in equal distribution across participating ports
    or PFE's create a bottleneck if a single flow running across a
    specific interface triggers the calculated carve-up factor for a
    specific member link in the LAG.

I think that there is a bad disconnect by the vendors on the
relationship between LAG's and the policers applied to them. On
core-facing LAG's, LAG's are straightforward from a policing standpoint,
i.e., policing is generally not done on core-facing ports. But on
customer-facing LAG's, the interaction between LAG's and policing has
been - I feel - ignored by the vendors. This is a use-case that they do
not appreciate. Treating both in isolation does not make sense anymore,
because LAG's are a real use-case, and so is policing.

In our case, per-packet load balancing is the only solution for
single-flow use-cases, primarily because of how policers are programmed
for LAG member links; but also because single-flow situations need a
solution.

Cisco's lack of support for per-packet load sharing in IOS XR in general
(never mind them lacking support for it at an interface-level on several
other platforms) is what keeps us from using their equipment in a
high-capacity edge scenario. In this case, the MX is a far superior
platform. Per-packet load balancing is certainly cheaper than 100Gbps
native ports when you're nowhere ready to upgrade to that kind of
capacity. And it will certainly be cheaper than 400Gbps native ports
when a LAG is composed of several 100Gbps member links, and so on and so
on...

Mark.