[j-nsp] MX304 Port Layout

[Mark Tinka]

On 6/27/23 19:44, Gert Doering wrote:

> The issues we see / have with "satellites that are not real satellites"
> are
>
>   - l2 link down -> l3 not going down
>   - (H-)QoS on the L3 device to avoid sending 10/100G worth of traffic
>     down to a 100M customer port, saturating the "vlan on trunk" link
>     for no good
>
> (we run config-management-system managed EX3400s off an Arista MLAG pair,
> so the benefits outweigh the drawbacks for us)

Same here.

I think this is the architecture most operators run, especially because 
despite centralized routers being less disjointed, they are still 
pricier than attaching a switch to a router port via an 802.1Q trunk. If 
you are aggregating plenty of 1Gbps or 10Gbps ports that aren't each 
carrying lots of traffic, centralized ports will be more expensive than 
a traditional switch<=>router architecture compared to even the cheapest 
and most dense centralized router.

I think centralized routers make a lot of sense when aggregating 100Gbps 
services, because doing these on a switch<=>router architecture is going 
to be tough when you try to aggregate N x 100Gbps links, or even a 
handful of 400Gbps links for the 802.1Q trunk, as most of those 
customers will be riding their 100Gbps port really hard, i.e., there is 
enough distance between 10Gbps and 100Gbps, while there really isn't any 
between 100Gbps and 400Gbps.

One of the biggest risks I find with a standard switch<=>router 
architecture is the outage that could happen if that trunk goes down. 
Yes, you could mitigate that by making the trunk a LAG of multiple 
links, but the challenge that creates is how to do policing on the 
router sub-interface because the policer is split across the member 
links in the LAG. On the other hand, one could get around this 
commercially by letting customers know the SLA associated with being 
connected to only "one device", and provide the option of connecting to 
"a second device".

The other issue I find with the switch<=>router architecture is where to 
do policing. With LAG's, shaping on the switch ports makes sense because 
you don't run into having to police across member links in a LAG on the 
router port. Most aggregation switches do not support egress policing 
with Broadcom chips, so shaping is your only tool. It has worked well 
enough for us on the Arista switches that we have deployed for this, but 
was not a reasonable option when we used to run the EX4550 that only had 
4MB of packet buffers.

Mark.