[j-nsp] MX304 Port Layout

[Mark Tinka]

On 6/28/23 08:44, Saku Ytti wrote:

> Apart from obvious stuff like QoS getting difficult, not full feature
> parity with VLAN and main interface, or counters becoming less useful
> as many are port level so identifying true source port may not be
> easy. There are things that you'll just discover over time that don't
> even come to your mind, and I don't know what those will be in your
> deployment. I can give anecdotes
>
> 2*VXR termination of metro L2 ring
>      - everything is 'ok'
>      - ethernet pseudowire service is introduced to customers
>      - occasionally there are loops now
>      - well VXR goes to promisc mode when you add ethernet pseudowire,
> because while it has VLAN local significancy, it doesn't have per-vlan
> MAC filter.
>      - now unrelated L3 VLAN, which is redundantly terminated to both
> VXR has customer CE down in the L2 metro
>      - because ARP timeout is 4h, and MAC timeout is 300s, the the
> metro will forget the MAC fast, L3 slowly
>      - so primary PE gets packet off of internet, sends to metro, metro
> floods to all ports, including secondary PE
>      - secondary PE sends packet to primary PE, over WAN
>      - now you learned 'oh yeah, i should have ensured there is
> per-vlan mac filter' and 'oh yeah, my MAC/ARP timeouts are
> misconfigured'
>      - but these are probably not the examples you'll learn, they'll be
> something different
>      - when you do satellite, you can solve lot of the problem scope by
> software as you control L2 and L3, and can do proprietary code
>
> L2 transparency
>      - You do QinQ in L2 aggregation, to pass customer frame to
> aggregation termination
>      - You do MAC rewrite in/out of the L2 aggregation (customer MAC
> addresses get rewritten coming in from customer, and mangled back to
> legitimate MAC going out to termination). You need this to pass STP
> and such in pseudowires from customer to termination
>      - In termination hardware physically doesn't consider VLAN+ISIS
> legitimate packet and will kill it, so you have no way of supporting
> ISIS inside pseudowire when you have L2 aggregation to customer.
> Technically it's not valid, technically ISIS isn't EthernetII, and
> 802.3 doesn't have VLANs. But technically correct rarely reduces the
> red hue in customers faces when they inform about issues they are
> experiencing.
>      - even if this works, there are plenty of other ways pseudowire
> transparency suffers with L2 aggregation, as you are experiencing set
> of limitations from two box, instead of one box when it comes to
> transparency, and these sets wont be identical
>      - you will introduce MAC limit to your point-to-point martini
> product, which didn't previously exist. Because your L2 ring is
> redundant and you need mac learning. If it's just single switch, you
> can turn off MAC learning per VLAN, and be closer to satellite
> solution
>
> Convergence
>      - your termination no longer observes hardware liveness detection,
> so you need some solution to transfer L2 port state to VLAN. Which
> will occasionally break, as it's new complexity.

So all the above sounds to me like scenarios where Metro-E rings are 
built on 802.1Q/Q-in-Q/REP/STP/e.t.c., rather than IP/MPLS.

We run fairly large Metro-E rings, but we run them as IP/MPLS rings, and 
all the issues you describe above are the reasons we pushed the vendors 
(Cisco in particular) to provide boxes that were optimized for the 
Metro-E applications, but had proper IP/MPLS support. In other words, 
these are largely solved problems.

I think many - if not all - of the issues you raise above can be fixed 
by, say, a Cisco ASR920 deployed at scale in the Metro, running IP/MPLS 
for the backbone, end-to-end.

Mark.