[j-nsp] MX304 Port Layout

[Saku Ytti]

On Tue, 27 Jun 2023 at 19:32, Mark Tinka <mark at tinka.africa> wrote:

> > Yes.
>
> How?

Apart from obvious stuff like QoS getting difficult, not full feature
parity with VLAN and main interface, or counters becoming less useful
as many are port level so identifying true source port may not be
easy. There are things that you'll just discover over time that don't
even come to your mind, and I don't know what those will be in your
deployment. I can give anecdotes

2*VXR termination of metro L2 ring
    - everything is 'ok'
    - ethernet pseudowire service is introduced to customers
    - occasionally there are loops now
    - well VXR goes to promisc mode when you add ethernet pseudowire,
because while it has VLAN local significancy, it doesn't have per-vlan
MAC filter.
    - now unrelated L3 VLAN, which is redundantly terminated to both
VXR has customer CE down in the L2 metro
    - because ARP timeout is 4h, and MAC timeout is 300s, the the
metro will forget the MAC fast, L3 slowly
    - so primary PE gets packet off of internet, sends to metro, metro
floods to all ports, including secondary PE
    - secondary PE sends packet to primary PE, over WAN
    - now you learned 'oh yeah, i should have ensured there is
per-vlan mac filter' and 'oh yeah, my MAC/ARP timeouts are
misconfigured'
    - but these are probably not the examples you'll learn, they'll be
something different
    - when you do satellite, you can solve lot of the problem scope by
software as you control L2 and L3, and can do proprietary code

L2 transparency
    - You do QinQ in L2 aggregation, to pass customer frame to
aggregation termination
    - You do MAC rewrite in/out of the L2 aggregation (customer MAC
addresses get rewritten coming in from customer, and mangled back to
legitimate MAC going out to termination). You need this to pass STP
and such in pseudowires from customer to termination
    - In termination hardware physically doesn't consider VLAN+ISIS
legitimate packet and will kill it, so you have no way of supporting
ISIS inside pseudowire when you have L2 aggregation to customer.
Technically it's not valid, technically ISIS isn't EthernetII, and
802.3 doesn't have VLANs. But technically correct rarely reduces the
red hue in customers faces when they inform about issues they are
experiencing.
    - even if this works, there are plenty of other ways pseudowire
transparency suffers with L2 aggregation, as you are experiencing set
of limitations from two box, instead of one box when it comes to
transparency, and these sets wont be identical
    - you will introduce MAC limit to your point-to-point martini
product, which didn't previously exist. Because your L2 ring is
redundant and you need mac learning. If it's just single switch, you
can turn off MAC learning per VLAN, and be closer to satellite
solution

Convergence
    - your termination no longer observes hardware liveness detection,
so you need some solution to transfer L2 port state to VLAN. Which
will occasionally break, as it's new complexity.

> > Like cat6500/7600 linecards without DFC, so SP gear with centralised
> > logic, and dumb 'low performance' linecards. Given low performance
> > these days is multi Tbps chips.
>
> While I'm not sure operators want that, they will take a look if the
> lower price does not impact performance.
>
> There is more to just raw speed.

I mean of course it affects performance, as you are now having all
ports in single chip, instead of having many chips.  But when it comes
to PPS people are confused about performance, no one* (well maybe 1 in
100k running some esoteric application) cares about wire-rate.
If you are running a card like 4x100 ASR9k, you absolutely want
wire-speed, because there is 1 chip per port, and you want the pool
port is drawing from to have 1 port wire rate free, to ingest dos in
mostly idle interface. But if you have 128x100GE in a chip, you're
happy with 1/3 PPS easily, probably much much less. Because you're not
gonna exhaust that massive pool in any practical scenario, and several
interfaces simultaneously can ingest dos.




-- 
  ++ytti