[j-nsp] Outgrowing a QFX5100

[Jason Healy]

On Sep 20, 2022, at 12:57 AM, Mike Gonnason <gonnason at gmail.com> wrote:
> Do you have any more details about what limitations you are encountering on the QFX?  Is it hardware related or software? 

The example that spurred my email was DDOS protection on the QFX.  We're getting lots of L3NHOP errors (still, I wrote to the list a while back about it) and have been trying to track them down.  On some platforms you can capture the flows causing the DDOS violation, but not the QFX.  We've been forced to perform random packet captures in the hope of finding the traffic on the right interface.

Another bug causes DHCP relay to fail when an ACL is applied on an interface, even if the filter explicitly permits DHCP traffic.

The chipset has a "feature" where IPv6 counters aren't incremented at all (they claim this is "working as designed").

Filter-based forwarding is not supported on IPv6 (the documentation on this has been corrected, but only after we escalated our case through ATAC).

There's a bug where setting a 0.0.0.0/0 match in an inet firewall filter prevents ipv6 traffic from passing (incorrect hardware programming).  We have to use ether-type instead in order to hack around it.

There are limitations on egress filters that don't appear to apply on other platforms.

Many of these issues were not stated in the official documentation, and some still aren't (you have to search KB articles to find the limitations).  That makes product evaluation very difficult and is part of why I was asking the list.

Most of our problems seem to center around L3 stuff (ACLs, forwarding, etc), which I why I asked about the router line.  It seems like I'm asking "too much" of the QFX as a core router, though it does pretty well as a switch.  The full router line is overkill for me (I don't need a full table, for example), but if it means some of these other features will actually work as designed, it might be worth it.

The mx304 is an interesting option, as is the ACX line.  Maybe one of the newer QFX models will fix some issues that the broadcom chipset had, but I'll need to test the heck out of that first.

Thanks,

Jason