[j-nsp] Managing MX480 fxp0

[Alain Hebert]

     Hi,

     How wrong we where doing that with our MX960, QFX5100, and a few 
MX104 =D.

     One of our OOB is a bunch of EX2300 switches using STP, on a 
different set of dark fiber linking a few Metro data centers together... 
but as usual with JNP...  one went nuts and started spewing packets from 
the other link while shifting left a few bytes.  When those packets hit 
our fpx0s, dos protect did <beep> all and killed their CPU dropping 
everything BGP and MPLS (thx JNP) on most routers connected to the OOB 
network.

     Now, at each site, we have a mini putter (Lenovo/Zotac/etc) with 
SSD, Sealink serial ports, Consumer xDSL/Coax, MFA encrypted VPN. We 
enable fxp0 *if* needed...


Other things to think about:

     1. We're even looking at swapping to Cisco L2 switches instead of 
JNPs, since this type of event never happened, in our collective 
experience, with that brand.

     2. Using OSPF3 (or IS-IS to limit OSPF injection) would have limit 
the fpx0 DoS to the local OOB switch...  Which is still too risky for 
our taste.

     3. You could use Serial->Ethernet devices instead of the Sealink 
but if the OOB switch goes down again, you cannot access the serials.


     PS: In our case it is our fiber bundles and we didn't need to 
invest in DWDM ... but its the same idea.  For years an associate of 
mine implemented a very large deployment of OOB over DWDM and Cisco L2 
switches with 0 downtime.

     Have fun and good luck.

-----
Alain Hebert                                ahebert at pubnix.net
PubNIX Inc.
50 boul. St-Charles
P.O. Box 26770     Beaconsfield, Quebec     H9W 6G7
Tel: 514-990-5911  http://www.pubnix.net    Fax: 514-990-9443

On 2019-11-26 06:09, Sander Steffann wrote:
> Hi,
>
>> I would personally not wire or use fxp0 unless I'm out of options.
>> Some other vendors today have real out-of-band ethernet for MGMT,
>> meaning own CPU, own memory, own OS not fate-sharing the
>> control-plane, which is the correct solution for OOB, but not
>> something we as a community are actively asking vendors to deliver.
> We built an OOB network exactly like that. Cheap L3 switches talking OSPF to each other over their own 1G DWDM channels, completely independent of the production network. A separate OOB network used to be crazy expensive, but with cheap DWDM gear suddenly all you need is a free DWDM channel and some cheap second hand L3 switches. And that's what we connect our fxp0 ports to.
>
> Cheers,
> Sander
>
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp