[j-nsp] MX punting packets to RE - why?

[Saku Ytti]

On 3 February 2016 at 02:18, Ross Halliday
<ross.halliday at wtccommunications.ca> wrote:

Hey,

> Yes, on the entire MPC I will see unrelated control plane protocols bounce, eg. spanning-tree. If I recall correctly some protocols are handled by the TRIO chips, right? I don't see any of my BFD-managed ISIS adjacencies drop.
>
>> If yes, just limit the mcast resolve to something reasonable 100pps should be plenty, provided we're not competing with  actual attack traffic.
>>
>> I would start with ddos-protection fixes and see if it behaves better with more restricted punting.
>
> I assume you're referring to "set forwarding-options multicast resolve-rate", right?

No, something like this:
edit system ddos-protection protocols resolve mcast-v4
set bandwidth 100
set burst 100
set flow-level-bandwidth logical-interface 20
set flow-level-detection subscriber off
set flow-level-detection logical-interface on

So we allow on aggregate 100pps of mcast-v4 resolve, but only 20pps
per IFL. So even if one IFL is misbehaving, another IFL's mcast-v4
resolve works fine.

There are only 4k policers available in HW for ddos-protection, which
makes me turn off subscriber detection, as it's easy for attacker to
generate 'new subscriber' (like in BGP change SADDR => new subscriber)
and congest those 4k policer slots.
I make logical-interface on (instead of 'automatic', which means they
are added dynamically if aggregate policer if out-of-contract, but
with 'on' they are always there, this guarantees well-behaving IFL
does not suffer or the duration software detects this and adds the IFL
policers)

This is just random recommendation. And funny thing is, you'd need to
make similar thing to _all_ protools available for 'ddos-protection',
there are quite many of them, like maybe 100, so you'll get several
thousand new lines of config just for this.
I wish there was way to set default explicit values, but there isn't.

> I'm in the habit of running VSTP for everything (the Cisco half of my brain keeps trying to type rapid-pvst+) that isn't a two-port affair. BPDUs are definitely making it through, everything checks out. The paths over the l2circuits are normally blocked via increased interface cost.

Hmm, then the redundancy really should have worked, unless you're
doing some IGMP Snooping or something in the switches (on by default)
which might require convergence on multicast states on the L2 too.
If you're rocking RSTP  or MST, and it's correctly configured (all
non-l2-core ports _MUST_ be portfast, because MST will not unblock
downstream ports, until there is explicit permission from all upstream
ports, and if you don't have portfast on in 1 port which is not
speaking MST, then this port will block whole MST convergence, as
you're waiting for explicit permission from that port, which will
never come).

> One of the VLANs carried as an l2circuit by the MXes between the EXes is actually not spanning-tree controlled, but a "backup" PIM interface. Essentially a clone of the EX-EX direct link, but with higher metric. Unlike the other VLANs this one always has the PIM and BGP adjacency sending traffic over it. The ddos-protection resolve-mcast4 action trips when multicast is slammed over that or one of the VSTP-managed VLANs transitions to a forwarding state.
>
>
> I can do up a diagram if that would help. I'm really not sure how I'd explain this to JTAC and wanted to get some real-world experience from guys who are working with this stuff.

Yeah multicast is tricky subject, I've had to learn it maybe 3 times
now, since it's needed so rarely it's easy to forget. I dislike
multicast, I believe there are probably lot of yet unknown DoS vectors
in it, and I would never run internet multicast. But for well
controlled internal applications, it may sometimes be least bad
solution.
But getting help on the setup probably is huge chore, just getting
external person to understand the network takes time.




-- 
  ++ytti