[j-nsp] DIRTY-TRAFFIC VRF Redirection via Flowspec

[Catalin Dominte]

Hello everyone.

I am having to deal with a really obscure issue. Full juniper MX core (10k8, 10k3 and MX480) all running the same firmware version (JUNOS 23.4R2.13).

All doing traffic mirroring to a Corero detector (via GRE Tunnels), which gets analysed and then Corero pushes a flowsepc route to push the traffic into the DIRTY-TRAFFIC routing instance.

However, only partial dirty traffic gets pushed into the DIRTY-TRAFFIC VRF, which is baffling, because from what I observed over the years, a route either works or not at all. I guess I should mention is that the redirect is using via next-hop via LT-0/0/0 interfaces, with one subif in Global, and one subif in Dirty.

Transit and peering configured in a mix of IRB and direct interfaces (with inet and inet6 details and sampling and traffic mirroring and counters for Corero).

Example:
DDOS incoming today. Total 5gbps on local transit on the router that has the scrubber directly connected. 400mbps makes it into the VRF, the rest gets forwarded to the customer. Single destination targeted, on UDP port 80.

In the end, the ephemeral database saved us, as that got updated to filter the traffic on the edge, but we have seen this happen over and over again where 70% of the traffic hits the customer, and only a quarter, if lucky, makes it into the VRF to head for the scrubbers.

Can see the correct telemetry sent to the detector, the detector sees the traffic, but somehow the customer ports still gets hit by the attacks. I can see the correct route in the inetflow.0 table, pointing to the correct next hop, and the correct interface gets resolved. So in theory it works, but the bandwidth graphs tell a different story.

Has anyone experienced anything similar in the wild, and if so, do you have any advice? Various attacks like that that get missed mostly.

I tried to do only flowspec without the VRF, but that causes blackholing, if the traffic destination is not on the local router with the scrubber, as all MX routers receive the same flowspec route and apply it inbound on any interface.

Any pointers would be appreciated.
Catalin