[j-nsp] DSCP-marked traffic mysteriously being dropped by MX960

[John Neiberger]

We have packet captures going at the endpoints, but not in between,
unfortunately. It would be nice if we had a sniffer at that location
so we could mirror the ports and get some data there.

The inbound filter on Router C looks like this:

term netmgmt {
    then {
        count fec-cs2;
        loss-priority high;
        forwarding-class MNGMT;

Elsewhere, MNGMT is defined as cs2.

The outbound filter on Router A toward the end device is very long,
but the first two terms should allow ICMP and traceroute from the
address space that Device B is in. Further down in the filter is
another term that specifically permits CS2, among other things. We
know that that filter is working because when we remove the ingress
marking filter everything starts working, which indicates that the
outbound filter on Router A is correctly matching the source IP
address of Device B. I'd like to post more of the config, but I'm
trying to keep it sanitized and anonymous.  :)  I don't want to
irritate any bosses by posting our configs publicly.

Thanks,
John

On Fri, Jul 20, 2012 at 3:04 PM, OBrien, Will <ObrienH at missouri.edu> wrote:
> Have you captured traffic before and after to validate the marking?
> Relavent config bits would help.
>
> On Jul 20, 2012, at 3:56 PM, John Neiberger wrote:
>
>> We've been troubleshooting a strange problem for a few days. JTAC is
>> on the case, too, but we have not found any resolution. I thought
>> maybe picking some minds here would be helpful. Here is a simplified
>> diagram:
>>
>> [Device A] -------   [Router A] -------  [Router B] ------- [Router C]
>> ----- [Device B]
>>
>> The problem is that packets from Device B to Device A are being
>> dropped at Router A. Routers A and C are MX960s. Router B is a CRS.
>> Router C has an ingress firewall filter that does nothing but mark
>> traffic as cs2. Router A has an egress firewall filter toward Device
>> A, but it specifically allows the source IP address of Device B as
>> well as any traffic marked as cs2.
>>
>> Here is where it really gets weird. If we remove the filter on Router
>> C that marks the traffic, everything starts working. Put the filter
>> back in place and the traffic stops. We've been looking at this for a
>> couple of days and JTAC has spent a few hours looking at it and we're
>> still no closer to figuring out why cs2 traffic is being dropped. With
>> the filter in place, traceroutes from Device B to A stop at Router A.
>> Remove the marking filter and traceroutes complete and pings start
>> succeeding.
>>
>> Can any of you think of a potential culprit that we're not seeing? I
>> would hope that if this were something obvious, JTAC would have caught
>> it by now. We're all stumped.
>>
>> Thanks!
>> John
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>