[c-nsp] 7600s, DSCP, ASICs, Bollocks

[James Bensley]

Hi All,

I have a TAC case open for this but it's not going anywhere. We have
two remote 7606 chassis with a 10G link between them, we have two
separate 10G transit feeds, one landing on each chassis and then
downstream customers hanging off the chassis.

R1 --10G-- R2

The problem is that for love nor money, I can't stop DSCP markings
coming in from the Internet on these remote PEs. Output from "show
modules", LAN line cards here and no DFCs so fairly pony:

Mod Ports Card Type                              Model
--- ----- -------------------------------------- -----------------
  1   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX
  2   48  CEF720 48 port 1000mb SFP              WS-X6748-SFP
  3   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX
  4    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE
  5    5  Route Switch Processor 720 10GE (Activ RSP720-3CXL-10GE

On R1 Transit is via port Te4/3, Te4/1 is the link to R2 where my main
testing customer is connected. Since these 7600s had some existing QoS
configured via MQC so I simply added a policy-map (so there was no mls
trust statement and mls QoS is enabled globally);

 policy-map Transit-Ingress
  class class-default
   set dscp 0
   exit
  exit
 int Te4/3
  service-policy input Transit-Ingress
  exit

The output of “show policy-map int te4/3” showed the class-default
counters going up so it looks like that should have fixed the issue. A
downstream customer is sending me packet captures showing traffic
coming into their edge with DSCP markings on it. ELAM shows the same,
traffic coming into R1 from the port the transit provider is connected
to (Te4/3) with DSCP marking on it.

TAC said maybe the policy-map wasn't programmed into the ASICs
properly because there is no class with a match statement only
class-default (if that is true, it’s a massive flaw in my opinion, so
I hope that is wrong, or maybe what he actually meant was policy-maps
aren’t well supported on LAN cards without DFCs?) and recommend I
change it to the following:

 ip access-list extended ACL-Transit-Ingress-DSCP
  permit ip any any
  exit

 class-map match-any CM-Transit-Ingress-DSCP
  match access-group name ACL-Transit-Ingress-DSCP
  exit

 policy-map PM-Transit-Ingress-DSCP
  class CM-Transit-Ingress-DSCP
   set dscp 0
   exit
  exit

Now the traffic counter stats are going up for this class under “show
policy-map int te4/3” but it still hasn’t fixed the issue (confirmed
by customer packet captures and ELAM).

I have removed the policy-map and since the port has no “mls qos trust
xxx” statement it should by default remove all incoming DSCP markings
(re-write to 0) however the customer is STILL seeing marked traffic
from the Internet and I can still see it via ELAM and local SPAN to a
Linux box in the PoP.

I’m pretty much out of ideas as I haven’t got the exact same tin in
the lab to simulate with, the only thing I can think is that it’s (1)
an IOS bug (currently 15.2(4)S4 with a 15.3(3)S6 upgrade planned soon)
 or (2) it’s somehow related to the fact that these are LAN cards
without any DFCs and because the WS-X6704-10GE has “mls qos trust xxx”
configured on Te4/1, Te4/2 and Te4/4, so just not Te4/3 facing the
transit provider.

Does this card actually have 4 ASICs (one per port) or 2 ASICs so one
per pair of 10G ports? Cisco.com is not clear though [1], [2]
different pages read differently:

R1#show interfaces te4/1 capabilities | i ASIC
  Ports-in-ASIC (Sub-port ASIC) : 1-2 (1)
R1#show interfaces te4/2 capabilities | i ASIC
  Ports-in-ASIC (Sub-port ASIC) : 1-2 (2)
R1#show interfaces te4/3 capabilities | i ASIC
  Ports-in-ASIC (Sub-port ASIC) : 3-4 (3)
R1#show interfaces te4/4 capabilities | i ASIC
  Ports-in-ASIC (Sub-port ASIC) : 3-4 (4)

So I’m wondering if by having Te4/4 configured with “mls qos trust
xxx” Te4/3 does too, from cisco.com "In the WS-X6704-10GE line card,
there are two port ASICs each supporting 2 x 10 Gigabit Ethernet
ports".

R1#show fabric fpoe interface te4/1
fpoe for TenGigabitEthernet4/1 is 7
R1#show fabric fpoe interface te4/2
fpoe for TenGigabitEthernet4/2 is 7
R1#show fabric fpoe interface te4/3
fpoe for TenGigabitEthernet4/3 is 6
R1#show fabric fpoe interface te4/4
fpoe for TenGigabitEthernet4/4 is 6

R1#show asic-version slot 4
Module in slot 4 has 3 type(s) of ASICs
        ASIC Name      Count      Version
            JANUS          2      (1.0)
              SSA          2      (9.0)
           ROHINI          4      (1.6)


Te4/3 & 4/4 are on the same fabric channel, and this card has 2 JANUS
ASICs however the card has 4 ROHINI ASICs which I thought were the
port ASICs so it does have 1 ASIC port port? So I'm not sure if my
theory is correct (and I can't disable QoS on Te4/4 since it's a link
between chassis). I have asked TAC if this theory is true, they just
skipped over it.


If anyone knows about these ASICs in more detail, I’m all ears.

Cheers,
James.


[1] http://www.cisco.com/c/en/us/td/docs/routers/7600/ios/15S/configuration/guide/7600_15_0s_book/qos.html#pgfId-1727470

[2] http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6500-series-switches/prod_white_paper0900aecd80673385.html