[c-nsp] MPLS qos on 6500/sup720 - setting EXP when it shouldn't?

[Phil Mayers]

We're about to start the complex task of using QoS on our MPLS core 
network (the main reason is that we're seeing drops on downstream LAN 
switches with tiny buffers, and want the upstream core to egress the 
"important" packets first, hopefully mitigating this)

Upon investigation I'm seeing traffic flowing across the network with 
MPLS EXP != 0. This is odd, because we have QoS enabled (for CoPP) but 
with all ports set to untrusted - my understanding is that the internal 
DSCP (and thus transmit CoS/EXP) should therefore be zero.

For example:

#sh mls qos last module 8

  ----- Module [8] -----
QoS last packet policing information:
     ---------------------------------------------------------------------
     Packet was transmitted
     Packet L3 Prot: 0, packet length: 300, dont_plc: yes
     Input COS: 0, TOS/DSCP: 0x0/0
     Output TOS/DSCP: 0x0/0[unchanged]   Output COS: 0[unchanged]
     Output MPLS EXP (if outgoing packet is MPLS): 3
     ---------------------------------------------------------------------
     Aggregate policer index: Input - 0(none), Output - 0(none)
     thr_hi_ip: 0x0   leak_hi_ip: 0x3FF drop_ena_ag_ip: no
     thr_lo_ip: 0x0   leak_lo_ip: 0x3FF
     thr_hi_op: 0x0   leak_hi_op: 0x3FF drop_ena_ag_op: no
     thr_lo_op: 0x0   leak_lo_op: 0x3FF
     ---------------------------------------------------------------------
     Microflow policer index: Input - 0(none), Output - 0(none)
     ---------------------------------------------------------------------
     Netflow policer: nf_hit: yes, nf_addr: 0x109D9, snap-shot matches
     NT&NS: l3_prot: 6(0), 155.198.77.100.0xDA5D ==> 155.198.63.71.0x01BD
     NT&NS: vlan_number: 4D
     cr_tstamp: 0x2EBBE8, ls_tstamp_nf: 0xBB39BE, tos_nf: 0x0, 
use_nf_thr: no
     thrshld_nf: 0x0, leak_nf: 0x0, drop_ena_nf: no,
     bkt_nf: 0x0, pkt_cnt_nf: 0x0, thr_excd_cnt_nf: 0x0
     ---------------------------------------------------------------------
     acl_redirect: no, acl_rdt_rpf: no, acl_drop: no, police_drop: no 
no_route: no
     err_pkt: no, l3_flow: yes, acl_bridge: no, rpf_fail: no, hw_error: 
no ip_dont_sc: no
     to_router: no, recirculated: no, to_be_recirculated:no
     apply_qos_ip: no, apply_qos_op: no, drop_ip: no drop_op: no
     plc_level2: no, plc_level3: no, tos_i: 0x0 vmap_data: 0x0

As you can see, the original packet has CoS and DSCP == 0. Vlan 0x4D is 
Vlan 77, which is tagged on a single switchport:

interface GigabitEthernet8/23
  description xxx
  switchport
  switchport trunk encapsulation do
  switchport trunk native vlan 2005
  switchport trunk allowed vlan 77,xxx
  switchport mode trunk
  switchport nonegotiate
  logging event spanning-tree statu
  storm-control broadcast level 0.1
  no cdp enable

#sh queueing interface g8/23
Interface GigabitEthernet8/23 queueing strategy:  Weighted Round-Robin
   Port QoS is enabled
Trust boundary disabled

   Port is untrusted
   Extend trust state: not trusted [COS = 0]
   Default COS is 0

...as you can see, no QoS config on this port. I'm baffled where these 
non-zero EXP are coming from; I'm seeing a mix of values (1-7) although 
most traffic is zero.

Any ideas?