[nsp] cisco 6500 sup1a w/ pfc+msfc: VLAN-based rate-limiting problem ?!

[Alexander Lucke]

Hi,

we operate a Cisco 6500 with dual-Sup1A/PFC+MSFC layer3 distribution switch
(IOS mode) with some Cisco 3548 rack switches (connected via gigachannel to
the 6500) in a data center environment.

	Upstream <-----[6500]===(gigachannel)===[3548]-----> Servers

I need to limit the upstream traffic for specific customers (i.e. all the
servers in one VLAN). I read a lot of documentation and tried everything I
found including PFC QoS (policy-maps).

Of course both incoming and outgoing would be nice, but I also read that most
things only work with incoming traffic. As we talk about a server farm, most
traffic is outgoing (i.e. incoming for the switch). So limiting the incoming
bandwidth of a port/vlan would be enough.

The problem is that I can configure it but it doesn't work.

Let's look at the configuration:

The gigachannel ports of the 6500 are configured as dot1Q-trunk layer2
interfaces:

	interface Port-channel1
	 no ip address
	 mls qos vlan-based
	 switchport
	 switchport trunk encapsulation dot1q
	 switchport trunk allowed vlan 200-209,213-215,217,218,220,221
	 switchport mode trunk
	!
	interface GigabitEthernet1/2
	 no ip address
	 mls qos statistics-export
	 mls qos vlan-based
	 switchport
	 switchport trunk encapsulation dot1q
	 switchport trunk allowed vlan 200-209,213-215,217,218,220,221
	 switchport mode trunk
	 channel-group 1 mode on
	!
	interface GigabitEthernet2/2
	 no ip address
	 mls qos statistics-export
	 mls qos vlan-based
	 switchport
	 switchport trunk encapsulation dot1q
	 switchport trunk allowed vlan 200-209,213-215,217,218,220,221
	 switchport mode trunk
	 channel-group 1 mode on
	!

I set up a police map:

	class-map match-any cmap-vlan217
	  match access-group name acl-vlan217
	!
	!
	policy-map pmap-fixed-1M
	  class cmap-vlan217
	    police aggregate qos-fix1M
	!
	mls flow ip full
	mls flow ipx destination
	mls nde sender
	mls qos statistics-export interval 120
	mls qos statistics-export aggregate-policer qos-fix2M
	mls qos statistics-export class-map cmap-vlan217
	mls qos statistics-export destination xxx.xxx.xxx.xxx syslog facility

		local6 severity info 
	mls qos statistics-export
	mls qos aggregate-policer qos-fix1M 1000000 62000 conform-action 
		transmit exceed-action drop
	mls qos

the access list matches all:

	ip access-list standard acl-vlan217
	 permit any
	!

The I attached the policy-map to the it to the customer vlan:

	interface Vlan217
	 description #### customer servers ####
	 ip address xxx.xxx.xxx.xxx yyy.yyy.yyy.yyy
	 ip access-group CUSTOMER-IN in
	 no ip redirects
	 no ip unreachables
	 ip route-cache flow
	 service-policy input pmap-fixed-1M
	!

When I do a "sh mls qos ip", I see that no traffic matches my class-map:

	colo-sw1.berlin1#sh mls qos ip                    
	 In QoS Summary [IP]:   (* - shared aggregates, F - install error)
	
	       Int  Class-map DSCP  AgId Trust FlId AgForward-Pk
AgPoliced-Pkts Last-See
	
-----------------------------------------------------------------------------
--
	      Vl217 cmap-vlan    0    4*     -    0            0
0 0x0       
	        All   Default    0    0*    No    0   1738971263
0 0x9A75D3  

Is there a configuration error or is vlan-based policing only supported with
PFC2 and above?

I was thinking about using the following hack: If it would not work with a
vlan interface but with a layer3 interface with my hardware, I could
physically connect two ports of my 6500 switch (let's say fasteth3/1 and
fasteth3/2) and do something like that:

	int fasteth3/1
	 desc #### layer2 vlan member - physically connected to fe3/2 ####
	 switchport
	 switchport access vlan 217
	!
	int fasteth3/2
	 desc #### layer3 upstream port - physically connected to fe3/1 ####
	 ip address xxx.xxx.xxx.xxx yyy.yyy.yyy.yyy
	 ip access-group CUSTOMER-IN in
	 no ip redirects
	 no ip unreachables
	 ip route-cache flow
	 service-policy input pmap-fixed-1M
	!

which would be a hack (I did not think about spanning tree yet but it should
be no problem to connect that two ports) but which should work.

For testing whether rate limiting works with a layer3 port or not, I did the
following:

I tried to attach the policy map to a layer3 interface which seemed to work
(the "sh mls qos ip" counters incremented) but my mrtg graph (and also a "sh
int fasteth x/y") still showed much more than 1 Mbps (about 4 Mbps, which was
normal for that port) incoming traffic on that port?!

So I'm a little confused now. Can anyone help with the configuration?

Regards,
Alexander Lucke

-- 
DNS:NET Internet Service GmbH, Ostseestraße 111, 10409 Berlin
Tel. 030-420278-22, Fax 030-420278-78, lucke at dns-net.de
http://www.dns-net.de