[c-nsp] ES20 - Port Queues

Byrd, William will at thoughtcrime.net
Wed Sep 30 09:17:20 EDT 2009


I usually don't reply to the list but I think this information could save
a lot of hours of someone's life.

The ES20 is a WAN card and not a switch card like the 6748 although it
does have switch card functionality. The QoS we use on these cards is all
done with the MQC and the queues are completely unlike any other Cisco
card.

Here's what we dug out on these cards after a protracted TAC case and in
depth work with the Cisco BU.

Full disclosure: This information is from a document that a colleague put
together here after working with TAC so I can't take credit for it. It is
tested and known to be working however and I have slightly edited it to
remove the company specific information.

Excerpt from the document:

ES20 QoS Queue Limit

The queue limit is the amount of buffer or queue space allocated to a
class of traffic in a service policy.  The queue limit for the ES20 line
card is designed around a 256 byte datagram buffer space rather than an
actual per packet buffer as the syntax indicates.  This means that a 1500
byte packet will take up 6 “packet” buffers in the class’s queue. 
If traffic for a class arrives at a higher rate than the queue can be
emptied the queue begins to fill.  Once the queue is full the traffic will
begin to tail drop in which all traffic arriving will be dropped until the
queue empties.  This indicates that if a class bursts traffic above the
queue limit (even if it is below the reserved bandwidth) the class will
begin to drop traffic.  The default queue limit is calculated
automatically and we have found that the default queue limit is not always
sufficient to accommodate the class’s reserved bandwidth.  To calculate
the appropriate queue limit for a class’s bandwidth reservation use the
following equation:

Kbps x 1000/8 = bytes per second

Once you have the kilobits converted to bytes you must then calculate the
queue limit as follows:

Bytes/256 = queue limit

An example of this issue arose on a subrated Gigabit Ethernet service
policy on a DS3 over Gigabit Ethernet link.

  Policy Map SubRateGE-DS3-ES20
    Class class-default
      Average Rate Traffic Shaping
      cir 39000000 (bps)
      service-policy IPUPLINK-ES20

  Policy Map IPUPLINK-ES20
    Class VOICE-RTP
      priority
     police cir 19500000 bc 609375 be 609375
       conform-action transmit
       exceed-action drop
    Class VOICE-SIGNALLING
      bandwidth 3900 (kbps)
    Class MGMT
      bandwidth 1950 (kbps)
    Class PREMIUM-CUSTOMER
      bandwidth 3900 (kbps)
    Class ROUTING
      bandwidth 1950 (kbps)
    Class class-default

This is an example of a nested or hierarchical policy in which the
“parent” (SubRateGE-DS3-ES20) calls on a “child” (IPUPLINK-ES20)
policy.  In this scenario the parent policy is shaping the traffic to a
rate of 39Mbps while the child policy allocates that bandwidth to each
class of traffic proportionate to the class’s bandwidth reservation.

The first class specified VOICE-RTP is given priority due to the delay
sensitive nature of voice traffic and policed to 19500Kbps which is 50% of
the available bandwidth.  The traffic is policed to prevent the voice
traffic from starving the other classes.  The remaining defined classes
receive a bandwidth reservation of 1950Kbps or 3900Kbps which is 5% and
10% of the available bandwidth.  The remaining bandwidth is available to
the class-default which matches any traffic not matched in a more specific
class.

When we let the IOS calculate the queue limits for us, we ended up with
the following queue limits:

router#show policy-map interface g9/0/4.3201
 GigabitEthernet9/0/4.3201

  Service-policy output: SubRateGE-DS3-ES20

  Counters last updated 00:00:00 ago

    Class-map: class-default (match-any)
      159917414 packets, 82219665048 bytes
      30 second offered rate 1589000 bps, drop rate 1000 bps
      Match: any
      Queueing
      queue limit 9750 packets
      (queue depth/total drops/no-buffer drops) 0/88553/0
      (pkts output/bytes output) 159829710/82141730024

      shape (average) cir 39000000, bc 156000, be 156000
      target shape rate 39000000

      Service-policy : IPUPLINK-ES20

      Counters last updated 00:00:00 ago

        queue stats for all priority classes:
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 80837512/17506671967

        Class-map: VOICE-RTP (match-any)
          80836920 packets, 17506542957 bytes
          30 second offered rate 720000 bps, drop rate 0 bps
          Match: ip precedence 7
          Match: mpls experimental topmost 7
          Match: ip precedence 5
          Match: mpls experimental topmost 5
          Priority: Strict, burst bytes 487500
          police:
              cir 19500000 bps, bc 609375 bytes, be 609375 bytes
            conformed 80837512 packets, 17506671967 bytes; actions:
              transmit
            exceeded 0 packets, 0 bytes; actions:
              drop
            violated 0 packets, 0 bytes; actions:
              drop
            conformed 573000 bps, exceed 0 bps, violate 0 bps

        Class-map: VOICE-SIGNALLING (match-any)
          155279 packets, 89526994 bytes
          30 second offered rate 0 bps, drop rate 0 bps
          Match: ip precedence 3
          Match: mpls experimental topmost 3
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 155279/89526994

          bandwidth 3900 kbps

        Class-map: MGMT (match-any)
          50389 packets, 3274933 bytes
          30 second offered rate 0 bps, drop rate 0 bps
          Match: access-group name MGMT-TELNET
          Match: ip precedence 2
          Match: mpls experimental topmost 2
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 50389/3274933

          bandwidth 1950 kbps

        Class-map: PREMIUM-CUSTOMER (match-any)
          12515000 packets, 14194344547 bytes
          30 second offered rate 3000 bps, drop rate 0 bps
          Match: ip precedence 1
          Match: mpls experimental topmost 1
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 12515002/14194345277

          bandwidth 3900 kbps

        Class-map: ROUTING (match-any)
          871200 packets, 187824161 bytes
          30 second offered rate 5000 bps, drop rate 0 bps
          Match: ip precedence 6
          Match: mpls experimental topmost 6
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 871216/187827176

          bandwidth 1950 kbps

        Class-map: class-default (match-any)
          65488626 packets, 50238151456 bytes
          30 second offered rate 856000 bps, drop rate 1000 bps
          Match: any
          Queueing
          queue limit 66 packets
          (queue depth/total drops/no-buffer drops) 0/88553/0
          (pkts output/bytes output) 65400312/50160083677

>From the above output you can see that the IOS calculated a queue limit of
66 “packets” for each of the defined classes and the class-default. 
Using the above method we can see that this leaves only enough buffer
space to accommodate 135Kbps for each class.   This was causing drops
during bursts of traffic that were not exceeding the available bandwidth.

If you apply the above calculation to the reserved bandwidths you get the
following queue limits:

Reservation of 1950 Kbps:

1950 x 1000 = 1950000
1950000/8 = 243750
243750/256 = 952.1484375

1950 Kbps requires 952.1484375 256 byte buffers

Reservation of 3900 Kbps

3900 x 1000 = 3900000
39000000/8 = 4875000
487500/256 = 1904.296875

3900 Kbps requires 1904.296875 256 byte buffers

We will then add a little extra buffer space to each class to ensure
adequate buffers and simplify configuration.

1950 = 1000
3900 = 2000

That leaves us with allocating buffer space to the class-default.  If you
calculate the buffer space needed for the entire shape rate it will come
out to queue limit of approximately 20000. If you subtract the buffer
space already allocated to the other classes you will be left with 14000. 
This will be assigned as the queue limit for the class-default.

In order to change the queue limit for a class use the following procedure:

router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
router(config)#policy-map IPUPLINK-ES20
router(config-pmap)#class VOICE-SIGNALLING
router(config-pmap-c)#queue-limit 2000
router(config-pmap-c)#end

Below is the IPUPLINK-ES20 policy map with the adjusted queue limits:

  Policy Map IPUPLINK-ES20
    Class VOICE-RTP
      priority
     police cir 19500000 bc 609375 be 609375
       conform-action transmit
       exceed-action drop
    Class VOICE-SIGNALLING
      bandwidth 3900 (kbps)
      queue-limit 2000 packets
    Class MGMT
      bandwidth 1950 (kbps)
      queue-limit 1000 packets
    Class PREMIUM-CUSTOMER
      bandwidth 3900 (kbps)
      queue-limit 2000 packets
    Class ROUTING
      bandwidth 1950 (kbps)
      queue-limit 1000 packets
    Class class-default
      queue-limit 14000 packets

Your mileage may vary but this fixed a particularly troublesome issue for
us and I think everyone should keep this in mind when working with these
cards.

We had some very bad degraded service events where we thought we had
plenty of bandwidth during a failure scenario but in reality could only
get approximately 75% of the loop bandwidth due to the queue depths we had
configured on our service policies prior to these changes.

-Will

----- Original Message -----
From: "Anthony McGarry" <anthony.mcgarry at plannet21.ie>
Sent: Wed, September 30, 2009 4:59
Subject:Re: [c-nsp] ES20 - Port Queues

Hi,

I have a 7609 with a WS-X6748-GE-TX. When I show the port capabilities I
can see rx-(1q8t), tx-(1p3q8t)
If I show queueing on an interface I see all the WRED and WRR queue
configuration.
This is great, I can make changes and view changes.

I also have an ES20 and ES20+ line card in the chassis. When I show the
port capabilities I see x-(NotDef-t), tx-(NotDef-t)
When I show queueing on an interface I see nothing for the ES20. For the
ES20+ I get

Interface GigabitEthernet1/1 queueing strategy:  Weighted Round-Robin
  Port QoS is enabled
  Port is untrusted
  Extend trust state: not trusted [COS = 0]
  Default COS is 0

So the ES20 tells me nothing and the ES20+ only shows a subset of what I
see with the WS-X6748-GE-TX.

Is there something I am missing in my config to show what queues I have
available on the ES line cards '(1p3q8t)'
 From reading the line card configuration guides I can see ingress
scheduling is not supported and egress is but when I configure a egress
policy-map with my random-detect configuration I still see no queueing
info for the ports.

Thanks
Anthony



_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


----- End of original message -----



More information about the cisco-nsp mailing list