[c-nsp] ES20 - Port Queues
Anthony McGarry
anthony.mcgarry at plannet21.ie
Wed Sep 30 11:15:49 EDT 2009
Will,
Thanks for the very complete explanation.
I have a similar policy-map attached to the interfaces on the ES20s.
For this particular 1GB link I can see the queue-limit by default is not
sufficient for the bandwidth of 120000k I have for this class.
Class-map: qos-cos4 (match-any)
7403774 packets, 10083940188 bytes
5 minute offered rate 88244000 bps, drop rate 0000 bps
Match: cos 4
Queueing
queue limit 6635 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 7450680/10147826160
bandwidth 120000 kbps
With this queue limit I could only pass 13588k of traffic before tail drops
However I see a 5 min offered rate of 88224k with no tail drops.
Is this because the link is not under congestion?
From the calculations below I have now set the queue limit manually
120000 x 1000 = 120000000
120000000/8 = 15000000
15000000/256 = 58593.75
Class-map: qos-cos4 (match-any)
12103420 packets, 16484858040 bytes
5 minute offered rate 96063000 bps, drop rate 0000 bps
Match: cos 4
Queueing
queue limit 60000 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 12103420/16484858040
bandwidth 120000 kbps
I also noticed on my priority queue that the queue limit is always set
to 66 packets by default.
I am assuming this is to reduce latency for rtp but by the above logic
that would equal 135.168k before tail drops
and I am seeing 287k with no drops.
queue stats for all priority classes:
Queueing
queue limit 66 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 229339/49963294
Class-map: qos-cos5 (match-any)
229339 packets, 49963294 bytes
5 minute offered rate 287000 bps, drop rate 0000 bps
Match: cos 5
police:
cir 50000000 bps, bc 1562500 bytes
conformed 226069 packets, 49250874 bytes; actions:
transmit
exceeded 0 packets, 0 bytes; actions:
drop
conformed 286000 bps, exceed 0000 bps
Priority: Strict, b/w exceed drops: 0
I believe you when you say the queue limit could cause drops as the
logic is sound but I can't see any impact from the outputs above. maybe
I' missing something.
Thanks
Anthony
Byrd, William wrote:
> I usually don't reply to the list but I think this information could save
> a lot of hours of someone's life.
>
> The ES20 is a WAN card and not a switch card like the 6748 although it
> does have switch card functionality. The QoS we use on these cards is all
> done with the MQC and the queues are completely unlike any other Cisco
> card.
>
> Here's what we dug out on these cards after a protracted TAC case and in
> depth work with the Cisco BU.
>
> Full disclosure: This information is from a document that a colleague put
> together here after working with TAC so I can't take credit for it. It is
> tested and known to be working however and I have slightly edited it to
> remove the company specific information.
>
> Excerpt from the document:
>
> ES20 QoS Queue Limit
>
> The queue limit is the amount of buffer or queue space allocated to a
> class of traffic in a service policy. The queue limit for the ES20 line
> card is designed around a 256 byte datagram buffer space rather than an
> actual per packet buffer as the syntax indicates. This means that a 1500
> byte packet will take up 6 “packet” buffers in the class’s queue.
> If traffic for a class arrives at a higher rate than the queue can be
> emptied the queue begins to fill. Once the queue is full the traffic will
> begin to tail drop in which all traffic arriving will be dropped until the
> queue empties. This indicates that if a class bursts traffic above the
> queue limit (even if it is below the reserved bandwidth) the class will
> begin to drop traffic. The default queue limit is calculated
> automatically and we have found that the default queue limit is not always
> sufficient to accommodate the class’s reserved bandwidth. To calculate
> the appropriate queue limit for a class’s bandwidth reservation use the
> following equation:
>
> Kbps x 1000/8 = bytes per second
>
> Once you have the kilobits converted to bytes you must then calculate the
> queue limit as follows:
>
> Bytes/256 = queue limit
>
> An example of this issue arose on a subrated Gigabit Ethernet service
> policy on a DS3 over Gigabit Ethernet link.
>
> Policy Map SubRateGE-DS3-ES20
> Class class-default
> Average Rate Traffic Shaping
> cir 39000000 (bps)
> service-policy IPUPLINK-ES20
>
> Policy Map IPUPLINK-ES20
> Class VOICE-RTP
> priority
> police cir 19500000 bc 609375 be 609375
> conform-action transmit
> exceed-action drop
> Class VOICE-SIGNALLING
> bandwidth 3900 (kbps)
> Class MGMT
> bandwidth 1950 (kbps)
> Class PREMIUM-CUSTOMER
> bandwidth 3900 (kbps)
> Class ROUTING
> bandwidth 1950 (kbps)
> Class class-default
>
> This is an example of a nested or hierarchical policy in which the
> “parent” (SubRateGE-DS3-ES20) calls on a “child” (IPUPLINK-ES20)
> policy. In this scenario the parent policy is shaping the traffic to a
> rate of 39Mbps while the child policy allocates that bandwidth to each
> class of traffic proportionate to the class’s bandwidth reservation.
>
> The first class specified VOICE-RTP is given priority due to the delay
> sensitive nature of voice traffic and policed to 19500Kbps which is 50% of
> the available bandwidth. The traffic is policed to prevent the voice
> traffic from starving the other classes. The remaining defined classes
> receive a bandwidth reservation of 1950Kbps or 3900Kbps which is 5% and
> 10% of the available bandwidth. The remaining bandwidth is available to
> the class-default which matches any traffic not matched in a more specific
> class.
>
> When we let the IOS calculate the queue limits for us, we ended up with
> the following queue limits:
>
> router#show policy-map interface g9/0/4.3201
> GigabitEthernet9/0/4.3201
>
> Service-policy output: SubRateGE-DS3-ES20
>
> Counters last updated 00:00:00 ago
>
> Class-map: class-default (match-any)
> 159917414 packets, 82219665048 bytes
> 30 second offered rate 1589000 bps, drop rate 1000 bps
> Match: any
> Queueing
> queue limit 9750 packets
> (queue depth/total drops/no-buffer drops) 0/88553/0
> (pkts output/bytes output) 159829710/82141730024
>
> shape (average) cir 39000000, bc 156000, be 156000
> target shape rate 39000000
>
> Service-policy : IPUPLINK-ES20
>
> Counters last updated 00:00:00 ago
>
> queue stats for all priority classes:
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/0/0
> (pkts output/bytes output) 80837512/17506671967
>
> Class-map: VOICE-RTP (match-any)
> 80836920 packets, 17506542957 bytes
> 30 second offered rate 720000 bps, drop rate 0 bps
> Match: ip precedence 7
> Match: mpls experimental topmost 7
> Match: ip precedence 5
> Match: mpls experimental topmost 5
> Priority: Strict, burst bytes 487500
> police:
> cir 19500000 bps, bc 609375 bytes, be 609375 bytes
> conformed 80837512 packets, 17506671967 bytes; actions:
> transmit
> exceeded 0 packets, 0 bytes; actions:
> drop
> violated 0 packets, 0 bytes; actions:
> drop
> conformed 573000 bps, exceed 0 bps, violate 0 bps
>
> Class-map: VOICE-SIGNALLING (match-any)
> 155279 packets, 89526994 bytes
> 30 second offered rate 0 bps, drop rate 0 bps
> Match: ip precedence 3
> Match: mpls experimental topmost 3
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/0/0
> (pkts output/bytes output) 155279/89526994
>
> bandwidth 3900 kbps
>
> Class-map: MGMT (match-any)
> 50389 packets, 3274933 bytes
> 30 second offered rate 0 bps, drop rate 0 bps
> Match: access-group name MGMT-TELNET
> Match: ip precedence 2
> Match: mpls experimental topmost 2
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/0/0
> (pkts output/bytes output) 50389/3274933
>
> bandwidth 1950 kbps
>
> Class-map: PREMIUM-CUSTOMER (match-any)
> 12515000 packets, 14194344547 bytes
> 30 second offered rate 3000 bps, drop rate 0 bps
> Match: ip precedence 1
> Match: mpls experimental topmost 1
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/0/0
> (pkts output/bytes output) 12515002/14194345277
>
> bandwidth 3900 kbps
>
> Class-map: ROUTING (match-any)
> 871200 packets, 187824161 bytes
> 30 second offered rate 5000 bps, drop rate 0 bps
> Match: ip precedence 6
> Match: mpls experimental topmost 6
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/0/0
> (pkts output/bytes output) 871216/187827176
>
> bandwidth 1950 kbps
>
> Class-map: class-default (match-any)
> 65488626 packets, 50238151456 bytes
> 30 second offered rate 856000 bps, drop rate 1000 bps
> Match: any
> Queueing
> queue limit 66 packets
> (queue depth/total drops/no-buffer drops) 0/88553/0
> (pkts output/bytes output) 65400312/50160083677
>
> From the above output you can see that the IOS calculated a queue limit of
> 66 “packets” for each of the defined classes and the class-default.
> Using the above method we can see that this leaves only enough buffer
> space to accommodate 135Kbps for each class. This was causing drops
> during bursts of traffic that were not exceeding the available bandwidth.
>
> If you apply the above calculation to the reserved bandwidths you get the
> following queue limits:
>
> Reservation of 1950 Kbps:
>
> 1950 x 1000 = 1950000
> 1950000/8 = 243750
> 243750/256 = 952.1484375
>
> 1950 Kbps requires 952.1484375 256 byte buffers
>
> Reservation of 3900 Kbps
>
> 3900 x 1000 = 3900000
> 39000000/8 = 4875000
> 487500/256 = 1904.296875
>
> 3900 Kbps requires 1904.296875 256 byte buffers
>
> We will then add a little extra buffer space to each class to ensure
> adequate buffers and simplify configuration.
>
> 1950 = 1000
> 3900 = 2000
>
> That leaves us with allocating buffer space to the class-default. If you
> calculate the buffer space needed for the entire shape rate it will come
> out to queue limit of approximately 20000. If you subtract the buffer
> space already allocated to the other classes you will be left with 14000.
> This will be assigned as the queue limit for the class-default.
>
> In order to change the queue limit for a class use the following procedure:
>
> router#configure terminal
> Enter configuration commands, one per line. End with CNTL/Z.
> router(config)#policy-map IPUPLINK-ES20
> router(config-pmap)#class VOICE-SIGNALLING
> router(config-pmap-c)#queue-limit 2000
> router(config-pmap-c)#end
>
> Below is the IPUPLINK-ES20 policy map with the adjusted queue limits:
>
> Policy Map IPUPLINK-ES20
> Class VOICE-RTP
> priority
> police cir 19500000 bc 609375 be 609375
> conform-action transmit
> exceed-action drop
> Class VOICE-SIGNALLING
> bandwidth 3900 (kbps)
> queue-limit 2000 packets
> Class MGMT
> bandwidth 1950 (kbps)
> queue-limit 1000 packets
> Class PREMIUM-CUSTOMER
> bandwidth 3900 (kbps)
> queue-limit 2000 packets
> Class ROUTING
> bandwidth 1950 (kbps)
> queue-limit 1000 packets
> Class class-default
> queue-limit 14000 packets
>
> Your mileage may vary but this fixed a particularly troublesome issue for
> us and I think everyone should keep this in mind when working with these
> cards.
>
> We had some very bad degraded service events where we thought we had
> plenty of bandwidth during a failure scenario but in reality could only
> get approximately 75% of the loop bandwidth due to the queue depths we had
> configured on our service policies prior to these changes.
>
> -Will
>
> ----- Original Message -----
> From: "Anthony McGarry" <anthony.mcgarry at plannet21.ie>
> Sent: Wed, September 30, 2009 4:59
> Subject:Re: [c-nsp] ES20 - Port Queues
>
> Hi,
>
> I have a 7609 with a WS-X6748-GE-TX. When I show the port capabilities I
> can see rx-(1q8t), tx-(1p3q8t)
> If I show queueing on an interface I see all the WRED and WRR queue
> configuration.
> This is great, I can make changes and view changes.
>
> I also have an ES20 and ES20+ line card in the chassis. When I show the
> port capabilities I see x-(NotDef-t), tx-(NotDef-t)
> When I show queueing on an interface I see nothing for the ES20. For the
> ES20+ I get
>
> Interface GigabitEthernet1/1 queueing strategy: Weighted Round-Robin
> Port QoS is enabled
> Port is untrusted
> Extend trust state: not trusted [COS = 0]
> Default COS is 0
>
> So the ES20 tells me nothing and the ES20+ only shows a subset of what I
> see with the WS-X6748-GE-TX.
>
> Is there something I am missing in my config to show what queues I have
> available on the ES line cards '(1p3q8t)'
> From reading the line card configuration guides I can see ingress
> scheduling is not supported and egress is but when I configure a egress
> policy-map with my random-detect configuration I still see no queueing
> info for the ports.
>
> Thanks
> Anthony
>
>
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
> ----- End of original message -----
>
>
More information about the cisco-nsp
mailing list