[c-nsp] Weird throughput issue

Wed Jul 27 00:29:23 EDT 2016

Ryan,

Thank you that fixed it!   I know ATT is known to do strict policing on
their ASE products but this other ILEC was really doing some tight
policing.   Speeds have drastically improved!   Now I only wish classic ios
went lower than 4ms Tc!

On Jul 26, 2016 8:42 AM, "Brault, Ryan" <RYAN.BRAULT at illinois.gov> wrote:

> Check your policy on the ASR9k.  We had this exact same issue with TCP on
> ASR9ks and specific transport providers.  In short, the problem was that
> the provider had a strict policer (for bursting), while we were shaping a
> sub-if on the ASR9k.  The ASR9k was bursting beyond the strict carrier
> policer, so the carrier was dropping.  Thus, TCP windowing came into play.
> The fix was to enable low-burst mode on the ASR9k and create a policy like
> this:
>
> Policy-map Default-child
>   Class class-default
>     Bandwidth percent 100
> !
> Policy-map Shape-100Mb
>    Class class-default
>      Service-policy Default-child
>      Shape average 100 mbps 1 us
>
>
> This fixed all our issues.
>
> --
> Ryan Brault
> Illinois Department of Innovation & Technology
> 815-936-4647
>
>
> -----Original Message-----
> From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of
> Curtis Piehler
> Sent: Sunday, July 24, 2016 9:41 AM
> To: Cisco Network Service Providers <cisco-nsp at puck.nether.net>
> Subject: [c-nsp] Weird throughput issue
>
> I've been scratching my head over the past week on this issue as it doesn't
> make any sense but here are the details.
>
> I have a 100M circuit that is sold through a transport provider to an ILEC
> (Type 2).  The circuit comes into one of our facilities with the following
> equipment:
>
> Customer Site (2911 Router) -> ILEC -> Transport Provider -> Carrier NNI
> (1G Copper) -> Cat4900 -> Cat6504 -> ASR9k
>
> The ASR9k terminates the IP for the customer so the in between devices are
> Layer 2 trunking all the way through.
>
> Latency from the customer site to the Cat infrastructure is only 3-4ms.
> Once it trunks over to the ASR9k for IP termination it increases to 17ms (A
> different facility).  We have several IPERF servers to test from however
> the latency never exceeds 40ms to any given server.
>
> When performing IPERF tests we can not pass anything more than 10M to the
> site (down stream).  Upload from the site maxes out around 50-60M when we
> test to anything behind the ASR9k IP network.  If we test to an IPERF
> server off of the Catalyst infrastructure where the NNI comes into we can
> pass full throughput.  We have tried all different size windows on IEPRF
> which do not yield any improved results.  The further away we get from the
> site (again not exceeding 40ms) the less throughput is seen.  UDP tests to
> the site are ok as we can force 85-90M and the site receives it with very
> little packet loss.  The issue is just TCP.
>
> We have a had a facility tech plug a laptop directly into the Cat 4900 on a
> separate access port and test back to the same IP termination the customer
> lands in our network and there are no issues.
>
> Now here is the interesting thing:
> The customer has another 100M site from us (a different carrier) that comes
> into the Cat6504 in the above path.  It trunks up to the same ASR9k that
> this site with the issues does.  No throughput issues are seen from this
> site at all.
>
> The ILEC and Transport provider have dispatched out multiple times and
> RFC2544 test end to end with no issues.  The transport provider also peeled
> the circuit off our NNI and configure it as Layer 3 back to their network
> and did RFC6544 testing with no issues.
>
> We've looked at the Cat4900 and Cat6504 configs and can find nothing that
> would inhibit this throughput.
>
> Most modern day computer OS's do dynamic TCP scaling so the bandwidth of
> distance theory shouldn't really apply here (also considering the distance
> isn't that significant).  I've tried decreasing the TCP-MSS on both WAN
> interfaces (Core and customer router) to as low as 1000 with no impact at
> all.
>
> For technical sanity sake all RJ-45 Ports on site are locked at 100M/Full
> Duplex.
>
> Does anyone have any experience with this type of scenario?
>
> Curtis
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
> CONFIDENTIALITY NOTICE: The information contained in this communication is
> confidential, may be attorney-client privileged or attorney work product,
> may constitute inside information or internal deliberative staff
> communication, and is intended only for the use of the addressee.
> Unauthorized use, disclosure or copying of this communication or any part
> thereof is strictly prohibited and may be unlawful. If you have received
> this communication in error, please notify the sender immediately by return
> e-mail and destroy this communication and all copies thereof, including all
> attachments. Receipt by an unintended recipient does not waive
> attorney-client privilege, attorney work product privilege, or any other
> exemption from disclosure.
>