[c-nsp] Weird throughput issue

[Andrew Miehs]

On Sun, Jul 24, 2016 at 3:40 PM, Curtis Piehler <cpiehler2 at gmail.com> wrote:
...
>
> Customer Site (2911 Router) -> ILEC -> Transport Provider -> Carrier NNI
> (1G Copper) -> Cat4900 -> Cat6504 -> ASR9k
>
> The ASR9k terminates the IP for the customer so the in between devices are
> Layer 2 trunking all the way through.
>
> Latency from the customer site to the Cat infrastructure is only 3-4ms.
> Once it trunks over to the ASR9k for IP termination it increases to 17ms (A
> different facility).  We have several IPERF servers to test from however
> the latency never exceeds 40ms to any given server.
..

How are you measuring this latency on the Cat 4900 if the IP remote
end of the L3 circuit is on the ASR9k

> When performing IPERF tests we can not pass anything more than 10M to the
> site (down stream).  Upload from the site maxes out around 50-60M when we
> test to anything behind the ASR9k IP network.  If we test to an IPERF
> server off of the Catalyst infrastructure where the NNI comes into we can
> pass full throughput.  We have tried all different size windows on IEPRF
> which do not yield any improved results.  The further away we get from the
> site (again not exceeding 40ms) the less throughput is seen.  UDP tests to
> the site are ok as we can force 85-90M and the site receives it with very
> little packet loss.  The issue is just TCP.

As you are only seeing this with TCP, it is usually a factor of
receiver window size (and latency) or packet loss causing congestion
avoidance to kick in. The fact that latency increased the problem
indicates that it could very likely be a receiver window issue. Check
whether window scaling has been enabled in the server and client
operating systems.

You might want to try out this calculator, and use 65535 for "TCP
window (RWND) size"

http://wintelguy.com/wanperf.pl



-- Andrew