[outages] Level 3 issues

Tue Jan 22 23:34:32 EST 2013

Did anyone ever get resolution on this issue?

-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of Matthew Laughlin
Sent: Tuesday, January 22, 2013 4:17 PM
To: Jeremy Chadwick
Cc: outages at outages.org
Subject: Re: [outages] Level 3 issues

That might explain why I seeing IP SLA time-outs to three disparate dst over these links this morning.

Matt

On Jan 22, 2013, at 12:45 PM, Jeremy Chadwick <jdc at koitsu.org> wrote:

> 1. Usually Level 3-owned routers use ICMP prioritisation; routing
> (forwarding) has priority over this.
> 
> 2. Many traceroute implementations use UDP for their probe packets (in 
> attempt to solicit ICMP TTL exceeded messages to learn the network 
> topology); please make sure you're using pure ICMP all the way through 
> if possible.
> 
> 3. Your below traces indicate a destination IP of 64.208.27.65, which 
> is not sufficient (ICMP prioritisation will be used to explain what 
> you're seeing, even if there is a real issue).  Please pick a 
> destination IP of an actual host that's somewhere behind that router 
> which will respond to ICMP packets reliably.
> 
> You need to see if the packet loss "trickles downward" past that hop 
> to every subsequent hop after that point (including the destination 
> host), or if high latency, if the latency "trickles downward" as well.
> 
> Here's an example of a bogus claim as a result of ICMP prio:
> 
> === Mon Jan 21 01:39:00 PST 2013  (1358761140)
> HOST: icarus.home.lan             Loss%   Snt   Rcv  Last   Avg  Best  Wrst
>  1.|-- 192.168.1.1                0.0%    40    40   0.3   0.3   0.2   0.4
>  2.|-- 67.180.84.1                0.0%    40    40  21.8  23.0  11.2  35.4
>  3.|-- 68.85.191.249              0.0%    40    40  10.1   9.8   8.1  14.2
>  4.|-- 69.139.198.82              0.0%    40    40  21.0  17.2   9.9  22.8
>  5.|-- 68.86.90.93                0.0%    40    40  23.1  19.2  11.8  30.3
>  6.|-- 68.86.87.190              15.0%    40    34  22.3  25.9  19.9 122.6
>  7.|-- 75.149.231.238            37.5%    40    25  56.3  26.5  20.0  56.3
>  8.|-- 67.199.135.102             0.0%    40    40  21.1  25.8  20.9  49.0
>  9.|-- 206.125.172.42             0.0%    40    40  20.7  21.3  19.8  22.5
> === END
> 
> While here's a real issue:
> 
> === Fri Jan  4 07:27:00 PST 2013  (1357313220)
> HOST: icarus.home.lan             Loss%   Snt   Rcv  Last   Avg  Best  Wrst
>  1.|-- 192.168.1.1                0.0%    40    40   0.2   0.3   0.2   0.4
>  2.|-- 67.180.84.1               92.5%    40     3  16.0  16.7  11.5  22.7
>  3.|-- 68.85.191.249             92.5%    40     3   9.4   9.3   8.7   9.7
>  4.|-- 69.139.198.82             90.0%    40     4  19.1  23.0  12.6  39.2
>  5.|-- 68.86.90.93               90.0%    40     4  12.4  14.8  12.4  17.5
>  6.|-- 68.86.87.190              90.0%    40     4  48.3  28.6  21.6  48.3
>  7.|-- 75.149.231.238            90.0%    40     4  32.7  23.9  20.4  32.7
>  8.|-- 67.199.135.102            90.0%    40     4 145.7  52.5  21.3 145.7
>  9.|-- 206.125.172.42            90.0%    40     4  20.8  21.3  19.9  22.4
> === END
> 
> All of this is covered here (read do not skim):
> 
> http://www.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Tracero
> ute_N47_Sun.pdf
> 
> --
> | Jeremy Chadwick                                   jdc at koitsu.org |
> | UNIX Systems Administrator                http://jdc.koitsu.org/ |
> | Mountain View, CA, US                                            |
> | Making life hard for others since 1977.             PGP 4BD6C0CB |
> 
> On Tue, Jan 22, 2013 at 11:40:39AM -0600, Jeremie Chism wrote:
>> Seeing intermittent issues with level 3 router at 64.208.27.65
>> 
>> Hop (ms) (ms) (ms)     IP Address Host name
>> 1   0   0   0      8.9.232.73  xe-5-3-0.edge3.dallas1.level3.net
>> 2   0   0   0      4.69.145.141  ae-3-80.edge4.dallas3.level3.net
>> 3   8   0   0      4.68.63.146  gblx-level3-40g.dallas3.level3.net
>> 4   0   0   0      67.16.150.190  ae2-80g.csr1.dal2.gblx.net
>> 5   0   0   0      64.208.27.65  te1-3-10g.ar3.dal2.gblx.net
>> 
>> Hop (ms) (ms) (ms)     IP Address Host name
>> 1   1   0   0      8.9.232.73  xe-5-3-0.edge3.dallas1.level3.net
>> 2   0   0   0      4.69.145.141  ae-3-80.edge4.dallas3.level3.net
>> 3   0   0   0      4.68.63.146  gblx-level3-40g.dallas3.level3.net
>> 4   0   0   0      67.16.150.190  ae2-80g.csr1.dal2.gblx.net
>> 5   7   1   208      64.208.27.65  te1-3-10g.ar3.dal2.gblx.net
>> Trace complete
>> 
>> Ping times from 8ms to 330ms.
>> 
>> --
>> Jeremie Chism
>> Triton Communications
> 
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
> 
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages