[outages] Fwd: above.net / he.net routing problems in LAX?

Thu Apr 26 19:31:18 EDT 2012

Hi Rob,

Thanks for the quick response.  Sorry for the confusion, the information 
I was provided at first was a little cloudy.  We also use GT-T/Packet 
Exchange and they're pointing this at above.net.

Here's the response we received from PKX/GT-T regarding the routing 
anomaly we're experiencing:

-------- Original Message --------
Subject: GTT TT#(49181), SVC ID: PEX/NATIVE LAYER 3/9014-1 RE: Routing 
issue preventing our network from reaching *
Date: Thu, 26 Apr 2012 21:52:27 +0000
From: GTT NOC <NOC at gt-t.net>

 From LAX core device the routes goes to abovenet for both the ip 
address and dies within the above net network.
We will investigate with abovenet and update you shortly.

cr1.lax1#traceroute 108.171.183.75

Type Ctrl-C to abort.

-------------------------------------------------------------------
Tracing the route to 108.171.183.75, 30 hops max, 40 byte packets
-------------------------------------------------------------------
  TTL Hostname            Probe1      Probe2      Probe3
   1  mpr1.lax2.us.above.net (206.223.123.71) 012.000 ms  001.000 ms 
001.000 ms
   2                        *           *           *
   3                        *           *           *
   4                        *           *           *
   5                        *

-- End forwarded message --

If they have not yet opened a ticket with you, this leads me to believe 
there's a problem elsewhere and we're being given the run-around.

The particular trace above is to one of our off-site boxes at rackspace.

This is an issue that seems to be impacting multiple networks and 
regions, so I'm not entirely convinced there's not something more going on.

For example, besides the IP referenced above, our 205.134.0.0 subnets 
are also failing.  I pulled this trace from Above.net's looking glass:

traceroute to 205.134.255.113 (205.134.255.113), 30 hops max, 40 byte 
packets
  1  xe-0-1-0.mpr1.lax12.us.above.net (64.125.31.189)  22.612 ms  0.421 
ms  0.368 ms
  2  10gigabitethernet1-3.core1.lax1.he.net (206.223.123.37)  0.944 ms 
12.178 ms  0.956 ms
  3  * * *
  4  * * *
  5  * * *
  6  * * *
  7  * * *
  8  * * *
  9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * corporate-colocation-inc.gigabitethernet3-16.core1.lax1.he.net 
(72.52.77.38)  1.135 ms !H *
28  * * *
29  * * *

Tracing out from 205.134.255.113 anywhere yields this:

root at wczverify1 [~]# traceroute above.net
traceroute to above.net (64.125.228.35), 30 hops max, 40 byte packets
  1  74.124.198.1 (74.124.198.1)  0.461 ms  0.465 ms  0.530 ms
  2  gige-g2-19.core1.lax2.he.net (64.62.194.161)  0.363 ms  0.433 ms 
0.509 ms
  3  * * *
  4  * * *
  5  * * *
  6  * * *

If you could shed any additional light on this issue for us I'd be most 
appreciative.

Thanks.

Best Regards,

Erik Soroka
Tier III System Administrator
InMotion Hosting, Inc.
eriks at inmotionhosting.com
888-321-4678 (ext 834)
757-416-6575 (int'l)

On 04/26/2012 07:20 PM, Rob Mosher wrote:
>   Hi Erik,
>
> Where did you get the information that we were blaming above.net for
> anything?  We are not aware of any issue between above.net and he.net,
> nor have we seen any tickets opened regarding this.  Can you please
> include the source and destination IP addresses of the traceroute quoted
> below, as well as a traceroute in the opposite direction if possible?
>
> As far as I can tell, everything looks clean here.
>
> root at tserv15:~# tcptraceroute lg.above.net
> traceroute to lg.above.net (64.124.253.67), 30 hops max, 60 byte packets
>   1  gige-g4-6.core1.lax1.he.net (66.220.18.41)  7.683 ms  7.725 ms
> 7.821 ms
>   2  mpr1.lax2.us.above.net (206.223.123.71)  0.781 ms  0.766 ms  0.767 ms
>   3  xe-3-0-0.cr1.lax112.us.above.net (64.125.30.17)  1.055 ms  1.057
> ms  1.293 ms
>   4  xe-2-1-0.cr1.sjc2.us.above.net (64.125.24.18)  10.125 ms  10.129
> ms  10.120 ms
>   5  xe-1-1-0.er1.sjc2.us.above.net (64.125.26.198)  9.815 ms  9.800 ms
> 9.800 ms
>   6  f1-1.itr3.sjc2.us.corp.above.net (64.124.248.122)  10.239 ms
> 10.002 ms  9.995 ms
>   7  lg.above.net (64.124.253.67)  10.227 ms  10.338 ms  10.267 ms
>
> --
> Rob Mosher
> Senior Network and Software Engineer
> Hurricane Electric / AS6939
>
>
>> ---------- Forwarded message ----------
>> From: "Erik S." <eriks at inmotionhosting.com
>> <mailto:eriks at inmotionhosting.com>>
>> Date: Apr 26, 2012 6:06 PM
>> Subject: [outages] above.net <http://above.net> / he.net
>> <http://he.net> routing problems in LAX?
>> To: "outages at outages.org <mailto:outages at outages.org>"
>> <outages at outages.org <mailto:outages at outages.org>>
>>
>> We're seeing major routing issues in Los Angeles with peering from
>> HE.net <-> Above.net, specifically after mpr1.lax2.us.above.net
>> <http://mpr1.lax2.us.above.net>.
>>
>> Above.net's looking glass in LA shows traces dying right after he.net
>> <http://he.net>,
>>
>>  1 xe-0-1-0.cr2.lax112.us.above.net
>> <http://xe-0-1-0.cr2.lax112.us.above.net> (64.125.30.250
>> <tel:%2864.125.30.250>)  0.780 ms  0.571 ms  0.479 ms
>>  2 xe-8-0-0.mpr1.lax12.us.above.net
>> <http://xe-8-0-0.mpr1.lax12.us.above.net> (64.125.30.30)  0.393 ms
>>  0.483 ms  0.404 ms
>>  3 10gigabitethernet1-3.core1.lax1.he.net
>> <http://10gigabitethernet1-3.core1.lax1.he.net> (206.223.123.37)
>>  0.884 ms 4.619 ms  1.100 ms
>>  4  * * *
>>
>> Both sides seem to be blaming the other.  Is anyone routing via
>> above.net <http://above.net> or he.net <http://he.net> seeing similar
>> issues out west?
>>
>> Best Regards,
>>
>> Erik Soroka
>> Tier III System Administrator
>> InMotion Hosting, Inc.
>> eriks at inmotionhosting.com <mailto:eriks at inmotionhosting.com>
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org <mailto:Outages at outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>

-- 