[outages] packet loss to us-east-1
Gunnar Guðvarðarson
gunnar.gudvardarson at advania.is
Mon Nov 23 15:41:05 EST 2015
Confirmed from Iceland
~ Gunnar
On Nov 23, 2015 19:55, Grant Ridder via Outages <outages at outages.org> wrote:
It is related for some since Slack is in us-east-1
On Mon, Nov 23, 2015 at 11:52 AM, Chris Swingler <chris at chrisswingler.com<mailto:chris at chrisswingler.com>> wrote:
I'm not sure these are related (I'm seeing zero packet loss to Slack from Chicago on Cogent) but Slack's definitely broken. Seemed to have been a cascading failure, a colleague of mine complained over IRC he got booted from his, then I got booted from one Slack team, then another.
On Nov 23, 2015, at 1:49 PM, Justin Oeder via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:
Slack, which is hosted with Amazon is currently down.
________________________________
From: "Grant Ridder via Outages" <outages at outages.org<mailto:outages at outages.org>>
To: "Mike Lyon" <mike.lyon at gmail.com<mailto:mike.lyon at gmail.com>>
Cc: outages at outages.org<mailto:outages at outages.org>, "Tomoc Field" <tomoc.field at sonic.com<mailto:tomoc.field at sonic.com>>
Sent: Monday, November 23, 2015 2:43:57 PM
Subject: Re: [outages] packet loss to us-east-1
Seeing this issue again
4. AS46375 50.ae4.gw.pao1.sonic.net<http://50.ae4.gw.pao1.sonic.net/> 0.0% 10 15.5 23.2 2.9 89.3 28.6
5. AS8121 199.127.107.189 0.0% 10 64.2 26.6 3.7 85.7 28.2
6. AS8121 xe0-0-0-7.core2.sv8.layer42.net<http://xe0-0-0-7.core2.sv8.layer42.net/> 0.0% 10 66.6 24.8 3.5 102.1 33.4
7. AS16509 72.21.221.176 0.0% 10 64.2 26.9 3.0 99.1 32.4
8. AS??? 54.240.243.50 0.0% 10 147.4 115.9 88.2 176.9 27.9
9. AS??? 54.240.243.61 90.0% 10 78.4 78.4 78.4 78.4 0.0
10. AS??? 54.239.42.38 10.0% 10 130.4 106.9 71.8 167.6 31.8
11. AS16509 54.239.108.120 70.0% 10 178.9 125.2 90.0 178.9 47.2
12. AS16509 54.239.111.235 20.0% 10 143.1 99.0 70.8 158.1 34.7
13. AS16509 54.239.111.230 70.0% 10 89.5 93.9 88.7 103.5 8.3
-Grant
On Fri, Nov 20, 2015 at 5:27 PM, Mike Lyon <mike.lyon at gmail.com<mailto:mike.lyon at gmail.com>> wrote:
Hmmm,
It shows as if it is getting there but look at that jump from 15ms to 85ms on hops 7 and 8:
traceroute amazon.com<http://amazon.com/>
traceroute: Warning: amazon.com<http://amazon.com/> has multiple addresses; using 176.32.103.205
traceroute to amazon.com<http://amazon.com/> (176.32.103.205), 64 hops max, 52 byte packets
1 192.168.20.1 (192.168.20.1) 0.708 ms 0.395 ms 0.328 ms
2 162.250.208.141 (162.250.208.141) 0.671 ms 0.622 ms 0.588 ms
3 10.25.0.1 (10.25.0.1) 1.985 ms 4.165 ms 21.626 ms
4 10.20.0.1 (10.20.0.1) 7.852 ms 7.626 ms 5.214 ms
5 10.0.0.1 (10.0.0.1) 8.041 ms 17.839 ms 34.089 ms
6 69.36.226.193 (69.36.226.193) 5.477 ms 14.228 ms 20.203 ms
7 xe0-2-0-4.core4.scl.layer42.net<http://xe0-2-0-4.core4.scl.layer42.net/> (65.50.198.237) 15.153 ms 13.475 ms 8.706 ms
8 205.251.229.145 (205.251.229.145) 85.231 ms
54.240.242.108 (54.240.242.108) 96.157 ms
205.251.229.145 (205.251.229.145) 175.633 ms
9 54.240.242.141 (54.240.242.141) 100.007 ms 234.602 ms
54.240.242.121 (54.240.242.121) 94.263 ms
10 54.239.109.102 (54.239.109.102) 110.966 ms
54.239.109.136 (54.239.109.136) 123.322 ms
54.239.110.50 (54.239.110.50) 105.029 ms
11 54.239.109.143 (54.239.109.143) 84.239 ms
54.239.110.23 (54.239.110.23) 91.540 ms
54.239.110.57 (54.239.110.57) 86.620 ms
12 54.239.109.143 (54.239.109.143) 91.382 ms
205.251.245.168 (205.251.245.168) 87.930 ms
54.239.109.143 (54.239.109.143) 76.879 ms
13 * 205.251.245.168 (205.251.245.168) 90.780 ms 96.386 ms
On Fri, Nov 20, 2015 at 5:24 PM, Grant Ridder <shortdudey123 at gmail.com<mailto:shortdudey123 at gmail.com>> wrote:
Looks like they solved it by routing around layer42 and go via cogent. That in itself is a bad idea :/
4. AS46375 50.ae4.gw.pao1.sonic.net<http://50.ae4.gw.pao1.sonic.net/> 1.2% 81 3.5 12.0 2.6 91.0 16.2
5. AS174 te0-0-0-15.ccr21.sjc04.atlas.cogentco.com<http://te0-0-0-15.ccr21.sjc04.atlas.cogentco.com/> 6.2% 81 6.6 10.5 3.2 79.9 11.7
6. AS174 be2016.ccr22.sfo01.atlas.cogentco.com<http://be2016.ccr22.sfo01.atlas.cogentco.com/> 3.7% 81 15.3 14.6 4.2 98.9 16.3
7. AS174 be2133.ccr22.mci01.atlas.cogentco.com<http://be2133.ccr22.mci01.atlas.cogentco.com/> 4.9% 81 47.7 50.4 41.8 113.6 13.3
8. AS174 be2157.ccr42.ord01.atlas.cogentco.com<http://be2157.ccr42.ord01.atlas.cogentco.com/> 70.0% 81 54.1 67.2 53.9 140.2 21.3
9. AS174 be2217.ccr41.ord03.atlas.cogentco.com<http://be2217.ccr41.ord03.atlas.cogentco.com/> 33.3% 81 57.3 63.6 54.3 158.4 17.0
10. AS174 ntt.ord03.atlas.cogentco.com<http://ntt.ord03.atlas.cogentco.com/> 66.2% 81 56.9 72.0 56.5 126.8 20.6
11. AS2914 ae-3.amazon.chcgil09.us.bb.gin.ntt.net<http://ae-3.amazon.chcgil09.us.bb.gin.ntt.net/> 34.6% 81 61.5 61.8 55.9 89.7 8.1
-Grant
On Fri, Nov 20, 2015 at 5:14 PM, Mike Lyon <mike.lyon at gmail.com<mailto:mike.lyon at gmail.com>> wrote:
Oh thank god! I was seeing it too but wasn't sure if was Layer42 or Amazon that was having the issue...
Thanks,
Mike
On Fri, Nov 20, 2015 at 5:09 PM, Grant Ridder via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:
Saw that, thanks!
-Grant
On Fri, Nov 20, 2015 at 4:49 PM, Tomoc Field <tomoc.field at sonic.com<mailto:tomoc.field at sonic.com>> wrote:
Yes. 70%+ packet loss from 7065 to all Amazon services as far as I can tell. This was experienced via both layer42 and XO, but Amazon appears to have resolved their issues at this time.
-Tomoc
On Fri, Nov 20, 2015 at 04:34:42PM -0800, Grant Ridder via Outages wrote:
> Anyone else seeing packet log into us-east-1?
>
> 54.240.243.121 (AS16509) is Amazon
>
> MTR report excerpt
> 4. AS46375 50.ae4.gw.pao1.sonic.net<http://50.ae4.gw.pao1.sonic.net/> 0.0% 10 2.5
> 8.3 2.5 37.5 10.7
> 5. AS8121 199.127.107.189 0.0% 10 3.8
> 7.9 3.4 34.9 9.5
> 6. AS8121 xe0-0-0-7.core2.sv8.layer42.net<http://xe0-0-0-7.core2.sv8.layer42.net/> 0.0% 10 3.9
> 6.0 3.4 17.6 4.6
> 7. AS16509 72.21.221.176 0.0% 10 4.6
> 5.2 3.0 15.5 3.6
> 8. AS??? 54.240.243.110 0.0% 10 98.6
> 96.4 86.9 104.4 6.2
> 9. AS??? 54.240.243.121 90.0% 10 85.7
> 85.7 85.7 85.7 0.0
> 10. AS16509 205.251.229.104 20.0% 10 87.5
> 85.2 82.5 87.8 2.0
> 11. AS??? 54.239.42.29 70.0% 10 85.0
> 84.3 82.8 85.2 1.0
> 12. AS16509 54.239.109.238 30.0% 10 87.4
> 96.8 87.4 105.2 6.5
> AS16509 54.239.110.16
>
>
> -Grant
> _______________________________________________
> Outages mailing list
> Outages at outages.org<mailto:Outages at outages.org>
> https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
--
Mike Lyon
408-621-4826
mike.lyon at gmail.com<mailto:mike.lyon at gmail.com>
http://www.linkedin.com/in/mlyon
--
Mike Lyon
408-621-4826
mike.lyon at gmail.com<mailto:mike.lyon at gmail.com>
http://www.linkedin.com/in/mlyon
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20151123/d4fd05b0/attachment.htm>
More information about the Outages
mailing list