[outages] Sprint sites
Andy Grosser
andy at meniscus.org
Tue Aug 16 10:07:16 EDT 2016
Just got off the phone with our Sprint reps. ETTR is approx 16:00 EDT for
power restoration (temporary generator from a local utility?). Sounds like
there were backup generator failures and battery backup has petered out.
Gonna be a long day.
Andy
On Tuesday, August 16, 2016, Rick Goodman via Outages <outages at outages.org>
wrote:
> I was able to get to Sprint.net but it's so slow I couldn't do anything
> there. I spoke to MNS and they have multiple DS3s down in DC due to a power
> issue and are working to restore but no ETR yet. My site that's down due to
> this is in Durham, NC.
>
>
>
>
>
>
>
>
> Rick GOODMAN
>
> Network Administrator
> Lesaffre Yeast Corporation
> Office: (414) 615-4091
> Mobile: (480) 364-6041
> rick.goodman at lsaf.com <javascript:;>
> www.lsaf.com
>
>
>
> -----Original Message-----
> From: Outages [mailto:outages-bounces at outages.org <javascript:;>] On
> Behalf Of outages-request at outages.org <javascript:;>
> Sent: Tuesday, August 16, 2016 6:39 AM
> To: outages at outages.org <javascript:;>
> Subject: Outages Digest, Vol 99, Issue 20
>
> Send Outages mailing list submissions to
> outages at outages.org <javascript:;>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://puck.nether.net/mailman/listinfo/outages
> or, via email, send a message with subject or body 'help' to
> outages-request at outages.org <javascript:;>
>
> You can reach the person managing the list at
> outages-owner at outages.org <javascript:;>
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of Outages digest..."
>
>
> Today's Topics:
>
> 1. Suwanee, GA Level 3 Outage (QTS Datacenter) (Bob Strecansky)
> 2. Re: Suwanee, GA Level 3 Outage (QTS Datacenter) (Jamie Stephens)
> 3. TWTC/Level3 Issues in RTP Area This Morning? (Devon True)
> 4. Sprint sites (frnkblk at iname.com <javascript:;>)
> 5. Re: Sprint sites (Lee Fuller)
> 6. Re: Sprint sites (Mark Tinka)
> 7. Re: Sprint sites (Mark Tinka)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 15 Aug 2016 12:55:11 -0400
> From: Bob Strecansky <bob at mailchimp.com <javascript:;>>
> To: outages at outages.org <javascript:;>
> Subject: [outages] Suwanee, GA Level 3 Outage (QTS Datacenter)
> Message-ID:
> <CAEGG-HXK0_kHxjwKHUhQTJMb_AiVYCH6JseUS_8aL9r2igz=
> 2g at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset=UTF-8
>
> Outages mailing list,
>
> Is anyone else impacted by a Level 3 Network outage in the QTS datacenter
> in Suwanee? We have redundant networking, but it seems the outage has been
> > 9 hours.
>
> Thanks,
>
> Bob
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 15 Aug 2016 18:12:07 +0000
> From: Jamie Stephens <jamie.stephens at gmail.com <javascript:;>>
> To: Bob Strecansky <bob at mailchimp.com <javascript:;>>, outages at outages.org
> <javascript:;>
> Subject: Re: [outages] Suwanee, GA Level 3 Outage (QTS Datacenter)
> Message-ID:
> <CABWMpWQukj0czibZhtatNR+v8BE1yGj0RT5vD=
> evz7Moq+mPsw at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Wr have a 100g link down between SC-VA being told a bad card , maybe
> related?
> On Mon, Aug 15, 2016 at 12:55 PM Bob Strecansky via Outages <
> outages at outages.org <javascript:;>> wrote:
>
> > Outages mailing list,
> >
> > Is anyone else impacted by a Level 3 Network outage in the QTS
> > datacenter in Suwanee? We have redundant networking, but it seems the
> > outage has been > 9 hours.
> >
> > Thanks,
> >
> > Bob
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org <javascript:;>
> > https://puck.nether.net/mailman/listinfo/outages
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://puck.nether.net/pipermail/outages/attachments/
> 20160815/14719702/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 15 Aug 2016 14:29:04 -0400
> From: Devon True <devon at noved.org <javascript:;>>
> To: "outages at outages.org <javascript:;>" <outages at outages.org
> <javascript:;>>
> Subject: [outages] TWTC/Level3 Issues in RTP Area This Morning?
> Message-ID: <57B209F0.6040807 at noved.org <javascript:;>>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Anyone see any issues in the RTP area this morning (15 August) around 1am
> Eastern? Our TWTC/Level3 bandwidth graphs dropped around 1:10am Eastern and
> several remote monitors reported reachability issues, but no actual circuit
> or BGP issues recorded on our equipment.
>
> --
> Devon
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 16 Aug 2016 08:09:43 -0500
> From: <frnkblk at iname.com <javascript:;>>
> To: <outages at outages.org <javascript:;>>
> Subject: [outages] Sprint sites
> Message-ID: <000601d1f7bf$75180e20$5f482a60$@iname.com>
> Content-Type: text/plain; charset="us-ascii"
>
> IPv4 and IPv6 access to http://www.sprint.net has been down since 5:41 am
> and IPv6 access to ipv6.sprint.com was down for a few minutes around 5:41
> am.
>
> I'm guessing some kind of change didn't go well?
>
> Frank
>
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 16 Aug 2016 14:16:38 +0100
> From: Lee Fuller <leefuller23 at gmail.com <javascript:;>>
> To: frnkblk at iname.com <javascript:;>
> Cc: outages at outages.org <javascript:;>
> Subject: Re: [outages] Sprint sites
> Message-ID:
> <
> CA+9D9DEf4mxmUpGfa3f+Y4Mk2O5+G7vrrNW887WQv-t6y3KQNQ at mail.gmail.com
> <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Sprint.net is loading albeit exceptionally slowly from London.
>
> Lee Fuller (mobile)
>
> PGP Fingerprint: 4ACAEBA4B9EE1B3A075034302D5C3D050E6ED55A
>
> On 16 Aug 2016 2:11 p.m., "frnkblk--- via Outages" <outages at outages.org
> <javascript:;>>
> wrote:
>
> > IPv4 and IPv6 access to http://www.sprint.net has been down since 5:41
> am
> > and IPv6 access to ipv6.sprint.com was down for a few minutes around
> 5:41
> > am.
> >
> > I'm guessing some kind of change didn't go well?
> >
> > Frank
> >
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org <javascript:;>
> > https://puck.nether.net/mailman/listinfo/outages
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://puck.nether.net/pipermail/outages/attachments/
> 20160816/f5eed69d/attachment-0001.html>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 16 Aug 2016 15:38:13 +0200
> From: Mark Tinka <mark.tinka at seacom.mu <javascript:;>>
> To: frnkblk at iname.com <javascript:;>, Karl Putland via Outages <
> outages at outages.org <javascript:;>>
> Subject: Re: [outages] Sprint sites
> Message-ID: <4fde605d-55dc-75a5-9093-1b31027865d4 at seacom.mu <javascript:;>
> >
> Content-Type: text/plain; charset="windows-1252"
>
>
>
> On 16/Aug/16 15:09, frnkblk--- via Outages wrote:
>
> > IPv4 and IPv6 access to http://www.sprint.net has been down since 5:41
> am
> > and IPv6 access to ipv6.sprint.com was down for a few minutes around
> 5:41
> > am.
> >
> > I'm guessing some kind of change didn't go well?
>
> It's good from South Africa, both on IPv4 and IPv6 (IPv6 latency is much
> better for me, though):
>
> Mark-Tinkas-MacBook:~ tinka$ traceroute -I www.sprint.net
> traceroute to www.sprint.net (208.24.22.50), 64 hops max, 72 byte packets
> 1 192.168.3.254 (192.168.3.254) 1.278 ms 0.822 ms 6.375 ms
> 2 ge-0-3-990.or-01-jnb.za.seacomnet.com (105.28.96.254) 1.165 ms
> 18.715 ms 0.798 ms
> 3 vl-22.es-07-jnb.za.seacomnet.com (105.27.115.1) 0.863 ms 1.539 ms
> 0.771 ms
> 4 xe-0-0-25.es-35-jnb.za.seacomnet.com (105.16.10.145) 1.556 ms
> 1.946 ms 1.702 ms
> 5 xe-0-2-0.er-06-jnb.za.seacomnet.com (105.16.9.237) 1.063 ms 1.653
> ms 1.188 ms
> 6 xe-0-0-0-4.cr-01-jnb.za.seacomnet.com (105.16.9.141) 158.850 ms
> 162.271 ms 159.782 ms
> 7 xe-0-0-0-8.cr-01-cpt.za.seacomnet.com (105.16.9.178) 159.047 ms
> 160.412 ms 159.847 ms
> 8 xe-0-1-0-3.cr-01-lhr.uk.seacomnet.com (105.16.11.78) 158.494 ms
> 159.646 ms 159.942 ms
> 9 xe-0-0-1-0.br-02-lhr.uk.seacomnet.com (105.16.35.253) 156.809 ms
> 156.791 ms 156.763 ms
> 10 213.130.48.217 (213.130.48.217) 159.068 ms 159.569 ms 159.391 ms
> 11 ae-15.r02.londen03.uk.bb.gin.ntt.net (129.250.3.30) 157.170 ms
> 157.255 ms 157.224 ms
> 12 213.206.131.105 (213.206.131.105) 168.365 ms 170.317 ms 167.510 ms
> 13 sl-crs1-nyc-0-5-2-0.sprintlink.net (144.232.9.163) 240.072 ms
> 239.430 ms 247.504 ms
> 14 144.232.5.216 (144.232.5.216) 245.227 ms 246.343 ms 243.813 ms
> 15 144.232.7.113 (144.232.7.113) 248.743 ms 251.928 ms 248.059 ms
> 16 144.232.6.108 (144.232.6.108) 261.845 ms 266.004 ms 260.843 ms
> 17 144.232.5.212 (144.232.5.212) 269.954 ms 265.539 ms *
> 18 144.232.5.209 (144.232.5.209) 268.886 ms 268.124 ms 267.949 ms
> 19 144.232.19.251 (144.232.19.251) 273.552 ms 274.332 ms 270.970 ms
> 20 sl-sprin881-320471-0.sprintlink.net (144.223.33.34) 275.041 ms
> 276.304 ms 278.641 ms
> 21 * www.sprint.net (208.24.22.50) 773.064 ms 721.662 ms
> 22 www.sprint.net (208.24.22.50) 758.549 ms * 735.113 ms
> Mark-Tinkas-MacBook:~ tinka$
>
> Mark-Tinkas-MacBook:~ tinka$ traceroute6 -I www.sprint.net
> traceroute6 to www.sprint.net (2600::) from
> 2c0f:feb0:1f::41b:2a6a:a147:855f, 64 hops max, 16 byte packets
> 1 2c0f:feb0:1f::ffff:ffff:ffff:fffe 0.288 ms 0.281 ms 0.246 ms
> 2 2c0f:feb0:1f:1::ff7f 0.557 ms 0.538 ms 0.447 ms
> 3 vl-22.es6-07-jnb.za.seacomnet.com 0.789 ms 1.010 ms 0.852 ms
> 4 xe-0-0-25.es6-35-jnb.za.seacomnet.com 1.452 ms 1.561 ms 1.545 ms
> 5 xe-0-2-0-0.er6-06-jnb.za.seacomnet.com 1.207 ms 1.001 ms 2.242 ms
> 6 xe-0-1-0-0.cr6-02-jnb.za.seacomnet.com 2.741 ms 3.688 ms 3.921 ms
> 7 xe-0-1-0-2.cr6-02-cpt.za.seacomnet.com 19.285 ms 19.752 ms 19.986
> ms
> 8 *
> xe-0-1-0-3.cr6-02-lhr.uk.seacomnet.com 157.985 ms 157.702 ms
> 9 xe-0-0-1-0.br6-02-lhr.uk.seacomnet.com 156.909 ms 156.971 ms
> 157.069 ms
> 10 xe-0-1-0-7.r00.londen10.uk.bb.gin.ntt.net 157.461 ms 157.516 ms
> 157.532 ms
> 11 ae-7.r25.londen12.uk.bb.gin.ntt.net 159.114 ms 160.592 ms 159.141
> ms
> 12 ae-2.r24.amstnl02.nl.bb.gin.ntt.net 165.887 ms 166.039 ms 167.139
> ms
> 13 ae-1.r02.amstnl02.nl.bb.gin.ntt.net 164.521 ms 164.463 ms 164.498
> ms
> 14 2001:728:0:4000::3e 164.799 ms 164.847 ms 164.684 ms
> 15 sl-bb21-ams-te0-0-0-0.v6.sprintlink.net 175.847 ms 174.262 ms
> 174.204 ms
> 16 sl-crs2-lon-te0-2-0-0.v6.sprintlink.net 173.277 ms 171.222 ms
> 172.053 ms
> 17 sl-crs1-nyc-po1-11-2-0.v6.sprintlink.net 247.319 ms 247.534 ms
> 248.460 ms
> 18 sl-crs1-akr-bu-3.v6.sprintlink.net 260.790 ms 260.364 ms 259.517 ms
> 19 sl-crs1-roa-bu-4.v6.sprintlink.net 261.222 ms 262.840 ms 263.882 ms
> 20 sl-crs1-nsh-bu-1.v6.sprintlink.net 273.406 ms 272.324 ms 280.691 ms
> 21 sl-crs1-atl-bu-2.v6.sprintlink.net 266.437 ms 274.022 ms 271.951 ms
> 22 sl-crs1-ffx-bu-5.v6.sprintlink.net 276.727 ms 275.727 ms 279.884 ms
> 23 sl-crs1-orl-bu-1.v6.sprintlink.net 280.213 ms 282.184 ms 285.036 ms
> 24 sl-lkdstr2-p1-0.v6.sprintlink.net 284.314 ms 285.523 ms 286.080 ms
> 25 www.sprint.net 278.852 ms 280.679 ms 274.640 ms
> Mark-Tinkas-MacBook:~ tinka$
>
> Mark.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://puck.nether.net/pipermail/outages/attachments/
> 20160816/2b863c4e/attachment-0001.html>
>
> ------------------------------
>
> Message: 7
> Date: Tue, 16 Aug 2016 15:39:03 +0200
> From: Mark Tinka <mark.tinka at seacom.mu <javascript:;>>
> To: Lee Fuller <leefuller23 at gmail.com <javascript:;>>, frnkblk at iname.com
> <javascript:;>
> Cc: outages at outages.org <javascript:;>
> Subject: Re: [outages] Sprint sites
> Message-ID: <6c866158-ff1b-73fc-e207-88f2983aee7f at seacom.mu <javascript:;>
> >
> Content-Type: text/plain; charset="windows-1252"
>
> Given the latency from my end (over an IPv6 transport - the IPv4 latency
> is pretty bad, but my laptop is not using it), seems to be inline with
> what I'd expect.
>
> Mark.
>
> On 16/Aug/16 15:16, Lee Fuller via Outages wrote:
> >
> > Sprint.net is loading albeit exceptionally slowly from London.
> >
> > Lee Fuller (mobile)
> >
> > PGP Fingerprint: 4ACAEBA4B9EE1B3A075034302D5C3D050E6ED55A
> >
> >
> > On 16 Aug 2016 2:11 p.m., "frnkblk--- via Outages"
> > <outages at outages.org <javascript:;> <mailto:outages at outages.org
> <javascript:;>>> wrote:
> >
> > IPv4 and IPv6 access to http://www.sprint.net has been down since
> > 5:41 am
> > and IPv6 access to ipv6.sprint.com <http://ipv6.sprint.com> was
> > down for a few minutes around 5:41
> > am.
> >
> > I'm guessing some kind of change didn't go well?
> >
> > Frank
> >
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org <javascript:;> <mailto:Outages at outages.org
> <javascript:;>>
> > https://puck.nether.net/mailman/listinfo/outages
> > <https://puck.nether.net/mailman/listinfo/outages>
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://puck.nether.net/pipermail/outages/attachments/
> 20160816/b1be0cd4/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org <javascript:;>
> https://puck.nether.net/mailman/listinfo/outages
>
>
> ------------------------------
>
> End of Outages Digest, Vol 99, Issue 20
> ***************************************
> _______________________________________________
> Outages mailing list
> Outages at outages.org <javascript:;>
> https://puck.nether.net/mailman/listinfo/outages
>
--
Andy Grosser
andy (at) meniscus [dot] org
---
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20160816/cc49574f/attachment.htm>
More information about the Outages
mailing list