[outages] Lightower/Sidera - NYC
Tim Durack
tdurack at gmail.com
Thu Nov 7 16:54:00 EST 2013
As another datapoint, the traditional HV/Datanet/Lightower side of the
network has been stable during this reported outage.
On Thu, Nov 7, 2013 at 4:45 PM, Charles Sprickman <spork at bway.net> wrote:
> On Nov 7, 2013, at 4:29 PM, Vincent Aniello wrote:
>
> Our circuits stabilized mid-afternoon, but shortly after 4pm EST we
> started seeing issues again.
>
>
> Confirmed here as well, 7 out of 7 L2 tail circuits not passing traffic
> after a good hour of instability.
>
> FWIW, I don't recall any recent maintenance announcements from them, so
> this is a bit troubling (I'd expect this if we saw a big rollout of new
> code the night previous) since it came out of the blue. I can also confirm
> we saw our first outage much earlier than they acknowledged, one circuit
> lost connectivity at 5:23 a.m. I can also confirm that we did not see the
> physical layer go down once on any of our circuits, this seems to be
> occurring in the core, not the rings that actually service the buildings.
>
> We're legacy ConEd/RCN/Sidera, have no idea if this is happening with the
> LT network or not.
>
> We got what appears to be a bogus "all restored" email at 2:50 p.m.
>
> Charles
>
> --Vincent
>
>
>
> From: Karl Clapp <kclapp at staff.gwi.net>
> To: "outages at outages.org" <outages at outages.org>
> Date: 11/07/2013 03:23 PM
> Subject: Re: [outages] Lightower/Sidera - NYC
> Sent by: "Outages" <outages-bounces at outages.org>
> ------------------------------
>
>
>
> I contacted their NOC about 45min ago. They have stated they are still
> working on resolution. They have reports about some services being
> functional, while other still experiencing packet-loss and latency. It does
> not sound like they have been able to identify the root cause and are still
> troubleshooting.
>
> Here is the number I used to get through to their NOC: 888-583-4237,
> option 1. From there, just follow the prompts.
>
> Fwiw, my customer circuit from NYC to NJ is still affected.
>
> On Thu, Nov 7, 2013 at 10:25 AM, John Kinsella <*jlk at thrashyour.com*<jlk at thrashyour.com>>
> wrote:
> Just as a data point, LT for me is routing into Equinix NY5 over HE, so at
> least there's connectivity...
>
> My traceroute [v0.82]
> icmp (0.0.0.0) Thu Nov 7
> 07:24:09 2013
> Keys: Help Display mode Restart statistics Order of fields quit
> Packets Pings
> Host Loss% Snt Last Avg Best
> Wrst StDev
> 1. 10.0.1.1 0.0% 29 6.7 3.2 0.8
> 12.2 2.7
> 2. 10.0.1.49 0.0% 28 8.0 4.3 1.5
> 13.1 3.0
> 3. 10.0.24.73 0.0% 28 6.7 4.5 1.9
> 14.7 3.4
> 4. 25.66.208.*web-pass.com* <http://web-pass.com/>
> 0.0% 28 1.9 11.2 1.9 48.0 12.3
> 5. 2-1.760.*core.fre.web-pass.com* <http://core.fre.web-pass.com/>
> 0.0% 28 9.2 6.4 3.1 19.7 4.0
> 6. gige-g6-2.*core1.fmt1.he.net* <http://core1.fmt1.he.net/>
> 0.0% 28 9.6 12.4 3.5 26.4 6.2
> 7. 10gigabitethernet1-2.*core1.sjc2.he.net* <http://core1.sjc2.he.net/> 0.0% 28 11.9 9.5 3.5 18.7 4.3
> 8. 10gigabitethernet9-1.*core1.nyc4.he.net* <http://core1.nyc4.he.net/> 0.0% 28 87.9 79.5 73.3 92.7 5.5
> 9. 10gigabitethernet4-1.*core1.nyc5.he.net* <http://core1.nyc5.he.net/> 0.0% 28 76.6 80.5 73.6 106.7 7.0
> 10. lightower-fiber-networks.10gigabitether 0.0% 28 78.0 82.3 74.4
> 108.0 9.4
> 11. *xe-8-3-0.nycmnyzrj91.lightower.net*<http://xe-8-3-0.nycmnyzrj91.lightower.net/> 0.0% 28 89.1 83.6 74.2 144.7 15.8
> 12. *ae2-jrcynj67j41.lightower.net*<http://ae2-jrcynj67j41.lightower.net/> 0.0% 28 85.4 88.6 80.1 131.5 12.1
> 13. 192.154.116.14.*lightower.net* <http://lightower.net/>
> 0.0% 28 85.6 83.9 80.2 95.6 3.5
> 14. ???
>
> John
>
> On Nov 7, 2013, at 7:19 AM, Karl Clapp <*kclapp at staff.gwi.net*<kclapp at staff.gwi.net>>
> wrote:
>
> LightTower/Sidera master ticker for this is LT1323725.
>
> I was able to get through their support line now.
>
> The ETR is 1-2 hours from now.
>
> ---
> Karl Clapp
> Network Engineering Manager
> Biddeford Internet Corp. d.b.a. GWI (AS5760)
> Office: *207.602.1144* <207.602.1144>
> Mobile: *207.641.3907* <207.641.3907>
> Fax: *207.286.2061* <207.286.2061>
>
>
> On Thu, Nov 7, 2013 at 9:39 AM, <*SRonan at eexchange.com*<SRonan at eexchange.com>>
> wrote:
> Likely related to the fiber cut they experienced on their network earlier
> this week which they routed around.
>
> Shane Ronan, Vice President – Technology Architect
> State Street Global Exchange | 1230 Avenue of the Americas, 18th Fl | New
> York, NY 10020
> P *(212) 259-3023* <%28212%29%20259-3023> | M *(347) 413-4503*<%28347%29%20413-4503>
> *sronan at eexchange.com* <sronan at eexchange.com>
>
> *From: *Jayson Navitsky <*jnavitsky at gmail.com* <jnavitsky at gmail.com>>
> * Date: *Thursday, November 7, 2013 at 6:55 AM
> * To: *"*outages at outages.org* <outages at outages.org>" <
> *outages at outages.org* <outages at outages.org>>
> * Subject: *[outages] Lightower/Sidera - NYC
>
> Anyone else seeing any issues in NYC with Lightower/Sidera? Starting
> around 6:05AM (EST) we saw heavy packet loss across their network (from
> Sidera legacy) and some of our point to points bounce. The P to P's have
> some what stabilized (though we've seen a couple more bounces since then)
> the IP side of things have been pretty much unusable (we shut our session
> down.) It also looks like their phone system is having a problem - calls
> into them seem to drop after trying to connect to the NOC - which I'm sure
> is pretty busy.
>
> -Jayson
>
> ------------------------------
>
> *http://www.statestreet.com/emaildisclaimer/*<http://www.statestreet.com/emaildisclaimer/>
>
> _______________________________________________
> Outages mailing list
> *Outages at outages.org* <Outages at outages.org>
> *https://puck.nether.net/mailman/listinfo/outages*<https://puck.nether.net/mailman/listinfo/outages>
>
>
> _______________________________________________
> Outages mailing list
> *Outages at outages.org* <Outages at outages.org>
> *https://puck.nether.net/mailman/listinfo/outages*<https://puck.nether.net/mailman/listinfo/outages>
>
>
> _______________________________________________
> Outages mailing list
> *Outages at outages.org* <Outages at outages.org>
> *https://puck.nether.net/mailman/listinfo/outages*<https://puck.nether.net/mailman/listinfo/outages>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
--
Tim:>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20131107/b8f9d808/attachment.htm>
More information about the Outages
mailing list