[Outages-discussion] [External Sender] Re: [outages] Fios in NYC area?

[Eric Bonjour]

Just started monitoring a connection this morning but I see a significant latency change (+2-5ms) at 11:10 CDT and then another change (-2-5ms) at 11:22 CDT.
Not seeing any packet loss since 9:40 CDT.

From: Outages-discussion <outages-discussion-bounces at outages.org> On Behalf Of Izzy Goldstein - TeleGo
Sent: Thursday, June 18, 2020 10:09 AM
To: David Rubin <drubin at cubexgroup.com>
Cc: Cary Wiedemann <carywiedemann at gmail.com>; outages-discussion at outages.org
Subject: [External Sender] Re: [Outages-discussion] [outages] Fios in NYC area?

and i can confirm this is also ongoing for my clients

Is anyone from verizon/alter online ?

On Thu, Jun 18, 2020 at 10:57 AM David Rubin <drubin at cubexgroup.com<mailto:drubin at cubexgroup.com>> wrote:
Yup - I have multiple clients complaining - same exact thing - VPN drops - remote dies - have to reconnect - repeatedly.  Driving them crazy - me too :)

On Thu, Jun 18, 2020 at 10:49 AM Matthew Huff <mhuff at ox.com<mailto:mhuff at ox.com>> wrote:
We have opened a case with CenturyLink/Level3 to see if they can put pressure on Verizon. It’s wrecking havoc with our Voip and VPN/Remote desktops.

From: Outages-discussion <outages-discussion-bounces at outages.org<mailto:outages-discussion-bounces at outages.org>> On Behalf Of David Rubin
Sent: Thursday, June 18, 2020 10:24 AM
To: Cary Wiedemann <carywiedemann at gmail.com<mailto:carywiedemann at gmail.com>>
Cc: outages-discussion at outages.org<mailto:outages-discussion at outages.org>
Subject: Re: [Outages-discussion] [outages] Fios in NYC area?

I'm still having users reporting issues today - do we know if this is ongoing / if Verizon is aware of what's going on?  I'm guessing they aren't - because why would they be?

On Wed, Jun 17, 2020 at 6:40 PM Cary Wiedemann via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:
I've definitely seen loss from FiOS circuits in Manhattan and Long Island to many routes (examples: AT&T and Cox, but not Google's local 8.8.8.8 resolver) starting around 8:45pm EDT last night.

Some graphs:
24-hour Long Island FiOS to Cox:
[image.png]
2-hour Long Island FiOS to Cox:
[image.png]
2-hour Manhattan to FiOS to AT&T:
[image.png]

And of course New York is one of the IPv6 FiOS regions that broke traditional traceroute, so it's tougher than normal to nail down which router is the culprit.  I have some UDP MTRs running now which should hopefully answer that question.

I've seen several days of high latency and intermittent loss to many Verizon peers via their old ALTER.NET<http://ALTER.NET> routers in the DC region too, which may or may not be related.

No, it's not just you.  But it's not the last-mile FiOS either.  Looks like Verizon backbone routing woes.

- Cary



On Wed, Jun 17, 2020 at 5:13 PM Izzy Goldstein - TeleGo via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:
getting complaints from customers using FiOS of packet loss to our data center
but getting word there is an issue in NYC area with other data centers too?

--

Izzy Goldstein

Chief Technology Officer

Main: (212) 477-1000 x 2085<tel:(212)%20477-1000>

Direct: (929) 477-2085<tel:(929)%20477-2085>

Website: www.telego.com<http://www.telego.net/>



 <http://www.telego.com/>

Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.<http://www.telego.com/>


TeleGo Hosted PBX<http://www.telego.com/>
_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages<http://www.telego.com/>
_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages<http://www.telego.com/>

<http://www.telego.com/>
 <http://www.telego.com/>
-- <http://www.telego.com/>
David Rubin <http://www.telego.com/>
Cubex Group
223 Wall Street - PMB  #267
Huntington, NY 11743
646.221.3283<http://www.telego.com/>
 <http://www.telego.com/>
If you have a support-related issue, please email to support at cubexgroup.com.<http://www.telego.com/>


--
David Rubin
Cubex Group
223 Wall Street - PMB  #267
Huntington, NY 11743
646.221.3283

If you have a support-related issue, please email to support at cubexgroup.com<mailto:support at cubexgroup.com>.
_______________________________________________
Outages-discussion mailing list
Outages-discussion at outages.org<mailto:Outages-discussion at outages.org>
https://puck.nether.net/mailman/listinfo/outages-discussion


--

Izzy Goldstein

Chief Technology Officer

Main: (212) 477-1000 x 2085<tel:(212)%20477-1000>

Direct: (929) 477-2085<tel:(929)%20477-2085>

Website: www.telego.com<http://www.telego.net/>



[https://lh4.googleusercontent.com/Ze7NkT2QtotXfVWbpWXrpSMRIcAdqb0Ci0_RZsAjQzmRrd-_v02-O86QlqwZnGUE8SlBECXBvJUPXSH7ZRyDvDVeaXQFSS5SiA4gan112DN-0s2D3prRdt2UeUprZQEo1YmUFfZ7]<http://www.telego.com/>



Confidentiality Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.

TeleGo Hosted PBX<https://youtu.be/DaT8YAZ4V0w>

[EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20200618/3860a918/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 33858 bytes
Desc: image001.png
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20200618/3860a918/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 19110 bytes
Desc: image002.png
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20200618/3860a918/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 21110 bytes
Desc: image003.png
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20200618/3860a918/attachment-0005.png>