[outages] FiOS issue in NYC Area?

[Dovid Bender]

The issue seems to be with traffic going to Verizon. From all the traces we
have seen (we looked from Level3, Cogent and others) the common denominator
is br[2]3.nyc4. The traffic does not go through that device(s) then there
isn't an issue. Also the issue seems to be with certain ports. So if you
have one phone using say port 5062 and another using 5063 the one using
5063 may have the issue and the one using 5062 may not. You will most
likely see the same devices having the issue over and over again, pulling
it off the network for the NAT hole to close up may help since the next
time it goes out it may use a different port. Over all it seems like they
are doing some sort of hashing which is causing the traffic to go over a
bad/overloaded card that is blackholing/dropping the packets.

I spoke with a few people today from different organizations and they all
said that they would be reaching out to Verizon. If anyone has the ear of
an engineer at VZ please let me know. I have a few PCAP's clearly showing
the issue along with many traces.



On Mon, May 13, 2019 at 6:21 PM Craig Haddow <craig at corevox.com> wrote:

> Is there any followup to this issue?  We have a couple of fios customers
> that were experiencing one way audio where they could not hear the caller.
> We have the recordings that indicate the caller audio is reaching our
> hosted platform and the audio is good in both directions.
>
> —
> Craig Haddow |  CoreVox
> Business Development Mgr
> +1 215-390-1100 ext 102
>
>
>     https://www.corevox.com
>
> On May 13, 2019, at 2:21 PM, Dovid Bender via Outages <outages at outages.org>
> wrote:
>
> Izzy,
>
> Please see my email on the nanog list. A few others reached out to me
> directly. I will call you in 5.
>
>
>
> On Mon, May 13, 2019 at 2:19 PM Izzy Goldstein - TeleGo via Outages <
> outages at outages.org> wrote:
>
>> Many of our clients who are using fios are reporting issues with their
>> voip phones
>>
>> when i switch them to their backup ISP (Optimum etc) it works fine
>>
>> anyone else experiencing any fios issues ?
>>
>> --
>> Izzy Goldstein
>> Chief Technology Officer
>> Main: (212) 477-1000 x 2085 <(212)%20477-1000>
>> Direct: (929) 477-2085
>> Website: www.telego.com <http://www.telego.net/>
>>
>> <http://www.telego.com/>
>>
>> Confidentiality Notice: This e-mail may contain confidential and/or
>> privileged information. If you are not the intended recipient or have
>> received this e-mail in error please notify us immediately by email reply
>> and destroy this e-mail. Any unauthorized copying, disclosure or
>> distribution of the material in this e-mail is strictly forbidden. Any
>> views or opinions presented in this email are solely those of the author
>> and do not necessarily represent those of TeleGo (T). Employees of TeleGo
>> are expressly required not to make defamatory statements and not to
>> infringe or authorize any infringement of copyright or any other legal
>> right by email communications. Any such communication is contrary to TeleGo
>> policy and outside the scope of the employment of the individual concerned.
>> TeleGo will not accept any liability in respect of such communication, and
>> the employee responsible will be personally liable for any damages or other
>> liability arising.
>>
>> TeleGo Hosted PBX <https://youtu.be/DaT8YAZ4V0w>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20190513/7dbd4d73/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-3.tiff
Type: image/tiff
Size: 13582 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/outages/attachments/20190513/7dbd4d73/attachment.tiff>