[Outages-discussion] [outages] Second Bandwidth Outage Sunday

Mon Sep 27 15:09:28 EDT 2021

Typically 3 :

https://bgp.he.net/AS12130#_graph4

- NTT
- L3
- Telia

[image: Screen Shot 2021-09-27 at 9.09.19 AM.png]

Aryn Nakaoka
anakaoka at trinet-hi.com
Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817

AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY

A Better Solution https://www.trinet-hi.com/abettersolution.pdf
<https://www.trinet-hi.com/abettersolution.pdf>
Aloha Tone PBX  https://www.youtube.com/watch?v=96YWPY9wCeU

CONFIDENTIALITY NOTICE:  The information contained in this email and any
attachments may be privileged, confidential and protected from disclosure.
Any disclosure, distribution or copying of this email or any attachments by
persons or entities other than the intended recipient is prohibited. If you
have received this email in error, please notify the sender immediately by
replying to the message and deleting this email and any attachments from
your system. Thank you for your cooperation.

On Mon, Sep 27, 2021 at 9:04 AM Charles Sprickman <spork at bway.net> wrote:

>
>
> On Sep 27, 2021, at 2:31 PM, Jared Geiger <jared at compuwizz.net> wrote:
>
> It looks like Bandwidth dropped all their BGP peers except NTT, presumably
> to use NTT's DDoS services.
>
>
> How many do they typically have?
>
>
> As far as peering, Bandwidth offers connectivity via Megaport. Peerless
> (21808) is on a bunch of Equinix exchanges https://peeringdb.com/net/16256.
> Inteliquent (40160) is on some exchanges as well
> https://peeringdb.com/net/18236.
>
> I did try and get Peerless to join Coresite Any2East last year because
> they are in the same building but they said no.
>
> On Mon, Sep 27, 2021 at 11:19 AM Douglas Vought <
> douglas.vought at themsgctr.com> wrote:
>
>> They are. Several customers have contacted me demanding that I plug the
>> phone line back in.
>> On 9/27/2021 13:54, Drew Matthews via Outages wrote:
>>
>> i believe they are having issues again, not sure if its ddos related. but
>> they are experiencing some major issues. https://status.bandwidth.com/
>>
>> Update - Bandwidth teams are also investigating delays with Toll-Free
>> Messaging at this time along with V2 Voice, Standard Voice, 911 traffic,
>> Portal & API with intermittent service disruptions.
>> Sep 27, 13:01 EDT
>> Update - We continue to investigate this incident.
>> Sep 27, 11:57 EDT
>> Update - Bandwidth teams are also investigating delayed ports at this
>> time along with V2 Voice, Standard Voice, 911 traffic, Portal & API with
>> intermittent service disruptions.
>> Sep 27, 11:07 EDT
>> Update - We continue to investigate this incidnet.
>> Sep 27, 11:02 EDT
>> Update - Bandwidth teams are investigating an incident impacting Voice,
>> V2 Voice, 911 traffic, Portal & API with intermittent service disruption.
>> Sep 27, 10:00 EDT
>> Update - Bandwidth teams are investigating an incident impacting Voice,
>> V2 Voice, and 911 traffic with intermittent service disruption.
>> Sep 27, 09:42 EDT
>> Investigating - Bandwidth teams are investigating an incident impacting
>> our V2 Voice services. Customers may notice intermittent service
>> disruptions.
>> Sep 27, 09:12 EDT
>>
>> On Sun, Sep 26, 2021 at 3:13 PM Peter Beckman via Outages <
>> outages at outages.org> wrote:
>>
>>> Again today, on the heels of the 2h 47m DDoS-related outage yesterday,
>>> right at
>>> 11:01am US EDT (15:01:28 UTC), our monitoring alerted us to another
>>> incident at
>>> Bandwidth. The impact to us was much reduced this time, as we saw some
>>> flapping
>>> of their SIP Inbound SBCs, but around 15:20:12 UTC it was responsive
>>> enough to
>>> fall below our thresholds.
>>>
>>> The API started acting up at 14:56:00 UTC, but recovered, and then has
>>> been
>>> flapping between OK and degraded since then. As of 21:52 UTC, the API and
>>> Portal/Dashboard are either non-responsive, slow, or degraded.
>>>
>>> Inbound Voice Traffic has remaind at expected historical levels
>>> throughout.
>>> I have not checked on Termination, but it was working earlier today
>>> without
>>> issue.
>>>
>>> Latest from BW https://status.bandwidth.com/
>>>
>>>      Update - Bandwidth teams have observed expected call behavior for
>>> 911
>>>      calls. 911 calls are processing as designed and are fully restored.
>>> Teams
>>>      continue to work on standard voice traffic.
>>>
>>>      Sep 26, 17:20 EDT
>>>
>>> I question the phrase "processing as designed" which implies it is not
>>> working as the end user expects... odd.
>>>
>>> Beckman
>>>
>>> ---------------------------------------------------------------------------
>>> Peter Beckman                                                  Internet
>>> Guy
>>> beckman at angryox.com
>>> https://www.angryox.com/
>>>
>>> ---------------------------------------------------------------------------
>>> _______________________________________________
>>> Outages mailing list
>>> Outages at outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>>
>>
>> _______________________________________________
>> Outages mailing listOutages at outages.orghttps://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> Outages-discussion mailing list
>> Outages-discussion at outages.org
>> https://puck.nether.net/mailman/listinfo/outages-discussion
>>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20210927/fc47e620/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2021-09-27 at 9.09.19 AM.png
Type: image/png
Size: 26034 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20210927/fc47e620/attachment-0001.png>