[VoiceOps] Bandwidth - Monday Outage

[Pete Eisengrein]

They have publicly acknowledge it as a DDoS (
https://www.bandwidth.com/blog/a-message-to-our-customers-and-partners/ ) ,
but being pretty tight-lipped with specifics on what it is or how they are
mitigating.

On Wed, Sep 29, 2021 at 12:29 PM Carlos Alvarez <caalvarez at gmail.com> wrote:

> Is this some sort of ransom event against them maybe?  And what are the
> rest of you telling your customers?  We seem to have only a few
> specifically complaining, but those are complaining a lot.
>
>
> On Tue, Sep 28, 2021 at 11:06 PM Ivan Kovacevic <
> ivan.kovacevic at startelecom.ca> wrote:
>
>> Happening again.
>>
>> https://status.bandwidth.com/
>>
>>
>> [image: Star Telecom - Cloud Communications and Customer Experience
>> Solutions] <https://www.startelecom.ca/>
>>
>> *Ivan Kovacevic*
>>
>> *Co-Founder and VP Client Services*
>>
>>
>>
>>
>> On Mon, Sep 27, 2021 at 10:19 PM Peter Beckman via VoiceOps <
>> voiceops at voiceops.org> wrote:
>>
>>> On Mon, 27 Sep 2021, Ryan Delgrosso wrote:
>>>
>>> > Nothing meaningful other than the normal public party line.
>>> >
>>> > I too have heard unofficially that its DDOS, which makes sense given
>>> the
>>> > recurring nature.
>>> >
>>> > 4.5hrs down Sat
>>>
>>>   Our monitoring showed 2 hours 47 minutes of actual service affecting
>>>   outages across Voice (Inbound and Outbound), Messaging, and API/Portal.
>>>
>>>   The issue started at 3pm and recovered at 5:47pm EDT. We reported it to
>>>   the TAC at 3:07pm, they did not post on Status until 3:31pm.
>>>
>>> > Some small downtime Sun
>>> >
>>> > Now deep into Monday with problems.
>>> >
>>> > Its not a good look, but id like some more transparency.
>>>
>>>   DDoS attacks are real and hard to null route. You've got millions of IP
>>>   addresses slamming you with data. Your router has a capacity, and your
>>>   router cannot handle all of that extra crap data along with all of our
>>>   traffic too.
>>>
>>>   I'm sure BW will be investing in some beefy hardware that will be able
>>> to
>>>   better handle DDoS attacks, as well as working more closely with their
>>>   peering providers. I have to assume that they were getting gigabits of
>>>   traffic, overwhelming their links in addition to their edge routers.
>>>
>>>   Cloudflare details how they do it here:
>>>
>>> https://support.cloudflare.com/hc/en-us/articles/200172676-Understanding-Cloudflare-DDoS-protection
>>>
>>>   Not much to be transparent about. The Internet is an unfriendly place,
>>> and
>>>   bad actors can rain hell upon any public IP they want. Unsecured
>>> laptops,
>>>   desktops, TVs, IOT devices, etc, all contribute just a little tiny bit,
>>>   and all focus on one single point, kinda like those giant solar farms
>>> with
>>>   the mirrors and single tower in the middle to boil the molten salt.
>>>
>>>   Well, Bandwidth is the molten salt, and the mirrors are a bunch of
>>>   unsecured devices on the Internet.
>>>
>>>
>>> ---------------------------------------------------------------------------
>>> Peter Beckman                                                  Internet
>>> Guy
>>> beckman at angryox.com
>>> https://www.angryox.com/
>>>
>>> ---------------------------------------------------------------------------_______________________________________________
>>> VoiceOps mailing list
>>> VoiceOps at voiceops.org
>>> https://puck.nether.net/mailman/listinfo/voiceops
>>> _______________________________________________
>>> VoiceOps mailing list
>>> VoiceOps at voiceops.org
>>> https://puck.nether.net/mailman/listinfo/voiceops
>>>
>>
>> NOTE: This email message and any attachments are for the sole use of the
>> intended recipient(s) and may contain confidential and/or privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by replying to this email, and destroy all copies of the original
>> message.
>> _______________________________________________
>> VoiceOps mailing list
>> VoiceOps at voiceops.org
>> https://puck.nether.net/mailman/listinfo/voiceops
>>
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/voiceops/attachments/20210929/15cdc0c0/attachment.htm>