[VoiceOps] Bandwidth - Monday Outage

[Jay Hennigan]

On 9/27/21 13:30, Darren via VoiceOps wrote:
> I know it’s hard to be patient but I can’t imagine they’re NOT all hands 
> on deck.
> 
> The reality is probably that the DDoS attack is now so big, they can’t 
> handle it on their own, so they’re scrambling to contract out with 
> another provider who can handle it. That would explain why the BGP 
> routes they advertise have shifted. These DDoS products typically take 
> weeks to setup, so they’re likely having to scramble. I’ll be surprised 
> if this does NOT continue tomorrow (unfortunately).

 From my understanding this is not your typical volumetric DDoS but 
something specific to SIP or VoIP and thus the typical scrubbing 
services aren't going to be effective against the voice side of things.

Obviously they are keeping things close to the vest in order not to give 
too much information to the bad guys but I agree that it may take some 
time to resolve.

> *From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos 
> Alvarez <caalvarez at gmail.com>
> *Date: *Monday, September 27, 2021 at 1:23 PM

> Generic SIP client here, and the ongoing "continue to investigate" 
> notices are infuriatingly like "we have no damn clue what we're doing."  
> Try explaining to customers why it's not "our fault*" and that there's 
> no way to estimate a repair time.

I think the ongoing "continue to investigate" messages are fine. They're 
obviously dealing with a major incident and trying their best to keep 
their customers informed. This IMHO beats silence.

> *Our fault for choosing them I guess, but not something we can fix in 
> minutes.

The same thing could and has affected others. Voip.ms has been dealing 
with a similar attack for at least a week. We've had excellent service 
from Bandwidth for years and I trust that they will be able to get 
through this as well as anyone.

It's the nature of the legacy PSTN that redundant providers or fast 
failover for inbound calling isn't (yet) a thing.

-- 
Jay Hennigan - jay at west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV