[VoiceOps] Second Bandwidth Outage Sunday

[Peter Beckman]

Again today, on the heels of the almost 3 hour DDoS-related outage
yesterday, right at 11:01am US EDT (15:01:28 UTC), our monitoring alerted
us to another incident at Bandwidth. The impact to us was much reduced this
time, as we saw some flapping of their SIP Inbound SBCs, but around
15:20:12 UTC it was responsive enough to fall below our thresholds.

The API started acting up at 14:56:00 UTC, but recovered, and then has been
flapping between OK and degraded since then. As of 21:52 UTC, the API and
Portal/Dashboard are either non-responsive, slow, or degraded.

Inbound Voice Traffic has remaind at expected historical levels throughout.
I have not checked on Termination, but it was working earlier today without
issue.

Latest from BW https://status.bandwidth.com/

     Update - Bandwidth teams have observed expected call behavior for 911
     calls. 911 calls are processing as designed and are fully restored. Teams
     continue to work on standard voice traffic.

     Sep 26, 17:20 EDT

I question the phrase "processing as designed" which implies it is not
working as the end user expects... odd.

Beckman
---------------------------------------------------------------------------
Peter Beckman                                                  Internet Guy
beckman at angryox.com                                https://www.angryox.com/
---------------------------------------------------------------------------