[VoiceOps] Bandwidth East Coast Issues
Peter Beckman
beckman at angryox.com
Sat Sep 25 16:48:53 EDT 2021
Spoke too soon, Inbound SBCs are now timing out again as of 20:45:38 UTC.
Monitoring these two IPs
67.231.9.112
67.231.13.113
On Sat, 25 Sep 2021, Peter Beckman wrote:
> Another Update
>
> Update - Bandwidth teams continue to investigate Voice and Messaging
> Failures. 911 calls are experiencing failures.
>
> Customers may also experience portal and portal API time-outs,
> slowness, or
> unresponsiveness.
>
> All teams are working towards a resolution at this time.
>
> Sep 25, 16:34 EDT
>
> Our monitoring cleared our Inbound SBCs as of 20:34:24 UTC (4:34:24pm EDT)
> as they are now responding at an acceptable rate. I just tested outbound
> and got a call through to a Verizon Mobile, though the PDD was about 5-6
> seconds.
>
> Inbound seems to be working, albeit a bit slower than expected.
>
> Seems like we're on the path to full recovery.
>
> Beckman
>
> On Sat, 25 Sep 2021, Peter Beckman wrote:
>
>> After a long silence, no updates other than "we're working on it."
>>
>> Update - Bandwidth teams continue to investigate Voice and Messaging
>> Failures.
>>
>> At this time, 911 calls are not impacted.
>>
>> Customers may also experience portal and portal API time-outs,
>> slowness, or unresponsiveness.
>>
>> All teams are working towards a resolution at this time.
>>
>> Sep 25, 16:22 EDT
>>
>> We are still seeing some calls get through. SMS traffic is being delivered
>> faster and we have not seen any missed SMS messages. Worst SMS delay was 9m
>> 37s. Last 5 tests in the last 50 minutes were all under 21 seconds, 3 of 5
>> were under 2 seconds.
>>
>> About half of our calls getting through are Canada DIDs.
>>
>> Beckman
>>
>> On Sat, 25 Sep 2021, Peter Beckman via VoiceOps wrote:
>>
>>> As of 3:31pm EDT Bandwidth has acknowledged the issue:
>>>
>>> Investigating - Bandwidth is investigating an incident impacting Voice
>>> and Messaging Services. Calls and Messages may experience unexpected
>>> failures. All teams are actively engaged.
>>>
>>> Sep 25, 15:31 EDT
>>>
>>> I will update here as I get updates.
>>>
>>> Outbound calls still failing with timeouts, Inbound calls failing, SMS
>>> messages are delayed, the delay now up to 8 minutes.
>>>
>>> On Sat, 25 Sep 2021, Peter Beckman via VoiceOps wrote:
>>>
>>>> We began detecting issues with Bandwidth Inbound and Outbound Voice and
>>>> SMS
>>>> starting at 19:00:38 UTC (3pm US Eastern Time). A TAC rep has
>>>> acknowledged
>>>> issues are detected but their status page as of yet has not been updated.
>>>>
>>>> I'm seeing outbound calls time out, inbound calls to BW DIDs receive a
>>>> "call cannot be completed" error on Verizon, and our SMS tests every 10
>>>> minutes that usually complete in under 2 seconds are taking 15 seconds to
>>>> 3.65 minutes to reach us.
>>>>
>>>> Hope they find the culprit and get it fixed quickly!
>>>>
>>>> As of 3:27:45pm EDT their status page shows zero incidents
>>>>
>>>> https://status.bandwidth.com/
>>> _______________________________________________
>>> VoiceOps mailing list
>>> VoiceOps at voiceops.org
>>> https://puck.nether.net/mailman/listinfo/voiceops
>>>
>>
>> ---------------------------------------------------------------------------
>> Peter Beckman Internet Guy
>> beckman at angryox.com https://www.angryox.com/
>> ---------------------------------------------------------------------------
>>
>
> ---------------------------------------------------------------------------
> Peter Beckman Internet Guy
> beckman at angryox.com https://www.angryox.com/
> ---------------------------------------------------------------------------
>
---------------------------------------------------------------------------
Peter Beckman Internet Guy
beckman at angryox.com https://www.angryox.com/
---------------------------------------------------------------------------
More information about the VoiceOps
mailing list