[VoiceOps] Vitelity / Voyant Colorado Outage

[Peter Beckman]

START: 02-09-2020 20:31:39 UTC (~3:31pm EST)
END:   02-09-2020 23:40:47 UTC (~6:41pm EST)

Services recovered approximately 3 hours and 10 minutes after we initially
detected the network issues. Round Trip Times dropped from 50% packet loss
and 375ms RTT back down to 68ms RTT with 0% packet loss.

We're waiting on an RFO from Vitelity/Voyant. Would that be an appropriate post
here?

On Sun, 9 Feb 2020, Peter Beckman wrote:

> START: 02-09-2020 20:31:39 UTC
> END: ongoing
>
> Vitelity is experiencing some sort of network issue. While the Voyant NOC
> says they are aware, their portal is unstable due to the 40-50% packet loss
> and jump from 40-50ms to 375ms round trip times.
>
> This is affecting all services AFAICT: API endpoints, Portal, incoming and
> outgoing calls.
>
> We have moved most of our traffic away from Vitelity, except for phone
> numbers they host, as there is no ETA and 375ms plus 40-50% packet loss
> means pretty terrible call quality.
>
> I'll update as I learn more.

---------------------------------------------------------------------------
Peter Beckman                                                  Internet Guy
beckman at angryox.com                                 http://www.angryox.com/
---------------------------------------------------------------------------