[outages] Level3 VoIP Issues?

Tue Oct 4 17:58:12 EDT 2016

There was a maintenance scheduled for a cut over that we had been notified of but I'm having trouble confirming if it's related. It seems like a maintenance or maintenance prep work gone bad. I got that email at 1:00am central.

GCR # 11424826<tel:11424826>

Summary

Level 3 will implement scheduled maintenance as part of a large scale global project to migrate traffic from our Voice platform to our IP platform. This work is taking place to migrate traffic off equipment that has reached end of life support and move it to a more stable environment. This GCR is one of several maintenances taking place across various locations. We will be performing this migration to increase network capacity, reliability, and stability. During this time you may experience up to three minutes of fast busies.

Updates

 04:11 GMT<x-apple-data-detectors://1> - This maintenance is now in progress.

 Harris Kleyman
 Data Center Engineer
 ----
 Atomic Data(r)  //  Safe. Simple. Smart.(tm)
 250 Marquette Ave, Suite 225
 Minneapolis, MN 55401
 Main  //  612.466.2000
 Direct  //  612.466.2031

 It's official. We're SOC 3 Certified, again!<http://www.atomicdata.com/articles/atomic-data%C2%AE-keeps-client%E2%80%99s-data-protected-soc-3%C2%AE-certified-security-and-controls>

On Oct 4, 2016, at 4:55 PM, Frank Bulk via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:

This is what we received from our partner,
"The Level 3 VOICE department has made a few configuration changes on core
network equipment, to provide a work around, and restore customer services.
Currently, Level 3 has a ticket open with one of their vendors and the work
around will stay in place until a root cause analysis is complete."

Frank

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Rob Hall via
Outages
Sent: Tuesday, October 04, 2016 3:35 PM
To: outages at outages.org<mailto:outages at outages.org>
Subject: Re: [outages] Level3 VoIP Issues?

Call flow appears to be normal as of 4:00PM Eastern across Charter, Comcast
and Cox.

Thanks!
--
Rob Hall, CCNP R&S, CCP-V
I.T. Manager
@Work Personnel Service
3815 Hwy 66 S, Suite 4
Rogersville, TN 37857
Office: (423)272-7897, x10113
Mobile: (423)312-6003

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Rob Hall via
Outages
Sent: Tuesday, October 4, 2016 2:17 PM
To: outages at outages.org<mailto:outages at outages.org>
Subject: Re: [outages] Level3 VoIP Issues?

Also confirmed with Comcast and Cox voice service in VA.

Thanks!
--
Rob Hall, CCNP R&S, CCP-V
I.T. Manager
@Work Personnel Service
3815 Hwy 66 S, Suite 4
Rogersville, TN 37857
Office: (423)272-7897, x10113
Mobile: (423)312-6003

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Rob Hall via
Outages
Sent: Tuesday, October 4, 2016 2:08 PM
To: outages at outages.org<mailto:outages at outages.org>
Subject: Re: [outages] Level3 VoIP Issues?

FWIW, we're still seeing an outage via Charter's voice service. Local
calling seems to be unaffected, but all inbound/outbound long distance is
out. We're seeing this behavior in every location in which we have Charter's
service, ranging across East TN and into Northern GA.

Thanks!
--
Rob Hall, CCNP R&S, CCP-V
I.T. Manager
@Work Personnel Service
3815 Hwy 66 S, Suite 4
Rogersville, TN 37857
Office: (423)272-7897, x10113
Mobile: (423)312-6003

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Mike Castoro
via Outages
Sent: Tuesday, October 04, 2016 1:36 PM
To: outages at outages.org<mailto:outages at outages.org>
Subject: Re: [outages] Level3 VoIP Issues?

Unofficially, I'm hearing that the Level 3 voice outage was caused by a
"fire at a large switching center caused by a major fiber cut"

-Mike

-----Original Message-----
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Michael
Cargile via Outages
Sent: Tuesday, October 04, 2016 12:09 PM
To: Peter Beckman <beckman at angryox.com<mailto:beckman at angryox.com>>
Cc: Aubrey Wells via Outages <outages at outages.org<mailto:outages at outages.org>>
Subject: Re: [outages] Level3 VoIP Issues?

Both Bandwidth and Vitelity are just VoIP resellers. They use larger
carriers like Level3 for their termination and origination. As such Level3's
outage is effecting their services as well.

On 2016-10-04 11:57, Peter Beckman via Outages wrote:
Hey Aubrey --

Did you see origination or termination issues with Bandwidth and
Vitelity?
Or both?

Beckman

On Tue, 4 Oct 2016, Aubrey Wells via Outages wrote:

Atlanta Ga, our trunks terminate in McLean VA and NYC. Same problem.

Seeing issues with other providers like Bandwidth.com<http://Bandwidth.com> and Vitelity as
well.

---------------------
Aubrey Wells
Manager, Network Engineering
Digium Cloud Services
Main: 888.305.3850
Support: 877.344.4861 or http://www.digium.com/en/support
<http://www.digium.com/en/support?elq=65516445a5964d3597e25eaf566bc2c
f&elqCampaignId=>

---------------------------------------------------------------------------
Peter Beckman                                                  Internet
Guy
beckman at angryox.com<mailto:beckman at angryox.com>
http://www.angryox.com/
----------------------------------------------------------------------
----- _______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

--
Michael Cargile
Director of Consulting
Vicidial Group
www.vicidial.com<http://www.vicidial.com>
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

--
BEGIN-ANTISPAM-VOTING-LINKS
------------------------------------------------------

Teach CanIt if this mail (ID 01RPs9KqN) is spam:
Spam:
https://antispam.roaringpenguin.com/canit/b.php?i=01RPs9KqN&m=fab7ef2658be&t
=20161004&c=s
Fraud/Phish:
https://antispam.roaringpenguin.com/canit/b.php?i=01RPs9KqN&m=fab7ef2658be&t
=20161004&c=p
Not spam:
https://antispam.roaringpenguin.com/canit/b.php?i=01RPs9KqN&m=fab7ef2658be&t
=20161004&c=n
Forget vote:
https://antispam.roaringpenguin.com/canit/b.php?i=01RPs9KqN&m=fab7ef2658be&t
=20161004&c=f
------------------------------------------------------
END-ANTISPAM-VOTING-LINKS

_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20161004/fc7ef2de/attachment.htm>