[outages] Denver Level3 problems?

[Sean Crandall]

Sorry to dredge up an old issue, but did anyone get anything better than "PEBKAC".

We were having some issues with voice that was going to Level3 last Tuesday and trying to figure out if what you all saw was related or not (I use Level3 for transit and voice termination), but since I didn't open a ticket at the time (we thought it was an internal issue), Level3 says they can't research the issue.

Any help?

-Sean


-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of Kevin Broderick
Sent: Wednesday, May 19, 2010 10:24 AM
To: outages at outages.org
Subject: Re: [outages] Denver Level3 problems?

L3 NOC called us back with reason for outage yesterday: PEBKAC.

- K

Bradley Bopp wrote:
> An internal source has informed me there was an issue with a Data Center in
> the Denver area in which something ³blew up² and this caused a chain effect
> with anything going through Denver to any other pop in the USA.
>
> The problem since has been resolved and was for a very short period.
>
> Brad
>
>
>
> On 5/18/10 2:32 PM, "Joseph Jackson" <jjackson at aninetworks.net> wrote:
>
>   
>> We also saw issues with customer end points that took level 3 around the same
>> time but in NYC ­ LA and DFW POPS.
>>  
>> Joseph
>>  
>>  
>>
>> From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On
>> Behalf Of Sean Clark
>> Sent: Tuesday, May 18, 2010 11:53 AM
>> To: Jayson Baker; randal k
>> Cc: outages at outages.org
>> Subject: Re: [outages] Denver Level3 problems?
>>  
>> We had many customer complaints and all our  IP SLA tests spiked between 1110
>> and 1135 est for Level 3 in our Denver market.  But we also saw the problems
>> in DAL on our level 3 peer (SLA test as well). Some event happened in the
>> Denver area on level 3, we have opened a ticket with them and they reported to
>> us they have had a lot of customer call in for the same thing. They have not
>> given us an RFO yetŠ
>>  
>>  
>>
>> From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On
>> Behalf Of Jayson Baker
>> Sent: Tuesday, May 18, 2010 12:31
>> To: randal k
>> Cc: outages at outages.org
>> Subject: Re: [outages] Denver Level3 problems?
>>  
>> We've been having issues staying connected to Google Chat for the last hour or
>> so.
>>
>> And it just so happens we're connected through you.  :-)
>>
>>
>> Best Wishes,
>> Jayson A. Baker
>> Peak Internet
>>
>> On Tue, May 18, 2010 at 9:51 AM, randal k <rkohutek at gmail.com> wrote:
>> Did anybody else see Denver Level3 service take a hit this morning? Our logs
>> indicated nothing, but we saw various blackholing all over their network for
>> ~20 minutes, starting around 9:15am MST.
>>
>> Everybody have a great day -
>> Randal
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>  
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>     
>
>
> Bradley D. Bopp
> Director Of  Engineering
> http://www.nationalnet.com
> http://as22384.peeringdb.net
>
> NationalNet is committed to the highest level of Customer Service available
> in the Web Hosting business.  If you have any questions, comments or
> concerns feel free to contact us at 678-247-7000 ext 1 (or toll-free,
> 888-4-NATNET).
>  
>
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>   
_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages