[outages] [EXT] Re: Loss of routes via Telia

[Eric Dugas]

I've got the following update a few minutes ago:

Dear Customer,

We regret to inform you that your services were affected by an
incident occurred at 16:00 UTC during a routine update of a routing
policy for aggregated prefixes in Telia Carrier IP Core network. This
caused traffic to prefixes contained within the aggregates to be
blackholed, resulting in a impact on some parts of the network.

When the underlying problem source was traced, the configuration was
rolled back to the earlier working version of the routing policy
(17:05 UTC). Affected services started to recover gradually after this
operation was applied. No further disturbances related to this
incident are expected.

Due to the wide impact on several customer services, a number of
complaints was received by our Customer Support Centre, resulting in
delays in the communication with customers via email and/or phone. We
apologize for any inconvenience this incident has caused to your
services.

On Thu, Oct 7, 2021 at 1:34 PM Jake Zack via Outages
<outages at outages.org> wrote:
>
> Hey Jared,
>
>
>
> I have an in-house global monitoring network that tracks availability of my DNS anycast clouds…and I’m seeing issues in Europe with Telia, too.
>
>
>
> Of my ~105 monitoring probes, ~18 of them are intermittently seeing certain things via Telia go unreachable for 8-10 minutes at a time.
>
>
>
> Has happened 3 times in the last hour.
>
>
>
> -Jacob Zack
>
> DNS Architect – CIRA (.CA TLD)
>
>
>
> From: Outages <outages-bounces at outages.org> on behalf of "outages at outages.org" <outages at outages.org>
> Reply-To: Jared Geiger <compuwizz at gmail.com>
> Date: Thursday, October 7, 2021 at 1:28 PM
> To: "outages at outages.org" <outages at outages.org>
> Subject: [EXT] Re: [outages] Loss of routes via Telia
>
>
>
> I am now seeing full routes from them. No word yet from the NOC.
>
>
>
> On Thu, Oct 7, 2021 at 10:04 AM Ross Tajvar <ross at tajvar.io> wrote:
>
> We are depeering Telia globally. I believe Cogent is as well. I've heard reports of a loss of routes. I wasn't looking for that but I did see traffic reach their network and drop.
>
>
>
> On Thu, Oct 7, 2021, 12:53 PM Jared Geiger via Outages <outages at outages.org> wrote:
>
> About 30 minutes ago, I lost most of my routes via Telia in Ashburn. I'm down to 81k on full transit. Opened a ticket but can't get ahold of the NOC. Anyone else seeing this?
>
>
>
> My NTT and GTT circuits are getting a spike of BGP updates currently so something seems to be happening.
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages