[Outages-discussion] Nightly AT&T outage on path to certain sites

Mon Jan 31 17:44:00 EST 2022

Hi CJ, 

We have not had any problems over the weekend, however AT&T has not communicated they had any issues or they have fixed any issues.

This is a great tool that I will keep in the toolbox however and if we see this again I will see that the route looks like.

The issue seems to be a router silently discarding packets or getting into some routing loop. When I did a packet capture my SYN packets went completely unanswered… I am curious to see what it shows however regarding the routes, I expect it to look OK as the route is the same while its working and while its down.

-Eduard

Date: Monday, January 31, 2022 at 2:23 PM
To: Eduard Tieseler <etieseler at metrolist.net>
Cc: Outages Discussion <outages-discussion at outages.org>
Subject: Re: [Outages-discussion] Nightly AT&T outage on path to certain sites

Might be worthwhile to login to the AT&T route server during that time and see how the routing table looks.

telnet route-server.ip.att.net

rviews/rviews

show route 72.26.113.0/24

-Chris

On Fri, Jan 28, 2022 at 8:54 AM Eduard <etieseler at metrolist.net> wrote:

Curious if anyone has been seeing issues accessing certain sites with AT&T as your ISP. Seems to mostly affect northern California.

It starts roughly around 7:30 PM Pacific time and is working again in the morning.

Below is a failed and successful traceroute. Anyone else seeing anything similar? AT&T support has been an excellent … firewall … at getting to anyone useful that would be able to resolve this issue.

Failed:

traceroute to sacmsso.rapmls.com (72.26.113.104), 64 hops max, 52 byte packets

1  172.16.40.1 (172.16.40.1)  6.608 ms  5.795 ms  134.370 ms

2  192.168.1.254 (192.168.1.254)  6.932 ms  5.998 ms  5.015 ms

3  172-9-164-1.lightspeed.frokca.sbcglobal.net (172.9.164.1)  7.681 ms  136.346 ms  10.492 ms

4  71.147.199.76 (71.147.199.76)  9.440 ms  120.867 ms  7.360 ms

5  12.122.160.166 (12.122.160.166)  10.686 ms  105.594 ms  9.686 ms

6  12.122.2.78 (12.122.2.78)  147.798 ms  47.183 ms  13.577 ms

7  12.122.114.5 (12.122.114.5)  12.394 ms  39.740 ms  11.997 ms

8  * * *

9  * * *

Working:

traceroute to sacmsso.rapmls.com (72.26.113.104), 64 hops max, 52 byte packets

1  172.16.40.1 (172.16.40.1)  6.398 ms  9.862 ms  3.846 ms

2  192.168.1.254 (192.168.1.254)  5.835 ms  4.889 ms  4.610 ms

3  172-9-164-1.lightspeed.frokca.sbcglobal.net (172.9.164.1)  5.983 ms  5.927 ms  5.980 ms

4  71.147.199.76 (71.147.199.76)  13.065 ms  8.224 ms  7.662 ms

5  12.122.160.166 (12.122.160.166)  15.948 ms  12.303 ms  17.810 ms

6  12.122.2.78 (12.122.2.78)  12.036 ms  15.065 ms  17.014 ms

7  12.122.114.5 (12.122.114.5)  12.024 ms  12.189 ms  11.051 ms

8  192.205.37.70 (192.205.37.70)  13.407 ms  13.516 ms  15.354 ms

9  if-ae-26-2.tcore2.sv1-santaclara.as6453.net (209.58.86.36)  21.055 ms

    if-ae-18-2.tcore2.sv1-santaclara.as6453.net (63.243.205.73)  20.964 ms

    if-ae-18-5.tcore2.sv1-santaclara.as6453.net (63.243.205.37)  23.039 ms

10  if-ae-0-2.tcore1.sv1-santaclara.as6453.net (63.243.251.1)  20.844 ms  22.154 ms  21.141 ms

11  if-ae-60-2.tcore1.lvw-losangeles.as6453.net (63.243.250.55)  22.369 ms

    if-ae-8-3.tcore1.lvw-losangeles.as6453.net (63.243.250.59)  24.226 ms

    if-ae-8-5.tcore1.lvw-losangeles.as6453.net (63.243.250.63)  21.236 ms

12  66.110.59.253 (66.110.59.253)  22.648 ms  22.208 ms  21.728 ms

13  xe-1-1-0.ar01-dtla01.alchemy.net (66.186.30.30)  22.228 ms  22.450 ms  20.991 ms

14  * * *

-Eduard

_______________________________________________
Outages-discussion mailing list
Outages-discussion at outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20220131/0b8404a4/attachment-0001.htm>