<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Brush Script MT";
panose-1:3 6 8 2 4 4 6 7 3 4;}
@font-face
{font-family:"\@Brush Script MT";}
@font-face
{font-family:"Andale Mono";
panose-1:2 11 5 9 0 0 0 0 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.gmail-p1, li.gmail-p1, div.gmail-p1
{mso-style-name:gmail-p1;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.gmail-s1
{mso-style-name:gmail-s1;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:655457606;
mso-list-type:hybrid;
mso-list-template-ids:1925625920 232527460 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
mso-ansi-font-size:34.5pt;
font-family:"Brush Script MT";
color:windowtext;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.25in;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.75in;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.25in;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.75in;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.25in;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.75in;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:4.25in;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:4.75in;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hi CJ, <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We have not had any problems over the weekend, however AT&T has not communicated they had any issues or they have fixed any issues.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>This is a great tool that I will keep in the toolbox however and if we see this again I will see that the route looks like.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>The issue seems to be a router silently discarding packets or getting into some routing loop. When I did a packet capture my SYN packets went completely unanswered… I am curious to see what it shows however regarding the routes, I expect it to look OK as the route is the same while its working and while its down.<o:p></o:p></p><div><p class=MsoNormal style='text-autospace:none'><span style='font-size:14.0pt'><o:p> </o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:14.0pt'>-Eduard<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:14.0pt'><o:p> </o:p></span></p></div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoListParagraph style='margin-left:.75in'><span style='font-size:12.0pt;color:black'><br><b>Date: </b>Monday, January 31, 2022 at 2:23 PM<br><b>To: </b>Eduard Tieseler <etieseler@metrolist.net><br><b>Cc: </b>Outages Discussion <outages-discussion@outages.org><br><b>Subject: </b>Re: [Outages-discussion] Nightly AT&T outage on path to certain sites<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p></div><div><div><p class=MsoNormal style='margin-left:.5in'>Might be worthwhile to login to the AT&T route server during that time and see how the routing table looks.<o:p></o:p></p><div><p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p></div><div><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span class=gmail-s1><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'>telnet <a href="http://route-server.ip.att.net">route-server.ip.att.net</a></span></span><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><o:p></o:p></span></p><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span class=gmail-s1><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'>rviews/rviews</span></span><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><o:p></o:p></span></p><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><br><br><o:p></o:p></span></p><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span class=gmail-s1><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'>show route <a href="http://72.26.113.0/24">72.26.113.0/24</a></span></span><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><o:p></o:p></span></p><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><br><br><o:p></o:p></span></p><p class=gmail-p1 style='mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;background-color:rgba(0,0,0,0.9)'><span style='font-size:9.0pt;font-family:"Andale Mono";color:#2FFF12'><br><br><o:p></o:p></span></p></div></div><div><div><p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-left:.5in'>-Chris<o:p></o:p></p></div><div><p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-left:.5in'>On Fri, Jan 28, 2022 at 8:54 AM Eduard <<a href="mailto:etieseler@metrolist.net">etieseler@metrolist.net</a>> wrote:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>Curious if anyone has been seeing issues accessing certain sites with AT&T as your ISP. Seems to mostly affect northern California.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>It starts roughly around 7:30 PM Pacific time and is working again in the morning.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>Below is a failed and successful traceroute. Anyone else seeing anything similar? AT&T support has been an excellent … firewall … at getting to anyone useful that would be able to resolve this issue.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>Failed:<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>traceroute to <a href="http://sacmsso.rapmls.com" target="_blank">sacmsso.rapmls.com</a> (72.26.113.104), 64 hops max, 52 byte packets<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>1 172.16.40.1 (172.16.40.1) 6.608 ms 5.795 ms 134.370 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>2 192.168.1.254 (192.168.1.254) 6.932 ms 5.998 ms 5.015 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>3 <a href="http://172-9-164-1.lightspeed.frokca.sbcglobal.net" target="_blank">172-9-164-1.lightspeed.frokca.sbcglobal.net</a> (172.9.164.1) 7.681 ms 136.346 ms 10.492 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>4 71.147.199.76 (71.147.199.76) 9.440 ms 120.867 ms 7.360 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>5 12.122.160.166 (12.122.160.166) 10.686 ms 105.594 ms 9.686 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>6 12.122.2.78 (12.122.2.78) 147.798 ms 47.183 ms 13.577 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>7 12.122.114.5 (12.122.114.5) 12.394 ms 39.740 ms 11.997 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>8 * * *<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>9 * * *<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>Working:<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>traceroute to <a href="http://sacmsso.rapmls.com" target="_blank">sacmsso.rapmls.com</a> (72.26.113.104), 64 hops max, 52 byte packets<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>1 172.16.40.1 (172.16.40.1) 6.398 ms 9.862 ms 3.846 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>2 192.168.1.254 (192.168.1.254) 5.835 ms 4.889 ms 4.610 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>3 <a href="http://172-9-164-1.lightspeed.frokca.sbcglobal.net" target="_blank">172-9-164-1.lightspeed.frokca.sbcglobal.net</a> (172.9.164.1) 5.983 ms 5.927 ms 5.980 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>4 71.147.199.76 (71.147.199.76) 13.065 ms 8.224 ms 7.662 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>5 12.122.160.166 (12.122.160.166) 15.948 ms 12.303 ms 17.810 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>6 12.122.2.78 (12.122.2.78) 12.036 ms 15.065 ms 17.014 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>7 12.122.114.5 (12.122.114.5) 12.024 ms 12.189 ms 11.051 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>8 192.205.37.70 (192.205.37.70) 13.407 ms 13.516 ms 15.354 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>9 <a href="http://if-ae-26-2.tcore2.sv1-santaclara.as6453.net" target="_blank">if-ae-26-2.tcore2.sv1-santaclara.as6453.net</a> (209.58.86.36) 21.055 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <a href="http://if-ae-18-2.tcore2.sv1-santaclara.as6453.net" target="_blank">if-ae-18-2.tcore2.sv1-santaclara.as6453.net</a> (63.243.205.73) 20.964 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <a href="http://if-ae-18-5.tcore2.sv1-santaclara.as6453.net" target="_blank">if-ae-18-5.tcore2.sv1-santaclara.as6453.net</a> (63.243.205.37) 23.039 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>10 <a href="http://if-ae-0-2.tcore1.sv1-santaclara.as6453.net" target="_blank">if-ae-0-2.tcore1.sv1-santaclara.as6453.net</a> (63.243.251.1) 20.844 ms 22.154 ms 21.141 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>11 <a href="http://if-ae-60-2.tcore1.lvw-losangeles.as6453.net" target="_blank">if-ae-60-2.tcore1.lvw-losangeles.as6453.net</a> (63.243.250.55) 22.369 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <a href="http://if-ae-8-3.tcore1.lvw-losangeles.as6453.net" target="_blank">if-ae-8-3.tcore1.lvw-losangeles.as6453.net</a> (63.243.250.59) 24.226 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <a href="http://if-ae-8-5.tcore1.lvw-losangeles.as6453.net" target="_blank">if-ae-8-5.tcore1.lvw-losangeles.as6453.net</a> (63.243.250.63) 21.236 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>12 66.110.59.253 (66.110.59.253) 22.648 ms 22.208 ms 21.728 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>13 <a href="http://xe-1-1-0.ar01-dtla01.alchemy.net" target="_blank">xe-1-1-0.ar01-dtla01.alchemy.net</a> (66.186.30.30) 22.228 ms 22.450 ms 20.991 ms<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'>14 * * *<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in'><span style='font-size:14.0pt;color:#18376A'>-Eduard</span><o:p></o:p></p></div></div><p class=MsoNormal style='margin-left:.5in'>_______________________________________________<br>Outages-discussion mailing list<br><a href="mailto:Outages-discussion@outages.org" target="_blank">Outages-discussion@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages-discussion" target="_blank">https://puck.nether.net/mailman/listinfo/outages-discussion</a><o:p></o:p></p></blockquote></div></div></div></body></html>