[outages] RESOLVED: Re: AT&T SFO to Twitter -- possible routing issue or BGP hijack

Jeremy Chadwick jdc at koitsu.org
Thu Jun 29 06:01:38 EDT 2023


To close this matter out: looks like someone noticed and now Twitter
traffic is going through PCCW Global's San Jose location.  The latency
is still higher than it should be (ideally), but a 3x improvement is
better than none.

                                                                      Packets               Pings
 Host                                                               Loss%   Snt   Rcv  Last   Avg  Best  Wrst
 1. AS???    192.168.1.254                                           0.0%     7     7   0.4   0.5   0.2   0.7
 2. AS7018   172-10-232-1.lightspeed.sntcca.sbcglobal.net (172.10.2  0.0%     7     7   2.1   2.5   1.5   3.4
 3. AS7018   71.148.149.42 (71.148.149.42)                           0.0%     7     7   2.7   2.8   1.3   4.0
 4. AS7018   12.242.117.22 (12.242.117.22)                           0.0%     7     7   7.0   6.1   4.0   8.6
 5. AS7018   att-gw.sfo.pccw.net (192.205.32.82)                     0.0%     7     7   7.6   6.7   5.6   7.7
 6. AS3491   Twitter.BE16.br04.sjo01.pccwbtn.net (63.218.179.202)    0.0%     7     7  66.0  64.4  63.3  66.0
 7. (waiting for reply)
 8. AS13414  104.244.42.65 (104.244.42.65)                           0.0%     7     7  55.2  54.7  53.9  55.7

-- 
| Jeremy Chadwick                              jdc_at_koitsu.org |
| UNIX Systems Administrator                      PGP 0x2A389531 |
| Making life hard for others since 1977.                        |

On Sat, Jun 24, 2023 at 06:17:12AM +0000, Jeremy Chadwick via Outages wrote:
> Found twitter.com was not loading tonight.  Dug in.
> 
> $ host www.twitter.com
> www.twitter.com is an alias for twitter.com.
> twitter.com has address 104.244.42.129
> twitter.com mail is handled by 30 ASPMX2.GOOGLEMAIL.com.
> twitter.com mail is handled by 20 alt2.aspmx.l.google.com.
> twitter.com mail is handled by 20 alt1.aspmx.l.google.com.
> twitter.com mail is handled by 10 aspmx.l.google.com.
> twitter.com mail is handled by 30 ASPMX3.GOOGLEMAIL.com.
> 
> $ dig ns twitter.com +short
> b.r06.twtrdns.net.
> a.r06.twtrdns.net.
> c.r06.twtrdns.net.
> d.r06.twtrdns.net.
> a.u06.twtrdns.net.
> b.u06.twtrdns.net.
> c.u06.twtrdns.net.
> d.u06.twtrdns.net.
> 
> $ dig @b.r06.twtrdns.net a twitter.com +short
> 104.244.42.129
> 
> And 104.244.42.129 does indeed point to Twitter (per WHOIS/ARIN), so doesn't
> appear to be a DNS-related thing.  Onward we go:
> 
> $ mtr www.twitter.com
>                                                                       Packets               Pings
>  Host                                                               Loss%   Snt   Rcv  Last   Avg  Best  Wrst
>  1. 192.168.1.254                                                    0.0%     9     9   0.6   0.6   0.5   0.8
>  2. 172-10-232-1.lightspeed.sntcca.sbcglobal.net (172.10.232.1)      0.0%     9     9   2.3   2.3   1.5   4.3
>  3. 71.148.149.42 (71.148.149.42)                                    0.0%     8     8   2.6   2.7   1.7   3.9
>  4. 12.242.117.22 (12.242.117.22)                                    0.0%     8     8   4.6   6.1   3.6   8.2
>  5. att-gw.sfo.pccw.net (192.205.32.82)                              0.0%     8     8   6.7   6.5   5.3   8.1
>  6. Bundle-Ether45.br04.osa01.pccwbtn.net (63.223.26.30)             0.0%     8     8 121.6 121.1 119.6 122.8
>  7. 63-222-51-222.static.pccwglobal.net (63.222.51.222)              0.0%     8     8 154.3 154.4 153.3 155.8
>  8. (waiting for reply)
>  9. 104.244.42.129 (104.244.42.129)                                  0.0%     8     8 151.5 152.6 151.4 153.8
> 
> $ mtr -z www.twitter.com
> 
>                                                                       Packets               Pings
>  Host                                                               Loss%   Snt   Rcv  Last   Avg  Best  Wrst
>  1. AS???    192.168.1.254                                           0.0%     8     8   0.8   0.6   0.3   0.8
>  2. AS7018   172-10-232-1.lightspeed.sntcca.sbcglobal.net (172.10.2  0.0%     8     8   1.3   2.2   1.0   4.1
>  3. AS7018   71.148.149.42 (71.148.149.42)                           0.0%     8     8   2.2   3.4   1.9   5.0
>  4. AS7018   12.242.117.22 (12.242.117.22)                           0.0%     8     8   6.0   6.1   4.5   7.6
>  5. AS7018   att-gw.sfo.pccw.net (192.205.32.82)                     0.0%     8     8   5.6   6.2   4.8   7.7
>  6. AS3491   Bundle-Ether45.br04.osa01.pccwbtn.net (63.223.26.30)    0.0%     8     8 120.4 120.7 120.0 121.5
>  7. AS3491   63-222-51-222.static.pccwglobal.net (63.222.51.222)     0.0%     8     8 154.1 155.4 153.6 161.5
>  8. (waiting for reply)
>  9. AS13414  104.244.42.129 (104.244.42.129)                         0.0%     7     7 152.7 152.2 150.8 154.0
> 
> AS3491 (pccwbtn.net) is PCCW Global, though a WHOIS on pccwbtn.net says
> they're PCCW-HKT out of Hong Kong, which would explains the huge jump in
> latency (6ms -> 121ms) since I'm located in California.  63.223.26.30 is
> also PCCW Global.
> 
> PeeringDB says https://www.peeringdb.com/net/674 (AT&T) has a looking
> glass server at http://route-server.ip.att.net/ but the webserver is not
> listening on TCP port 80, nor 443:
> 
> $ telnet route-server.ip.att.net 80
> Trying 12.0.1.28...
> telnet: connect to address 12.0.1.28: Connection refused
> telnet: Unable to connect to remote host
> $ telnet route-server.ip.att.net 443
> Trying 12.0.1.28...
> telnet: connect to address 12.0.1.28: Connection refused
> telnet: Unable to connect to remote host
> 
> And for those that want source and destinations:
> 
> src IP: 107.197.104.143 (AT&T Fibre)
> dst IP: 104.244.42.129  (Twitter)
> 
> -- 
> | Jeremy Chadwick                              jdc_at_koitsu.org |
> | UNIX Systems Administrator                      PGP 0x2A389531 |
> | Making life hard for others since 1977.                        |
> 
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
> 


More information about the Outages mailing list