[outages] Reachability issues AS7018 <-> AS63267

Fri Feb 8 21:07:37 EST 2019

Thanks everyone for the feedback I'm continuing to work on things. I have fixed the ROA/RPKI object after slapping myself for not adding my own AS number as a valid advertiser when I set that up two years ago.
It's showing up valid at some places, invalid at others, I'm not sure how long it will take to overwrite the invalid information.

I'm working with Charter but they're head scratching as well. Iris is a relatively small provider and hands off to many others very quickly and if I shut down the Charter side link my iris traffic still routes via charter right through the same telia router trying to get to AT&T.

I'll take a bit of time to let things propagate and for Charter to push and then probably take this to Nanog to see if I can find someone AT&T over there. Cause their ARIN contacts sure don't answer phones or emails.

(I don't even remember why I dropped outages and Nanog, I should be on them...)

-----Original Message-----
From: Ryan Kearney [mailto:ryan at kearney.io]
Sent: Friday, February 8, 2019 6:29 PM
To: Eric J Esslinger <eesslinger at fpu-tn.com>
Cc: outages at outages.org
Subject: Re: [outages] Reachability issues AS7018 <-> AS63267

You have ROAs for 104.171.208.0/20 allowing ASN 3356, 11758, and 20115 to originate that block. AS AS63267 is not listed. Perhaps AT&T is filtering out your advertisement as a result?

I would suggest fixing your ROA / RPKI objects for that route.

On Fri, Feb 8, 2019 at 5:40 PM Eric J Esslinger via Outages <outages at outages.org> wrote:
>
> Let me preface this with I'm sorry for some very sketchy info here and only a couple of traceroutes, but I've got pneumonia and am working from home atm.
>
> Howdy. I'm Eric Esslinger, IS Manager at FPU, owner of AS63267 and our
> associated netblock of 104.171.208.0/20 We're unable to receive communication from AS7018 (AT&T Incorporated) through either of our upstream providers (Spectrum Enterprise aka Charter, and Iris networks).
>
> Looking glass sites in general show our bgp advertisements to be correct.
>
> Traceroutes to any site that requires AT&T that we've identified as a problem show the following (in the Charter case, which is our prefer:
>
>   1     6 ms     5 ms     6 ms  104.171.208.1
>   2     6 ms     5 ms     5 ms  96-33-106-193.static.kgpt.tn.charter.com [96.33.106.193]
>   3     5 ms     5 ms     5 ms  dtr02mnchtn-tge-0-7-0-6.mnch.tn.charter.com [96.34.71.10]
>   4    11 ms    11 ms    11 ms  crr01kgpttn-bue-200.kgpt.tn.charter.com [96.34.69.168]
>   5    20 ms    17 ms    17 ms  crr02spbgsc-bue-401.spbg.sc.charter.com [96.34.69.254]
>   6    22 ms    20 ms    24 ms  bbr01spbgsc-bue-4.spbg.sc.charter.com [96.34.2.50]
>   7    21 ms    20 ms    20 ms  https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fcha-b1-link.telia.net&c=E,1,0n5m0mg2rs8SmGqPXFqTY-y-u3Zc6YTKckX6rs0lsaW_KHFsTtj468hGafX54eTuC6lJxT8hHJ3sD5yuIostnhrEXVXZP9MwopTeHvpDyrK2vID0PA,,&typo=1 [213.248.98.25]
>   8    24 ms    23 ms    26 ms  https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fatl-b22-link.telia.net&c=E,1,FJaGtSZD6d_s3FQz0M2pDmXEhxwOB5mjEicKqV0sj_yTVAE3rdlrt0VEX-eB6myYyAvKicEZ6ChR5q7HTGwbACf5P9lEyT2Mu6w8ywea_eoKvysNF8us&typo=1 [213.155.137.122]
>   9     *        *        *     Request timed out.
>
> From a Charter cable customer in town, the same trace gets farther:
>
> 1  75-130-85-45.static.kgpt.tn.charter.com (75.130.85.45)  1.832 ms
> 1.174 ms  0.982 ms
> 2  *  *  *
> 3  acr09mnchtn-gbe-2-28.mnch.tn.charter.com (96.34.70.72)  13.900 ms
> 12.677 ms  12.753 ms
> 4  96-34-12-55.static.unas.mo.charter.com (96.34.12.55)  16.337 ms
> 14.036 ms  16.950 ms
> 5  96-34-13-27.static.unas.mo.charter.com (96.34.13.27)  12.622 ms
> 15.170 ms  11.405 ms
> 6  96-34-12-133.static.unas.mo.charter.com (96.34.12.133)  19.547 ms
> 96-34-12-143.static.unas.mo.charter.com (96.34.12.143)  23.507 ms
> 19.327 ms
> 7  crr02spbgsc-bue-401.spbg.sc.charter.com (96.34.69.254)  23.754 ms
> 31.793 ms  24.025 ms
> 8  bbr01spbgsc-bue-4.spbg.sc.charter.com (96.34.2.50)  30.195 ms
> 23.882 ms  28.211 ms
> 9
> https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fcha-b1-link.telia.
> net&c=E,1,vYtDy2_lF_5tK6YSF5pBvPMuhLPAbFJ94XtFX7ALWzmsg6hnmcb0LCLb8YrM
> uklnW8LT2ngwpjEe6wWaHTDLvWZhW9FJXkow5p5SUiGMqV9ja50,&typo=1
> (213.248.98.25)  39.141 ms  39.260 ms  38.552 ms
> 10
> https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fatl-b22-link.telia
> .net&c=E,1,cjSat-D2lK8sGVHUDb42ZtdtBdiuMhVyvm1MwyF0mgKlZxbMb8SiSvA5uSk
> jFhvqsYGoq8HuqHSESpH5tnHp4JZQQKjqIwWMk7TgP6xh_LDbTJJN&typo=1
> (213.155.137.122)  40.886 ms  44.736 ms  39.673 ms
> 11  192.205.33.41 (192.205.33.41)  54.064 ms  45.201 ms  44.614 ms
> (continues)
>
> So at a minimum there is a problem at the 192.205.33.41 device.
> This is currently having a major impact on us, as both a provider and the Electric, Water, Sewer, and Gas utility for Fayetteville and Lincoln County TN, the Fayetteville/Lincoln County Tennessee hospital, City and County municipal government, the Police, Sheriff's department and 911, local EMA, and 4 of the 5 biggest banks, just to cover the more critical people affected.
>
> Does AT&T have anyone on this list that can help us out here? We've been beating against a brick wall for two days, hampered somewhat by my having walking pneumonia. Most of our contacts with AT&T are met with the standard brick wall of 'you're not our customer'.
>
> If anyone wants a reachability test, feel free to ping 104.171.208.1
>
> I can be contacted off list via eesslinger at fpu-tn.com
>
> Thanks for your attention.
>
> Eric Esslinger
> Information Services Manager
> Fayetteville Public Utilities
> 408 College St W
> Fayetteville, TN 37334
> 931-433-1522
>
> This message may contain confidential and/or proprietary information and is intended for the person/entity to whom it was originally addressed. Any use by others is strictly prohibited.
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2f
> mailman%2flistinfo%2foutages&c=E,1,ggkzxy9Dof9rMnhObjpwDGLTyUza1fl-Wq8
> IyNcVj0MThxFD3di_juigoULPiT3L6_ljlE1_BXx2ZXvnpyvGrKYH44iwmJw4lv4clrkor
> sQ,&typo=1

This message may contain confidential and/or proprietary information and is intended for the person/entity to whom it was originally addressed. Any use by others is strictly prohibited.