[Outages-discussion] [outages] Spectrum/Charter IPv6 connectivity to CloudFlare down in Texas
Daniel Marks
daniel.marks at maymobility.com
Thu Apr 20 01:31:00 EDT 2023
On my end, it seems that it's not only Cloudflare, I also can't reach Quad9
or our AWS EC2 instances at us-east-1 over IPv6. Google and Facebook seem
to be reachable over IPv6, though.
On Thu, Apr 20, 2023 at 1:23 AM Jeremy Chadwick via Outages-discussion <
outages-discussion at outages.org> wrote:
> (Moving to -discussion because there's nothing concrete in what I'm
> about to say, i.e. nothing worth waking people up for.)
>
> 1. Your mtrs have lost their monospace/whitespace formatting for
> whatever reason, making them difficult to read.
>
> 2. Your IPv6 mtr looks like it indicates POSSIBLE packet loss starting
> at hop #4 and seems to trickle down through hops #5/6/7. I can't be
> 100% certain the loss is real (it could be ICMP deprio on Charter's
> routers, but only for IPv6?!) because I have not seen what a working v6
> mtr from you which uses the same routers/path. If it IS real loss, then
> the problem is much more likely within Spectrum/Charter's network, but I
> can't say that with absolute certainty, leading me to:
>
> 2. You have no return-path mtr for either v4 or v6, so you only are able
> to see 50% of the entire picture. Asymmetric routing is commonplace
> these days, which is why you need it from both directions.
>
> If you really and absolutely think this is a CloudFlare problem (I am
> not sold on that given the above two points), then we continue with:
>
> 3. Based on 141.101.74.104 in your v4 mtr, it looks like
> Spectrum/Charter has a peering agreement with CloudFlare.
>
> 4. CloudFlare does not offer a public looking glass, but they do offer
> an HTTP API-based traceroute, though it is sub-par in quality and kind
> of bizarre, not to mention I don't know if it supports v6:
>
> https://blog.clucas.fr/2021/11/18/traceroute-from-cloudflare-without-looking-glass/
>
> Given items #3/4, have you tried reaching out to CloudFlare for a
> return-path mtr back to whatever your v6 source IP is?
>
> Their contact information is public: https://www.peeringdb.com/net/4224
>
> --
> | Jeremy Chadwick jdc_at_koitsu.org |
> | UNIX Systems Administrator PGP 0x2A389531 |
> | Making life hard for others since 1977. |
>
> On Wed, Apr 19, 2023 at 11:37:10PM -0400, Daniel Marks via Outages wrote:
> > Have been dealing with this for 2 days, nobody at Spectrum can figure
> this out.
> >
> > Daniel Marks
> > May Mobility, Inc
> >
> > >
> > > On Apr 19, 2023, at 19:26, Carl Perry via Outages <outages at outages.org>
> wrote:
> > >
> > > Things have been rather slow the last couple of days, just figured
> out why; No IPv6 connectivity to any CloudFlare services.
> > >
> > > > mtr -6 --report-wide cloudflare.com
> > > Start: 2023-04-19T23:20:42+0000
> > > HOST: defiant Loss% Snt Last Avg Best Wrst StDev
> > > 1.|-- 2603-8081-2340-2868-0215-17ff-fe94-6457.res6.spectrum.com
> 0.0% 10 3.6 19.1 1.5 93.2 36.1
> > > 2.|-- 2603-90c5-0003-0180-0000-0000-0000-0001.inf6.spectrum.com
> 0.0% 10 14.5 36.6 11.3 171.2 50.0
> > > 3.|-- lag-63.hcr02rdrktxaz.netops.charter.com 0.0% 10 18.8
> 37.3 12.0 143.9 45.3
> > > 4.|-- lag-19.ausxtxir02r.netops.charter.com 60.0% 10 65.8
> 30.2 11.7 65.8 24.9
> > > 5.|-- lag-22.rcr01hstqtx02.netops.charter.com 80.0% 10 26.5
> 25.5 24.5 26.5 1.4
> > > 6.|-- lag-16.hstqtx0209w-bcr00.netops.charter.com 90.0% 10
> 25.7 25.7 25.7 25.7 0.0
> > > 7.|-- lag-12.dllstx976iw-bcr00.netops.charter.com 40.0% 10
> 25.4 26.3 18.2 44.4 9.5
> > > 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
> > >
> > > IPv4 works however:
> > >
> > > > mtr -4 --report-wide cloudflare.com
> > > Start: 2023-04-19T23:20:21+0000
> > > HOST: defiant Loss% Snt Last
> Avg Best Wrst StDev
> > > 1.|-- _gateway 0.0% 10 2.1
> 1.8 1.4 2.1 0.2
> > > 2.|-- cpe-70-114-160-1.austin.res.rr.com 0.0% 10 51.9
> 34.4 14.3 149.7 42.1
> > > 3.|-- lag-63.hcr01rdrktxaz.netops.charter.com 0.0% 10 13.1
> 38.5 13.1 159.1 44.6
> > > 4.|-- lag-19.ausutxla01r.netops.charter.com 0.0% 10 18.1
> 21.0 14.3 34.8 6.0
> > > 5.|-- lag-22.rcr01dllatx37.netops.charter.com 0.0% 10 23.4
> 23.3 18.3 33.3 4.3
> > > 6.|-- lag-14.dllstx976iw-bcr00.netops.charter.com 0.0% 10 19.8
> 27.0 16.1 51.2 11.9
> > > 7.|-- lag-0.pr3.dfw10.netops.charter.com 0.0% 10 39.3
> 28.5 14.1 62.3 16.7
> > > 8.|-- 141.101.74.104 40.0% 10 53.2
> 44.3 19.4 120.2 39.2
> > > 9.|-- 172.71.168.2 0.0% 10 28.3
> 28.8 10.8 79.4 19.5
> > > 10.|-- 104.16.133.229 0.0% 10 122.0
> 42.8 16.9 122.0 34.4
> > >
> > > It's kind of wild how many things don't fall back to IPv4 quickly
> (lots of Android TV apps) or at all (.NET). I tried reaching out to
> Spectrum support, but being a lowly consumer I was only told that "The
> modem doesn't have any filters" and that "the problem may be on the
> CloudFlare side", which was sadly what I predicted. So, I figured I would
> raise the issue here in the hopes that actual network people would see it.
> > >
> > > -Carl
> > >
> > >
> > > _______________________________________________
> > > Outages mailing list
> > > Outages at outages.org
> > >
> https://www.google.com/url?q=https://puck.nether.net/mailman/listinfo/outages&source=gmail-imap&ust=1682551587000000&usg=AOvVaw0omaxNXXnf3Ff4dHAQDhA8
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org
> > https://puck.nether.net/mailman/listinfo/outages
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
--
-Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages-discussion/attachments/20230420/e530f66c/attachment-0001.htm>
More information about the Outages-discussion
mailing list