[Outages-discussion] [outages] OVH IPv6 100% Package Loss in BHS (Canada DC)
Jim Popovitch
jimpop at domainmail.org
Mon Sep 7 14:18:12 EDT 2020
On Mon, 2020-09-07 at 20:15 +0300, Vlad Dascalu via Outages wrote:
> Hey
>
> Does anyone have a proper contact point for OVH NOC (besides noc at ovh.net ?)
>
> Traceroute is showing 100% package loss on an IPv6 path in their AS, despite being routable from other ASes. I've reported it for hours but have no reaction from them. Is it normal?
>
> ( Should we, as an Internet community, make a plan of attack regarding accountability on responsiveness for large ISP providers in the light of the recent Level3 incident? )
>
> Anyway, below you can find the problematic traceroute.
>
> # wget http://[2607:5300:60:6195::1]
> --2020-09-07 05:39:39-- http://[2607:5300:60:6195::1]/
> Connecting to [2607:5300:60:6195::1]:80... failed: Connection timed out.
>
> # traceroute 2607:5300:60:6195::1
> traceroute to 2607:5300:60:6195::1 (2607:5300:60:6195::1), 30 hops max, 80 byte packets
> 1 * * *
> 2 xe-0-0-25-1.a01.atlnga05.us.bb.gin.ntt.net (2001:418:0:5000::622) 1.012 ms 1.012 ms 1.017 ms
> 3 ae-6.r23.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::22a) 0.635 ms ae-16.r22.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::20e) 1.004 ms ae-6.r23.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::22a) 0.646 ms
> 4 ae-9.r04.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::14d) 1.077 ms 1.079 ms 1.075 ms
> 5 2001:418:0:4000::a2 (2001:418:0:4000::a2) 1.052 ms * 1.036 ms
> 6 rest-bb1-v6.telia.net (2001:2000:3019:73::1) 16.727 ms ash-bb2-v6.telia.net (2001:2000:3019:74::1) 13.431 ms 13.480 ms
> 7 * * *
> 8 be100-1007.nwk-5-a9.nj.us (2607:5300::2) 17.916 ms * *
> 9 be102.bhs-g2-nc5.qc.ca (2607:5300::1ce) 27.694 ms 25.405 ms 29.696 ms
> 10 vl100.bhs-d2-a75.qc.ca (2607:5300::1cd) 27.329 ms 27.455 ms 24.885 ms
> 11 2001:41d0:0:50::6:80b (2001:41d0:0:50::6:80b) 24.216 ms 24.391 ms 24.381 ms
> 12 * * *
> 13 * * *
> 14 * * *
> 15 * * *
> 16 * * *
> 17 * * *
> 18 * * *
> 19 * * *
> 20 * * *
> 21 * * *
> 22 * * *
> 23 * * *
> 24 * * *
> 25 * * *
> 26 * * *
> 27 * * *
> 28 * * *
> 29 * * *
> 30 * * *
>
> If it helps, I tested connectivity through 3 other continents and they work ok. Seems BGP resolves it to the right ASN but there is a routing dead-end inside the BHS router config.
>
> Traceroute working from other locations, if it helps:
>
> Germany (Hetzner):
> traceroute to ca1.beetux.com (2607:5300:60:6195::1), 30 hops max, 80 byte packets
> 1 ex9k1.dc7.fsn1.hetzner.com (2a01:4f8::a:16:a) 0.489 ms 0.478 ms 0.471 ms
> 2 core23.fsn1.hetzner.com (2a01:4f8:0:3::271) 0.464 ms core24.fsn1.hetzner.com (2a01:4f8:0:3::275) 0.434 ms core23.fsn1.hetzner.com (2a01:4f8:0:3::271) 0.447 ms
> 3 core5.fra.hetzner.com (2a01:4f8:0:3::33a) 16.099 ms core0.fra.hetzner.com (2a01:4f8:0:3::365) 5.107 ms core5.fra.hetzner.com (2a01:4f8:0:3::33a) 16.090 ms
> 4 core9.fra.hetzner.com (2a01:4f8:0:3::2fa) 5.438 ms 5.433 ms core8.fra.hetzner.com (2a01:4f8:0:3::33d) 5.409 ms
> 5 * * *
> 6 2001:41d0::257b (2001:41d0::257b) 6.192 ms * 2001:41d0::2579 (2001:41d0::2579) 5.766 ms
> 7 2001:41d0:aaaa:100::3 (2001:41d0:aaaa:100::3) 22.947 ms 16.798 ms 23.640 ms
> 8 * * 2001:41d0:aaaa:100::2 (2001:41d0:aaaa:100::2) 6.062 ms
> 9 * * *
> 10 * * *
> 11 * * *
> 12 be102.bhs-g1-nc5.qc.ca (2607:5300::1c6) 95.367 ms 96.431 ms be102.bhs-g2-nc5.qc.ca (2607:5300::1ce) 98.105 ms
> 13 vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 94.740 ms vl100.bhs-d2-a75.qc.ca (2607:5300::1cd) 95.564 ms 93.632 ms
> 14 2001:41d0:0:50::6:809 (2001:41d0:0:50::6:809) 94.240 ms 2001:41d0:0:50::6:80d (2001:41d0:0:50::6:80d) 91.366 ms 2001:41d0:0:50::6:80b (2001:41d0:0:50::6:80b) 94.574 ms
> 15 * * *
> 16 ca1.beetux.com (2607:5300:60:6195::1) 91.282 ms !X 92.104 ms !X 92.496 ms !X
>
> Brazil:
> traceroute to ca1.beetux.com (2607:5300:60:6195::1), 30 hops max, 80 byte packet s
> 1 hsrp.gcore.lu (2a03:90c0:186::3) 0.704 ms 0.457 ms 0.669 ms
> 2 2a03:90c0:180:1:253::226 (2a03:90c0:180:1:253::226) 0.695 ms 0.668 ms 0.6 44 ms
> 3 g-core-labs.sanpaolo7.spa.seabone.net (2001:41a8:5200:2::1e1) 0.734 ms 2001 :1498:1:bf1::1 (2001:1498:1:bf1::1) 0.723 ms 0.701 ms
> 4 mia15-loop0-v6.mia.seabone.net (2001:41a8:4020::f) 109.816 ms 110.905 ms 2 001:1498:1::100:138 (2001:1498:1::100:138) 134.126 ms
> 5 2001:1498:1::32:113 (2001:1498:1::32:113) 110.835 ms * 2001:1498:1::32:146 (2001:1498:1::32:146) 131.040 ms
> 6 * * *
> 7 * be102.bhs-g1-nc5.qc.ca (2607:5300::1c6) 137.615 ms *
> 8 vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 137.997 ms * be102.bhs-g1-nc5.qc.ca (2607:5300::1c6) 133.250 ms
> 9 vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 132.065 ms 2001:41d0:0:50::6:80d (2 001:41d0:0:50::6:80d) 130.865 ms vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 132.1 55 ms
> 10 2001:41d0:0:50::6:80f (2001:41d0:0:50::6:80f) 132.028 ms * *
> 11 ca1.beetux.com (2607:5300:60:6195::1) 121.118 ms !X 132.094 ms !X 148.398 ms !X
>
> Singapore:
> traceroute to ca1.beetux.com (2607:5300:60:6195::1), 30 hops max, 80 byte packets
> 1 2400:6180:0:d1:ffff:ffff:ffff:fff2 (2400:6180:0:d1:ffff:ffff:ffff:fff2) 0.659 ms 0.638 ms 0.619 ms
> 2 2604:a880:ffff:9::26a (2604:a880:ffff:9::26a) 0.960 ms 2604:a880:ffff:9::276 (2604:a880:ffff:9::276) 1.077 ms 1.057 ms
> 3 2604:a880:ffff:9::265 (2604:a880:ffff:9::265) 0.865 ms 2604:a880:ffff:9::271 (2604:a880:ffff:9::271) 0.884 ms 2604:a880:ffff:9::263 (2604:a880:ffff:9::263) 0.835 ms
> 4 2604:a880:ffff:9:0:1:0:6 (2604:a880:ffff:9:0:1:0:6) 1.148 ms 2604:a880:ffff:9:0:1:0:e (2604:a880:ffff:9:0:1:0:e) 1.135 ms 2604:a880:ffff:9:0:1:0:4 (2604:a880:ffff:9:0:1:0:4) 1.105 ms
> 5 2403:e800:508:100::1 (2403:e800:508:100::1) 1.205 ms * *
> 6 * * *
> 7 * * *
> 8 * * *
> 9 * * *
> 10 be100-104.pao-sv8-bb1-a9.ca.us (2607:5300::24) 168.764 ms 186.737 ms 186.765 ms
> 11 be100-1371.chi-5-a9.il.us (2607:5300::af) 211.537 ms be100-1370.chi-1-a9.il.us (2607:5300::59) 222.876 ms 204.043 ms
> 12 be103.bhs-g1-nc5.qc.ca (2607:5300::1c8) 408.785 ms 265.277 ms 230.064 ms
> 13 vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 230.897 ms 230.898 ms 230.683 ms
> 14 2001:41d0:0:50::6:809 (2001:41d0:0:50::6:809) 228.107 ms 246.649 ms 2001:41d0:0:50::6:80b (2001:41d0:0:50::6:80b) 250.099 ms
> 15 * * *
> 16 ca1.beetux.com (2607:5300:60:6195::1) 244.269 ms !X 230.678 ms !X 245.223 ms !X
First, your post is not worthy of the outages at outages.org list. Please
use outages-discussion at outages.org in the future. Real outages should
go to outages at outages.org BUT understand that posting there will ring
some peoples pagers/mobiles and other's inbox whistles. Please reserve
outages at outages.org for real outages.
Secondly, OVH has never had perfect IPv6 routing, and generally doesn't
care what you or I have to say about their IPv6/routing choices.
Now, you need to know that traceroute is an old tool that no longer is
accurate for testing purposes. Use something like mtr instead. Here's
an mtr to that IP from Atlanta:
~$ mtr -c 1 --report-wide --aslookup --show-ips 2607:5300:60:6195::1
Start: 2020-09-07T14:10:18-0400
HOST: work Loss% Snt Last Avg Best Wrst StDev
1. AS??? ap-ipv6-1494-udp.vpn (fd10:10:23::1) 0.0% 1 12.7 12.7 12.7 12.7 0.0
2. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
3. AS2914 xe-0-0-25-1.a01.atlnga05.us.bb.gin.ntt.net (2001:418:0:5000::622) 0.0% 1 12.9 12.9 12.9 12.9 0.0
4. AS2914 ae-16.r22.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::20e) 0.0% 1 24.8 24.8 24.8 24.8 0.0
5. AS2914 ae-9.r04.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::14d) 0.0% 1 13.3 13.3 13.3 13.3 0.0
6. AS2914 2001:418:0:4000::a2 0.0% 1 12.7 12.7 12.7 12.7 0.0
7. AS1299 ash-bb2-v6.telia.net (2001:2000:3019:74::1) 0.0% 1 26.3 26.3 26.3 26.3 0.0
8. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
9. AS16276 be100-1007.nwk-5-a9.nj.us (2607:5300::2) 0.0% 1 32.3 32.3 32.3 32.3 0.0
10. AS16276 be102.bhs-g2-nc5.qc.ca (2607:5300::1ce) 0.0% 1 38.7 38.7 38.7 38.7 0.0
11. AS16276 vl100.bhs-d2-a75.qc.ca (2607:5300::1cd) 0.0% 1 36.5 36.5 36.5 36.5 0.0
12. AS16276 2001:41d0:0:50::6:80f 0.0% 1 39.3 39.3 39.3 39.3 0.0
13. AS16276 2001:41d0:0:50::2:404d 0.0% 1 37.1 37.1 37.1 37.1 0.0
14. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
Here is a mtr from San Jose:
Start: 2020-09-07T18:14:07+0000
HOST: sjc Loss% Snt Last Avg Best Wrst StDev
1. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
2. AS20473 2001:19f0:ac00:2300::1 0.0% 1 0.5 0.5 0.5 0.5 0.0
3. AS20473 2001:19f0:ac00::ada:15d 0.0% 1 1.1 1.1 1.1 1.1 0.0
4. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
5. AS174 2001:550:2:3::b6:1 0.0% 1 1.1 1.1 1.1 1.1 0.0
6. AS??? ??? 100.0 1 0.0 0.0 0.0 0.0 0.0
7. AS174 be3142.ccr41.sjc03.atlas.cogentco.com (2001:550:0:1000::9a36:1c2) 0.0% 1 2.1 2.1 2.1 2.1 0.0
8. AS16276 sjo-sv5-bb1-a9.ca.us (2607:5300::255e) 0.0% 1 1.9 1.9 1.9 1.9 0.0
9. AS16276 be100-1368.pao-sv8-bb1-a9.ca.us (2607:5300::26) 0.0% 1 22.1 22.1 22.1 22.1 0.0
10. AS16276 be100-1370.chi-1-a9.il.us (2607:5300::59) 0.0% 1 49.6 49.6 49.6 49.6 0.0
11. AS16276 be103.bhs-g1-nc5.qc.ca (2607:5300::1c8) 0.0% 1 72.9 72.9 72.9 72.9 0.0
12. AS16276 vl100.bhs-d1-a75.qc.ca (2607:5300::1c3) 0.0% 1 72.1 72.1 72.1 72.1 0.0
13. AS16276 2001:41d0:0:50::6:809 0.0% 1 71.8 71.8 71.8 71.8 0.0
14. AS16276 2001:41d0:0:50::2:4049 0.0% 1 73.5 73.5 73.5 73.5 0.0
15. AS16276 ca1.beetux.com (2607:5300:60:6195::1) 0.0% 1 71.9 71.9 71.9 71.9 0.0
Those 2 mtr's show me that the drop is interestingly close to your host.
What do you get mtr'ing from your OVH host to 2604:180::9b07:1d5b
(RamNode's ATL IPv6 Test site)
Good luck!
-Jim P.
More information about the Outages-discussion
mailing list