[outages] IPv6 tunnels in FRA1 on HE.net down?

[Constantine A. Murenin]

I'm not a network engineer, but it sounds like all specific routes for
tserv1.fra1.he.net are expired / not announced, and the default route
is taken instead.  40+ minutes so far...

C.

On 12 June 2013 15:30, Tony McCrory <tony.mccrory at gmail.com> wrote:
> Indeed, traceroute to my fra1 tunnel is definately getting mislaid within
> he.net.
>
> Time to ask for a refund?  oh wait... ;)
>
> Tracing route to tmcc.me [2001:470:1f0a:3ee::2]
> over a maximum of 30 hops:
>
>   1    <1 ms    <1 ms    <1 ms
> 6.9.1.2.2.d.e.f.f.f.f.f.6.7.a.0.3.f.7.9.d.8.4.6.0.b.8.0.1.0.0.2.ip6.arpa
> [2001:8b0:648d:97f3:a76:ffff:fed2:2196]
>   2    65 ms    65 ms    66 ms  c.gormless.thn.aa.net.uk [2001:8b0:0:53::53]
>   3    56 ms    58 ms    61 ms  2001:7f8:4::50e8:1
>   4    70 ms    74 ms    74 ms  40gigabitethernet1-1.core1.lon1.he.net
> [2001:7f8:4::1b1b:1]
>   5   145 ms   141 ms   132 ms  10gigabitethernet10-4.core1.nyc4.he.net
> [2001:470:0:128::1]
>   6   146 ms   150 ms   154 ms  100gigabitethernet7-2.core1.chi1.he.net
> [2001:470:0:298::1]
>   7   211 ms   216 ms   212 ms  10gigabitethernet11-4.core1.pao1.he.net
> [2001:470:0:283::1]
>   8     *        *        *     Request timed out.
>   9     *        *        *     Request timed out.
>  10     *        *        *     Request timed out.
>
>
> On 12 June 2013 23:19, Constantine A. Murenin <mureninc at gmail.com> wrote:
>>
>> I smell a big outage:
>>
>> % date; traceroute tserv1.fra1.he.net
>> Wed Jun 12 15:17:04 PDT 2013
>> traceroute to tserv1.fra1.he.net (216.66.80.30), 30 hops max, 60 byte
>> packets
>>  1  192.168.105.3 (192.168.105.3)  0.673 ms  0.773 ms  0.923 ms
>>  2  10gigabitethernet7-6.core3.fmt2.he.net (65.49.10.217)  1.795 ms
>> 1.811 ms  1.795 ms
>>  3  10gigabitethernet12-1.core1.lax1.he.net (184.105.213.26)  19.773
>> ms 10gigabitethernet10-1.core1.sjc2.he.net (184.105.222.14)  0.786 ms
>> 0.756 ms
>>  4  10gigabitethernet10-8.core1.nyc4.he.net (72.52.92.225)  71.714 ms
>> 76.569 ms 10gigabitethernet14-2.core1.nyc4.he.net (184.105.213.198)
>> 71.690 ms
>>  5  * * *
>>  6  * * *
>>  7  * * *
>>  8  * * *
>>  9  * * *
>> 10  * * *
>> 11  * * *
>> 12  * * *
>> 13  * * *
>> 14  * * *
>> 15  * *^C
>>
>> However, a "reverse" traceroute works fine:
>>
>> % date; traceroute ns1.he.net
>> Wed Jun 12 15:18:47 PDT 2013
>> traceroute to ns1.he.net (216.218.130.2), 64 hops max, 40 byte packets
>>  1  static.33.203.4.46.clients.your-server.de (46.4.203.33)  0.682 ms
>> 0.531 ms  0.481 ms
>>  2  hos-tr4.juniper2.rz13.hetzner.de (213.239.224.97)  0.245 ms
>> hos-tr3.juniper2.rz13.hetzner.de (213.239.224.65)  0.242 ms
>> hos-tr2.juniper1.rz13.hetzner.de (213.239.224.33)  0.241 ms
>>  3  hos-bb2.juniper4.ffm.hetzner.de (213.239.240.150)  5.951 ms
>> hos-bb1.juniper1.ffm.hetzner.de (213.239.240.224)  4.803 ms  4.786 ms
>>  4  30gigabitethernet4-3.core1.fra1.he.net (80.81.192.172)  6.354 ms
>> 6.81 ms  5.451 ms
>>  5  10gigabitethernet10-2.core1.par2.he.net (72.52.92.26)  22.803 ms
>> 24.531 ms  24.787 ms
>>  6  10gigabitethernet15-1.core1.ash1.he.net (184.105.213.93)  101.504
>> ms  99.563 ms  99.958 ms
>>  7  10gigabitethernet11-1.core1.pao1.he.net (184.105.213.177)  163.687
>> ms  171.711 ms  175.34 ms
>>  8  10gigabitethernet1-2.core1.fmt1.he.net (184.105.213.65)  163.940
>> ms  171.362 ms  167.67 ms
>>  9  ns1.he.net (216.218.130.2)  163.52 ms  165.265 ms  164.143 ms
>>
>> C.
>>
>> On 12 June 2013 15:11, Constantine A. Murenin <mureninc at gmail.com> wrote:
>> > he.net fra1 tserv1 is down since about 10 minutes ago (~14:57 PT),
>> > this time it seems like the whole thing is down, cannot even ping
>> > ordns.he.net over IPv6; a connection of my friend who's running a
>> > smokeping is also down, e.g. this is definitely widespread.
>> >
>> > Not sure what's exactly down, but it seems to be bgp-related, perhaps:
>> >
>> >  1  2600:3c01::8678:acff:fe0d:79c1 (2600:3c01::8678:acff:fe0d:79c1)
>> > 0.699 ms  0.828 ms  0.970 ms
>> >  2  10gigabitethernet7-6.core3.fmt2.he.net (2001:470:1:3b8::1)  5.936
>> > ms  5.877 ms  5.859 ms
>> >  3  10gigabitethernet5-4.core1.pao1.he.net (2001:470:0:263::2)  6.743
>> > ms  6.877 ms  6.978 ms
>> >  4  * * *
>> >  5  * * *
>> >  6  * * *
>> >  7  * * *
>> >  8  * * *
>> >  9  * * *
>> > 10  * * *
>> > 11  * * *
>> > 12  * * *
>> > 13  * * *
>> > 14  * * *
>> > 15  * * *
>> > 16  * * *
>> > 17  * * *
>> > 18  * * *
>> >
>> > C.