[Outages-discussion] CenturyLink 10G Waves Down Between PA and FL

[Darren S.]

On Fri, May 31, 2019 at 5:36 PM Scott Weeks <surfer at mauigateway.com> wrote:
> Root Cause: A loose connector on a local provider's network
> between Atlanta, GA and College Park, GA was impacting
> unprotected services.
>
> Fix Action: The connection was secured, and the equipment
> rebooted, thus restoring services.
> -----------------------------------------------
>
>
> Rebooted because of a 'loose connector'?  WTF???

I'm not surprised by anything anymore. I have an Unbound DNS resolver
on a network I support and when my upstream router loses link to ISP,
unbound eventually faults and stops serving DNS queries until daemon
is restarted. No sound reason that should happen.

If the loose connector causes equipment to fault in some way and
require re-initialization (firmware defect?) I wouldn't be shocked. I
feel like product quality isn't increasingly linearly with time, it's
almost moving the other way it seems like. Look how terrible modern
mobile and IoT platforms are. Logic tells us with so much history
behind us and lessons learned, things should be more robust than ever.

- Darren