<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Brush Script MT";
panose-1:3 6 8 2 4 4 6 7 3 4;}
@font-face
{font-family:Times;
panose-1:2 2 6 3 5 4 5 2 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Another good article on the topic:<o:p></o:p></p><p class=MsoNormal><a href="https://www.datacenterdynamics.com/news/gcp-outage-mainone-leaked-google-cloudflare-ip-addresses-china-telecom/">https://www.datacenterdynamics.com/news/gcp-outage-mainone-leaked-google-cloudflare-ip-addresses-china-telecom/</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Frank <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> Outages-discussion <outages-discussion-bounces@outages.org> <b>On Behalf Of </b>frnkblk@iname.com<br><b>Sent:</b> Tuesday, November 13, 2018 11:28 PM<br><b>To:</b> outages-discussion@outages.org<br><b>Subject:</b> Re: [Outages-discussion] [outages] Problems With Google?<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Here’s QRator’s blog entry on this topic:<o:p></o:p></p><p class=MsoNormal><a href="https://radar.qrator.net/blog/mistake-mistake-blackhole">https://radar.qrator.net/blog/mistake-mistake-blackhole</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Frank <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> Outages-discussion <<a href="mailto:outages-discussion-bounces@outages.org">outages-discussion-bounces@outages.org</a>> <b>On Behalf Of </b><a href="mailto:frnkblk@iname.com">frnkblk@iname.com</a><br><b>Sent:</b> Monday, November 12, 2018 10:29 PM<br><b>To:</b> <a href="mailto:outages-discussion@outages.org">outages-discussion@outages.org</a><br><b>Subject:</b> Re: [Outages-discussion] [outages] Problems With Google?<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><a href="https://blog.thousandeyes.com/internet-vulnerability-takes-down-google/">https://blog.thousandeyes.com/internet-vulnerability-takes-down-google/</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Good blog entry on the event. Was hoping to see Dyn (now Oracle) or BGPmon (OpenDNS => now Cisco) and QRrator to chime in, but haven’t seen anything from them, yet.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Frank <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><b>From:</b> Outages <<a href="mailto:outages-bounces@outages.org">outages-bounces@outages.org</a>> <b>On Behalf Of </b>Chris Boulton via Outages<br><b>Sent:</b> Monday, November 12, 2018 4:17 PM<br><b>To:</b> <a href="mailto:josh@imaginenetworksllc.com">josh@imaginenetworksllc.com</a><br><b>Cc:</b> outages <<a href="mailto:outages@outages.org">outages@outages.org</a>><br><b>Subject:</b> Re: [outages] Problems With Google?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div><div><p class=MsoNormal>It sounds like Google are aware of this, and working on getting some messaging out (at least for Google Cloud customers)<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Probably related: <a href="https://twitter.com/thousandeyes/status/1062102171506765825">https://twitter.com/thousandeyes/status/1062102171506765825</a><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><i>"BREAKING: Potential hijack underway. ThousandEyes detected intermittent availability issues to Google services from some locations. Traffic to certain Google destinations appears to be routed through an ISP in Russia & black-holed at a China Telecom gateway router."</i><o:p></o:p></p></div></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Mon, Nov 12, 2018 at 2:15 PM Josh Luthman via Outages <<a href="mailto:outages@outages.org">outages@outages.org</a>> wrote:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><div><div><div><p class=MsoNormal>Google Music and Youtube won't load the content for me, but I can load the HTML of the pages...<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I can ping <a href="http://google.com" target="_blank">google.com</a> - 1 5 ms 1 ms 1 ms 10.10.10.1<o:p></o:p></p></div><div><p class=MsoNormal> 2 1 ms 3 ms 1 ms <a href="http://fwbb-192-149-254-1.inxnet.net" target="_blank">fwbb-192-149-254-1.inxnet.net</a> [192.149.254.1]<o:p></o:p></p></div><div><p class=MsoNormal> 3 4 ms 1 ms 3 ms <a href="http://fwb-74-218-88-129.inxnet.net" target="_blank">fwb-74-218-88-129.inxnet.net</a> [74.218.88.129]<o:p></o:p></p></div><div><p class=MsoNormal> 4 6 ms 7 ms 9 ms <a href="http://troyoh0901h.mw.twcbiz.com" target="_blank">troyoh0901h.mw.twcbiz.com</a> [65.189.190.189]<o:p></o:p></p></div><div><p class=MsoNormal> 5 23 ms 21 ms 22 ms <a href="http://be39.dytnoh5501r.midwest.rr.com" target="_blank">be39.dytnoh5501r.midwest.rr.com</a> [65.29.38.112]<o:p></o:p></p></div><div><p class=MsoNormal> 6 8 ms 13 ms 14 ms <a href="http://be10.10.blasohdp01r.midwest.rr.com" target="_blank">be10.10.blasohdp01r.midwest.rr.com</a> [65.29.1.48]<o:p></o:p></p></div><div><p class=MsoNormal> 7 9 ms 11 ms 13 ms <a href="http://be28.clmkohpe01r.midwest.rr.com" target="_blank">be28.clmkohpe01r.midwest.rr.com</a> [65.29.1.44]<o:p></o:p></p></div><div><p class=MsoNormal> 8 31 ms 29 ms 29 ms <a href="http://bu-ether15.chctilwc00w-bcr00.tbone.rr.com" target="_blank">bu-ether15.chctilwc00w-bcr00.tbone.rr.com</a> [66.109.6.68]<o:p></o:p></p></div><div><p class=MsoNormal> 9 22 ms 22 ms 28 ms <a href="http://bu-ether11.chcgildt87w-bcr00.tbone.rr.com" target="_blank">bu-ether11.chcgildt87w-bcr00.tbone.rr.com</a> [66.109.6.20]<o:p></o:p></p></div><div><p class=MsoNormal> 10 18 ms 19 ms 70 ms <a href="http://0.ae2.pr1.chi10.tbone.rr.com" target="_blank">0.ae2.pr1.chi10.tbone.rr.com</a> [107.14.17.196]<o:p></o:p></p></div><div><p class=MsoNormal> 11 19 ms 17 ms 18 ms <a href="http://ix-ae-27-0.tcore2.ct8-chicago.as6453.net" target="_blank">ix-ae-27-0.tcore2.ct8-chicago.as6453.net</a> [64.86.79.97]<o:p></o:p></p></div><div><p class=MsoNormal> 12 29 ms 29 ms 31 ms <a href="http://if-ae-22-2.tcore1.ct8-chicago.as6453.net" target="_blank">if-ae-22-2.tcore1.ct8-chicago.as6453.net</a> [64.86.79.2]<o:p></o:p></p></div><div><p class=MsoNormal> 13 44 ms 29 ms 32 ms <a href="http://if-ae-26-2.tcore2.nto-new-york.as6453.net" target="_blank">if-ae-26-2.tcore2.nto-new-york.as6453.net</a> [216.6.81.28]<o:p></o:p></p></div><div><p class=MsoNormal> 14 30 ms 28 ms 28 ms <a href="http://if-ae-12-2.tcore1.n75-new-york.as6453.net" target="_blank">if-ae-12-2.tcore1.n75-new-york.as6453.net</a> [66.110.96.5]<o:p></o:p></p></div><div><p class=MsoNormal> 15 31 ms 29 ms 29 ms 72.14.195.232<o:p></o:p></p></div><div><p class=MsoNormal> 16 * * * Request timed out.<o:p></o:p></p></div><div><p class=MsoNormal> 17 31 ms 31 ms 31 ms 108.170.229.254<o:p></o:p></p></div><div><p class=MsoNormal> 18 29 ms 29 ms 29 ms 108.170.248.20<o:p></o:p></p></div><div><p class=MsoNormal> 19 33 ms 31 ms 30 ms 216.239.56.17<o:p></o:p></p></div><div><p class=MsoNormal> 20 36 ms 36 ms 35 ms 216.239.57.196<o:p></o:p></p></div><div><p class=MsoNormal> 21 36 ms 36 ms 36 ms 209.85.250.147<o:p></o:p></p></div><div><p class=MsoNormal> 22 35 ms 35 ms 35 ms 108.170.244.1<o:p></o:p></p></div><div><p class=MsoNormal> 23 35 ms 35 ms 38 ms 209.85.249.99<o:p></o:p></p></div><div><p class=MsoNormal> 24 38 ms 38 ms 38 ms <a href="http://ord31s21-in-f206.1e100.net" target="_blank">ord31s21-in-f206.1e100.net</a> [216.58.216.206]<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>There's a recent spike for Youtube, Google, Spotify, Nest, Snapchat, Gmail, Discord, etc... <a href="https://downdetector.com" target="_blank">https://downdetector.com</a><o:p></o:p></p></div></div></div></div><div><p class=MsoNormal><br clear=all><o:p></o:p></p><div><div><div><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>Josh Luthman<br>Office: 937-552-2340<br>Direct: 937-552-2343<br>1100 Wayne St<br>Suite 1337<br>Troy, OH 45373<o:p></o:p></p></div></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Mon, Nov 12, 2018 at 5:07 PM, Eduard via Outages <<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>From sacramento, cannot ping or access <a href="http://google.com" target="_blank">google.com</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.0pt;color:#18376A'>Regards,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:34.5pt;font-family:"Brush Script MT";color:#18376A'>Eduard Tieseler</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='color:#6D6D6D'>Sr. Network / Systems Administrator</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='color:#6D6D6D'>1164 West National Drive #60</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='color:#6D6D6D'>Sacramento, CA 95834</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='color:#6D6D6D'>Office 916-922-7584 ext. 288</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='color:#6D6D6D'>Fax 916-922-1835</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.5pt'><a href="mailto:etieseler@metrolist.net" target="_blank"><span style='font-size:12.0pt;font-family:Times;color:#18376A'>etieseler@metrolist.net</span></a></span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.5pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.5pt;color:blue'><img border=0 width=139 height=48 style='width:1.4479in;height:.5in' id="_x0000_i1025" src="cid:image001.png@01D47A91.0F5BFB80" alt="/Users/Eduard/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_756775869"></span><span style='font-size:14.5pt;color:#18376A'> </span><a href="https://www.facebook.com/metrolistmls" target="_blank"><span style='font-size:14.0pt;text-decoration:none'><img border=0 width=24 height=26 style='width:.25in;height:.2708in' id="_x0000_i1026" src="cid:image002.png@01D47A91.0F5BFB80" alt="/Users/Eduard/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1238741448"></span></a><a href="https://www.youtube.com/channel/UCmhaqKw5Kv2TBJ-gohS8TvQ" target="_blank"><span style='font-size:14.5pt;text-decoration:none'><img border=0 width=29 height=26 style='width:.302in;height:.2708in' id="_x0000_i1027" src="cid:image003.png@01D47A91.0F5BFB80" alt="/Users/Eduard/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_87266948"></span></a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:14.5pt;color:#18376A'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-autospace:none'><span style='font-size:10.5pt;color:#64822E'>Please consider the environment before printing this e-mail</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:8.5pt;color:#A6A6A6'>THE INFORMATION CONTAINED IN THIS ELECTRONIC MESSAGE IS CONFIDENTIAL. THE INFORMATION IS INTENDED ONLY FOR THE USE OF THE INDIVIDUAL OR ENTITY TO WHOM IT IS ADDRESSED. IF YOU ARE NOT THE INTENDED RECIPIENT, ANY USE, DISSEMINATION, OR DISTRIBUTION OF THIS COMMUNICATION IS PROHIBITED. IF YOU HAVE RECEIVED THIS ELECTRONIC MESSAGE IN ERROR, PLEASE NOTIFY US IMMEDIATELY AND DELETE THE MESSAGE. ANY USE, MODIFICATION, OR REPUBLICATION OF THIS COMMUNICATION, INCLUDING ANY ATTACHED FILES, DOCUMENTS, DATA OR OTHER INFORMATION WHICH HAS NOT BEEN EXPRESSLY AUTHORIZED BY US IS PROHIBITED. WE SPECIFICALLY DISCLAIM RESPONSIBILITY FOR ANY UNAUTHORIZED USE OF THIS COMMUNICATION OR ANY ATTACHMENTS TO IT.</span><o:p></o:p></p></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>_______________________________________________<br>Outages mailing list<br><a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><o:p></o:p></p></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>_______________________________________________<br>Outages mailing list<br><a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><o:p></o:p></p></blockquote></div><p class=MsoNormal><br clear=all><o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>-- <o:p></o:p></p><div><div><div><div><div><div><div><div><div><div><p class=MsoNormal>Regards,<br><br>Chris Boulton<o:p></o:p></p></div><div><p class=MsoNormal>Director, Infrastructure Engineering<br>BigCommerce<br><br>Email: <a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a><o:p></o:p></p></div><div><p class=MsoNormal>Web: <a href="http://www.bigcommerce.com/" target="_blank">http://www.bigcommerce.com/</a><o:p></o:p></p></div></div></div></div></div></div></div></div></div></div></div></div></body></html>