<span style="font-family: Arial, Helvetica, Sans-Serif; font-size: 12px"><div>In my experience. It breaks DNS Masquerading. So any DNS query sent to the router goes unanswered. Verified this with a customer by having them hard-set their Local DNS (On their computer) to my public resolvers.</div>
<div> </div>
<div>I suspect anything that didn't require DNS worked fine (RDP inbound..SIP Outbound once it resolved the SBC Address. Assuming it had a DNS Hostname).</div>
<div> </div>
<div>The current thought (Around the NOC here) is that DNS is breaking due to another user friendly feature. Where the router forwards you to it's config page automatically, if it thinks you have no internet..</div>
<div> </div>
<div>Nick Olsen<br />
<span style="font-size: 10pt; font-family: arial,helvetica,sans-serif;">Network Operations</span>
<div>(855) FLSPEED x106<br />
<br />
<img alt="" height="56" src="http://www.flhsi.com/files/emaillogo.jpg" width="168" /></div>
</div>
<div> </div>
<hr align="center" size="2" width="100%" />
<div><span style="font-family: tahoma,arial,sans-serif; font-size: 10pt;"><b>From</b>: "John Neiberger via Outages" <outages@outages.org><br />
<b>Sent</b>: Tuesday, October 07, 2014 2:32 PM<br />
<b>To</b>: "Frank Bulk" <frnkblk@iname.com><br />
<b>Cc</b>: "outages@outages.org" <outages@outages.org><br />
<b>Subject</b>: Re: [outages] Belkin [heartbeat.belkin.com]</span>
<div> </div>
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">I just saw a report from another ISP who says they had a customer whose VoIP continued to work and they were able to RDP into their computer from the outside. It appears that only web traffic was broken. I wonder if parental controls or web filtering are a component in this fiasco.</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"> </div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">John</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">On Tue, Oct 7, 2014 at 12:04 PM, Frank Bulk <span dir="ltr"><<a href="mailto:frnkblk@iname.com" target="_blank">frnkblk@iname.com</a>></span> wrote:
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">No, we’re still getting customer reports about the light staying orange. I was just hoping they were having other issues. </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">I’ve just added 67.20.176.130 as a loopback in our network, too.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">Frank</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> John Neiberger [mailto:<a href="mailto:jneiberger@gmail.com" target="_blank">jneiberger@gmail.com</a>]<br />
<b>Sent:</b> Tuesday, October 07, 2014 1:01 PM<br />
<b>To:</b> Frank Bulk<br />
<b>Cc:</b> <a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a></span></p>
<div>
<div class="h5"><br />
<b>Subject:</b> Re: [outages] Belkin [<a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a>]</div>
</div>
<p> </p>
<div>
<div class="h5">
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">Given the number of current social media postings, it doesn't appear that they are back up yet, but I suppose it is hard to say for sure given the number of people affected. Twitter is all aflutter with new reports of people still having problems. Yours is the only report that indicated they were working again. It could just be that it's taking a while for the devices to correct themselves, or maybe they even need to be restarted or something like that.</span></p>
</div>
</div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Tue, Oct 7, 2014 at 11:57 AM, Frank Bulk <<a href="mailto:frnkblk@iname.com" target="_blank">frnkblk@iname.com</a>> wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">Why do that if Belkin is up and running again? Or are you suggesting that Belkin doesn’t have it all worked out?</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">Frank</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Outages [mailto:<a href="mailto:outages-bounces@outages.org" target="_blank">outages-bounces@outages.org</a>] <b>On Behalf Of </b>John Neiberger via Outages<br />
<b>Sent:</b> Tuesday, October 07, 2014 12:42 PM<br />
<b>To:</b> <a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a></span></p>
<div>
<div>
<p class="MsoNormal"><br />
<b>Subject:</b> Re: [outages] Belkin [<a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a>]</p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">Has anyone else tried the loopback/heartbeat workaround with success? I've seen a few reports so far that this has temporarily solved the problem.</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">John</span></p>
</div>
</div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Tue, Oct 7, 2014 at 10:28 AM, Frank Bulk <<a href="mailto:frnkblk@iname.com" target="_blank">frnkblk@iname.com</a>> wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">I can ping <a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a> and the DSL Reports link and other connected links are stating it was back up roughly 30 to 45 minutes ago, so I think we’re out of the woods.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a">Frank</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#44546a"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Outages [mailto:<a href="mailto:outages-bounces@outages.org" target="_blank">outages-bounces@outages.org</a>] <b>On Behalf Of </b>John Neiberger via Outages<br />
<b>Sent:</b> Tuesday, October 07, 2014 11:04 AM<br />
<b>To:</b> Josh Luthman<br />
<b>Cc:</b> <a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>; John Barbieri<br />
<b>Subject:</b> Re: [outages] Belkin [<a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a>]</span></p>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">I think it was initially suspected to be a firmware upgrade, but I doubt that's the case now. It's interesting to me that there are two seemingly unrelated fixes:</span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">1. Put the IP for <a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a> on a loopback </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">2. Manually add working DNS entries to affected PCs</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">Reports indicate that either fix works, but why? They seem unrelated to one another unless the failing heartbeat messages mess up DNS handling on the router.</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Tahoma","sans-serif"">John</span></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Tue, Oct 7, 2014 at 9:48 AM, Josh Luthman via Outages <<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Where? I keep reading "bad firmware" problems but everything suggests there is no automatic upgrade and I think we can safely assume resi users didn't upgrade their routers over night.</p>
</div>
<div>
<p class="MsoNormal"> </p>
<div>
<div>
<div>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal"><br />
Josh Luthman<br />
Office: <a href="tel:937-552-2340" target="_blank">937-552-2340</a><br />
Direct: <a href="tel:937-552-2343" target="_blank">937-552-2343</a><br />
1100 Wayne St<br />
Suite 1337<br />
Troy, OH 45373</p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Oct 7, 2014 at 11:35 AM, John Barbieri via Outages <<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">Funny,<br />
<br />
Just saw something about Belkin routers and a botched firmware upgrade.<br />
<br />
I wonder if this is what they meant.<br />
<br />
That being said, it is working fine from Zito Media in Clanton, AL.<br />
<br />
--John</p>
<div>
<div>
<p class="MsoNormal"><br />
On 10/07/2014 10:17 AM, Paul Miller via Outages wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">On one of my netblocks my subscribers can nolonger ping Belkin. I can ping<br />
<a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a> from my webservers and from subscriber units on a<br />
different netblock.<br />
<br />
Apparently Belkin routers ping the home base every couple seconds to determine<br />
the correct color of the Internet light on the router (orange or blue *sigh*).<br />
Worse, when it can't ping the Belkin site the router seems to pack up and<br />
redirect my subscribers to their router configuration page.<br />
<br />
I found this pretty easy to work around. I simply DNATed the pings intended<br />
for <a href="http://heartbeat.belkin.com" target="_blank">heartbeat.belkin.com</a> to one of my local nameservers and that seems to get<br />
them back online. It's a silly workaround, but if anyone else is seeing all<br />
the Belkin routers on their network pack up, this could be why.<br />
<br />
-Paul<br />
<br />
_______________________________________________<br />
Outages mailing list<br />
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br />
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></p>
</blockquote>
<p class="MsoNormal"><br />
_______________________________________________<br />
Outages mailing list<br />
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br />
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br />
_______________________________________________<br />
Outages mailing list<br />
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br />
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></p>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div></span>