<div dir="ltr"><div><div>VoIP Innovations DDoS (internal ENG notes):</div><div><br></div><div>One of our customers has used VI since at least 2009, for specific (unique) types of ingress traffic for which it was not prudent [cost-effective] for us to on-ramp to our national TDM network, i.e., select rural off-net markets. </div><div><br></div><div>They've had their ups and downs with VI, but, for the most part, VI's progressively increased in reliability...until the recent DDoS events.</div><div><br></div><div>That said - when the DDoS issues started popping up, we were left with the same intrigue as most other VI consumers on this list. Is this really a DDoS style issue/attack? If so - what kind of DDoS issue, what can we do to help, if anything? And , if not - what's really going on and how quickly can we "jump ship"...</div><div><br></div><div>Interestingly, and cautiously, without pushing out any more information than what is already known to the public, or is publically accessible, we can confirm that they are, in fact, and as of late, using <b><font color="#ff0000">NeuStar's DDoS mitigation services</font></b>.<b> It also appears, based on BGP data, that the latest maintenance window was to integrate fully such systems.</b></div><div><br></div><div>Our engineering team created a pretty intriguing video for an internal update on the issue, showing the AS paths of their primary and secondary SBC's, which, if you follow the timeline and the ending ASN's, at least, in our opinion, show VI's commitment level.</div><div><br></div><div>That said - kudos to the VI team, and as frustrated as our team has been, the last thought that always comes to mind is "it could have any of us".</div><div><br></div><div><a href="http://cdn.enguity.com/vid/VoIPInnovations-DDoS-Mitigation-2016.mp4">http://cdn.enguity.com/vid/VoIPInnovations-DDoS-Mitigation-2016.mp4</a><br></div><div><br></div><div>...just a thought</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 16, 2016 at 12:30 PM, Tim Linn <span dir="ltr"><<a href="mailto:timothyl@voipinnovations.com" target="_blank">timothyl@voipinnovations.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<p>All,<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>I can confirm that we did have 2 outages this morning spanning little over an hour due to another DDoS attack.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>We have been careful of the information that we have been giving out in public updates due to the possibility of the attacker reading that information and using it against us in some way.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>The biggest change that we’ve made to combat this is a DDoS protection service, which has caught a handful of attacks, but was largely useless in this latest attack. We’ve made mitigating changes to our existing setup which appears to
have stopped today’s attack so far.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>We do have a disaster recovery site and have, honestly, appear to have done a poor job communicating that to our customers. If you are a Voip Innovations customer, please contact me for the IP address of the disaster recovery site. During
these outages, the site was taking origination calls from most of our DID vendors. We’re currently working on figuring out why all of them did not switch over.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>I know that words aren’t much right now, but I am truly sorry for the issues that you have been having. I understand the importance of reliability to you and your customers, and I know that those expectations have not been met over the
last few weeks. We are working diligently with several different companies to continue to make changes to our network to mitigate these attacks. The changes have been successful for a number of the attacks that we have seen, but it’s clear that we are still
vulnerable and have work to do. <u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Please contact me if you have any questions, and I will do my best to answer.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p><u></u> <u></u></p>
<p class="MsoNormal"><b><span style="color:rgb(31,73,125)">Timothy Linn<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="color:rgb(31,73,125)">Lead Systems Engineer<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="color:rgb(31,73,125)">Voip Innovations<u></u><u></u></span></b></p>
<p><u></u> <u></u></p>
<p><span style="color:black"><u></u> <u></u></span></p>
<p>----------------------------------------------------------------------<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Message: 1<u></u><u></u></p>
<p>Date: Wed, 16 Mar 2016 11:24:04 -0400<u></u><u></u></p>
<p>From: Jeff Waddell <<a href="mailto:jeff+voiceops@waddellsolutions.com" target="_blank"><span style="color:windowtext;text-decoration:none">jeff+voiceops@waddellsolutions.com</span></a>><u></u><u></u></p>
<p>To: Nathan Anderson <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>><u></u><u></u></p>
<p>Cc: Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>>, "<a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a>"<u></u><u></u></p><span class="">
<p> <<a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a>><u></u><u></u></p>
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
</span><p>Message-ID:<u></u><u></u></p>
<p> <<a href="mailto:CAO+x49LG2+RQyRU6a0zmgdgaAKd0yYyxASHm6+=9z97iDxjMqA@mail.gmail.com" target="_blank"><span style="color:windowtext;text-decoration:none">CAO+x49LG2+RQyRU6a0zmgdgaAKd0yYyxASHm6+=9z97iDxjMqA@mail.gmail.com</span></a>><u></u><u></u></p>
<p>Content-Type: text/plain; charset="utf-8"<u></u><u></u></p><span class="">
<p><u></u> <u></u></p>
<p>Not that it helps a lot - this was posted on the VI facebook page<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>"[Update] As of this morning, we regret to inform you that we are again currently experiencing issues that are related to the DDoS attack being committed by unknown parties against our network. Our Network Engineers are currently working
diligently with our upstream carriers as well as a network security firm to combat the attack and will continue to do so until all of the issues are resolved. Our Senior Leadership has also been in contact with the FBI. The FBI is considering these events
a national security issue due to the number of firms impacted, the magnitude of the attacks, and the persistence of these attacks. The Social Media Team will post here the very moment we have further information."<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>On Wed, Mar 16, 2016 at 10:29 AM, Nathan Anderson <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>> wrote:<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>> Down again. **** me.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
</span><p>> *From:* VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org</span></a>] *On Behalf Of
<u></u><u></u></p>
<p>> *Nate Burke<u></u><u></u></p>
<p>> *Sent:* Wednesday, March 16, 2016 6:51 AM<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> *To:* <a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> *Subject:* Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p><span class="">
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> Looks like it just came back up for me. Just over 30 min.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Nate<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> On 3/16/2016 8:45 AM, Shripal Daphtary wrote:<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> We are experiencing an outage as well.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Thanks,<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> Shripal<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> On Mar 16, 2016, at 9:36 AM, Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>> wrote:<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Problems again this morning? Looks to be acting the same as it has been.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> On 3/11/2016 6:00 PM, Alexander Lopez wrote:<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> I added them to our monitoring platform, stated getting alarms this
<u></u><u></u></p>
<p>> past hour or so.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Up and down.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> -------- Original message --------<u></u><u></u></p>
</span><p>> From: Nathan Anderson <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>> <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>><u></u><u></u></p><span class="">
<p>> Date: 3/11/2016 6:31 PM (GMT-05:00)<u></u><u></u></p>
</span><p>> To: 'Nate Burke' <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>> <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>>,
<u></u><u></u></p><div><div class="h5">
<p>> <a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> ...aaaaaand we're back.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -----Original Message-----<u></u><u></u></p>
<p>> From: VoiceOps [mailto:<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">voiceops-bounces@voiceops.org</a><u></u><u></u></p>
<p>> <<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops-bounces@voiceops.org</span></a>>] On Behalf Of Nathan Anderson<u></u><u></u></p>
<p>> Sent: Friday, March 11, 2016 3:30 PM<u></u><u></u></p>
<p>> To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> It *feels* like they are under attack again, since I get a response to
<u></u><u></u></p>
<p>> a ping once every 20 or so.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -----Original Message-----<u></u><u></u></p>
<p>> From: VoiceOps [mailto:<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">voiceops-bounces@voiceops.org</a><u></u><u></u></p>
<p>> <<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops-bounces@voiceops.org</span></a>>] On Behalf Of Nathan Anderson<u></u><u></u></p>
<p>> Sent: Friday, March 11, 2016 3:28 PM<u></u><u></u></p>
<p>> To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Confirmed.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -----Original Message-----<u></u><u></u></p>
<p>> From: VoiceOps [mailto:<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">voiceops-bounces@voiceops.org</a><u></u><u></u></p>
<p>> <<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops-bounces@voiceops.org</span></a>>] On Behalf Of Nate Burke<u></u><u></u></p>
<p>> Sent: Friday, March 11, 2016 3:26 PM<u></u><u></u></p>
<p>> To: <a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Anyone else show them down again right now? My traceroutes aren't
<u></u><u></u></p>
<p>> even leaving Chicago. Dying at a Chicago hop on Level3.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Nate<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> On 3/6/2016 6:50 PM, Nathan Anderson wrote:<u></u><u></u></p>
<p>> > Did anybody else just suffer another 45-minute-ish long outage from<u></u><u></u></p>
<p>> about 4:00p PST to 4:45p PST (ending about 5 minutes ago)?<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > -- Nathan<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > -----Original Message-----<u></u><u></u></p>
<p>> > From: <a href="mailto:frnkblk@iname.com" target="_blank"><span style="color:windowtext;text-decoration:none">frnkblk@iname.com</span></a> [mailto:<a href="mailto:frnkblk@iname.com" target="_blank">frnkblk@iname.com</a>
<u></u><u></u></p>
<p>> > <<a href="mailto:frnkblk@iname.com" target="_blank"><span style="color:windowtext;text-decoration:none">frnkblk@iname.com</span></a>>]<u></u><u></u></p>
<p>> > Sent: Sunday, March 06, 2016 4:43 PM<u></u><u></u></p>
<p>> > To: Nathan Anderson; <a href="mailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> > Subject: RE: VoIP Innovations reliability<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > More here: <a href="http://blog.voipinnovations.com/blog" target="_blank">
<span style="color:windowtext;text-decoration:none">http://blog.voipinnovations.com/blog</span></a><u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > Frank<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > -----Original Message-----<u></u><u></u></p>
<p>> > From: VoiceOps [mailto:<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">voiceops-bounces@voiceops.org</a><u></u><u></u></p>
<p>> <<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops-bounces@voiceops.org</span></a>>] On Behalf Of Nathan<u></u><u></u></p>
<p>> > Anderson<u></u><u></u></p>
<p>> > Sent: Tuesday, March 01, 2016 8:27 PM<u></u><u></u></p>
<p>> > To: <a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>> > Subject: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > Holy schlamoly. Anybody else use them here and being handed outage
<u></u><u></u></p>
<p>> > after outage over the last 2 days? Seriously thinking at this point
<u></u><u></u></p>
<p>> > about<u></u><u></u></p>
<p>> doing<u></u><u></u></p>
<p>> > something else. This is ridiculous.<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > I desperately need sleep and if my cell goes off one more time...<u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > -- Nathan<u></u><u></u></p>
<p>> > _______________________________________________<u></u><u></u></p>
<p>> > VoiceOps mailing list<u></u><u></u></p>
<p>> > <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> > <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> ><u></u><u></u></p>
<p>> > _______________________________________________<u></u><u></u></p>
<p>> > VoiceOps mailing list<u></u><u></u></p>
<p>> > <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> > <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
<p>> <a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
</div></div><p>-------------- next part --------------<u></u><u></u></p>
<p>An HTML attachment was scrubbed...<u></u><u></u></p>
<p>URL: <<a href="https://puck.nether.net/pipermail/voiceops/attachments/20160316/64ce223c/attachment-0001.html" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/pipermail/voiceops/attachments/20160316/64ce223c/attachment-0001.html</span></a>><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>------------------------------<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Message: 2<u></u><u></u></p>
<p>Date: Wed, 16 Mar 2016 08:27:51 -0700<u></u><u></u></p>
<p>From: Nathan Anderson <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>><u></u><u></u></p>
<p>To: 'Jeff Waddell' <<a href="mailto:jeff+voiceops@waddellsolutions.com" target="_blank"><span style="color:windowtext;text-decoration:none">jeff+voiceops@waddellsolutions.com</span></a>><u></u><u></u></p>
<p>Cc: Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>>, "<a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a>"<u></u><u></u></p><span class="">
<p> <<a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a>><u></u><u></u></p>
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
</span><p>Message-ID:<u></u><u></u></p>
<p> <<a href="mailto:C12CA40900F4E0448A2D4E66DFA6A59B231A8686F1@Demekin.FSI.local" target="_blank"><span style="color:windowtext;text-decoration:none">C12CA40900F4E0448A2D4E66DFA6A59B231A8686F1@Demekin.FSI.local</span></a>><u></u><u></u></p>
<p>Content-Type: text/plain; charset="utf-8"<u></u><u></u></p><span class="">
<p><u></u> <u></u></p>
<p>Are more firms than VI provenly being targeted, or are they talking about #s of firms being impacted because they are talking about their downstream customers and customers-of-customers?<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>This might sound weird and conspiracy-theory-ish, which is very unlike me (you can ask my friends), but..."Anonymous" vowed to target Trump, and VI started getting attacked the day before Super Tuesday. If I had to be honest, it *has*
made me wonder.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-- Nathan<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>From: <a href="mailto:jeff@waddellsystems.com" target="_blank"><span style="color:windowtext;text-decoration:none">jeff@waddellsystems.com</span></a> [<a href="mailto:jeff@waddellsystems.com" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:jeff@waddellsystems.com</span></a>]
On Behalf Of Jeff Waddell<u></u><u></u></p>
<p>Sent: Wednesday, March 16, 2016 8:24 AM<u></u><u></u></p>
<p>To: Nathan Anderson<u></u><u></u></p>
<p>Cc: Nate Burke; <a href="mailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org</span></a><u></u><u></u></p>
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Not that it helps a lot - this was posted on the VI facebook page<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>"[Update] As of this morning, we regret to inform you that we are again currently experiencing issues that are related to the DDoS attack being committed by unknown parties against our network. Our Network Engineers are currently working
diligently with our upstream carriers as well as a network security firm to combat the attack and will continue to do so until all of the issues are resolved. Our Senior Leadership has also been in contact with the FBI. The FBI is considering these events
a national security issue due to the number of firms impacted, the magnitude of the attacks, and the persistence of these attacks. The Social Media Team will post here the very moment we have further information."<u></u><u></u></p>
<p><u></u> <u></u></p>
</span><span class=""><p>On Wed, Mar 16, 2016 at 10:29 AM, Nathan Anderson <<a href="mailto:nathana@fsr.com%3cmailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com<mailto:nathana@fsr.com</span></a>>> wrote:<u></u><u></u></p>
<p>Down again. **** me.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-- Nathan<u></u><u></u></p>
<p><u></u> <u></u></p>
</span><p>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org%3cmailto:voiceops-bounces@voiceops.org%3e" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org<mailto:voiceops-bounces@voiceops.org></span></a>]
On Behalf Of Nate Burke<u></u><u></u></p><span class="">
<p>Sent: Wednesday, March 16, 2016 6:51 AM<u></u><u></u></p>
<p><u></u> <u></u></p>
</span><p>To: <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Looks like it just came back up for me. Just over 30 min.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Nate<u></u><u></u></p>
<p>On 3/16/2016 8:45 AM, Shripal Daphtary wrote:<u></u><u></u></p>
<p>We are experiencing an outage as well.<u></u><u></u></p>
<p>Thanks,<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Shripal<u></u><u></u></p>
<p><u></u> <u></u></p>
</span><span class=""><p>On Mar 16, 2016, at 9:36 AM, Nate Burke <<a href="mailto:nate@blastcomm.com%3cmailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com<mailto:nate@blastcomm.com</span></a>>> wrote:<u></u><u></u></p>
<p>Problems again this morning? Looks to be acting the same as it has been.<u></u><u></u></p>
<p>On 3/11/2016 6:00 PM, Alexander Lopez wrote:<u></u><u></u></p>
<p>I added them to our monitoring platform, stated getting alarms this past hour or so.<u></u><u></u></p>
<p>Up and down.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p><u></u> <u></u></p>
<p>-------- Original message --------<u></u><u></u></p>
</span><p>From: Nathan Anderson <<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">nathana@fsr.com</span></a>><<a href="mailto:nathana@fsr.com" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:nathana@fsr.com</span></a>><u></u><u></u></p><span class="">
<p>Date: 3/11/2016 6:31 PM (GMT-05:00)<u></u><u></u></p>
</span><p>To: 'Nate Burke' <<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">nate@blastcomm.com</span></a>><<a href="mailto:nate@blastcomm.com" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:nate@blastcomm.com</span></a>>,
<a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability ...aaaaaand we're back.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-- Nathan<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-----Original Message-----<u></u><u></u></p>
<p>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org</span></a>] On Behalf Of Nathan Anderson<u></u><u></u></p>
<p>Sent: Friday, March 11, 2016 3:30 PM<u></u><u></u></p>
</span><p>To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>It *feels* like they are under attack again, since I get a response to a ping once every 20 or so.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-- Nathan<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-----Original Message-----<u></u><u></u></p>
<p>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org</span></a>] On Behalf Of Nathan Anderson<u></u><u></u></p>
<p>Sent: Friday, March 11, 2016 3:28 PM<u></u><u></u></p>
</span><p>To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Confirmed.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-- Nathan<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-----Original Message-----<u></u><u></u></p>
<p>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org</span></a>] On Behalf Of Nate Burke<u></u><u></u></p>
<p>Sent: Friday, March 11, 2016 3:26 PM<u></u><u></u></p>
</span><p>To: <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>Subject: Re: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Anyone else show them down again right now? My traceroutes aren't even leaving Chicago. Dying at a Chicago hop on Level3.<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Nate<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>On 3/6/2016 6:50 PM, Nathan Anderson wrote:<u></u><u></u></p>
<p>> Did anybody else just suffer another 45-minute-ish long outage from about 4:00p PST to 4:45p PST (ending about 5 minutes ago)?<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -----Original Message-----<u></u><u></u></p>
<p>> From: <a href="mailto:frnkblk@iname.com%3cmailto:frnkblk@iname.com" target="_blank">
<span style="color:windowtext;text-decoration:none">frnkblk@iname.com<mailto:frnkblk@iname.com</span></a>>
<u></u><u></u></p>
<p>> [<a href="mailto:frnkblk@iname.com" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:frnkblk@iname.com</span></a>]<u></u><u></u></p>
<p>> Sent: Sunday, March 06, 2016 4:43 PM<u></u><u></u></p>
<p>> To: Nathan Anderson; <u></u><u></u></p>
</span><p>> <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>> Subject: RE: VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> More here: <a href="http://blog.voipinnovations.com/blog" target="_blank">
<span style="color:windowtext;text-decoration:none">http://blog.voipinnovations.com/blog</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Frank<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -----Original Message-----<u></u><u></u></p>
<p>> From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">mailto:voiceops-bounces@voiceops.org</span></a>] On Behalf Of
<u></u><u></u></p>
<p>> Nathan Anderson<u></u><u></u></p>
<p>> Sent: Tuesday, March 01, 2016 8:27 PM<u></u><u></u></p>
</span><p>> To: <a href="mailto:voiceops@voiceops.org%3cmailto:voiceops@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">voiceops@voiceops.org<mailto:voiceops@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>> Subject: [VoiceOps] VoIP Innovations reliability<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> Holy schlamoly. Anybody else use them here and being handed outage
<u></u><u></u></p>
<p>> after outage over the last 2 days? Seriously thinking at this point
<u></u><u></u></p>
<p>> about doing something else. This is ridiculous.<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> I desperately need sleep and if my cell goes off one more time...<u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>> -- Nathan<u></u><u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
</span><p>> <a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>><u></u> <u></u></p>
<p>> _______________________________________________<u></u><u></u></p>
<p>> VoiceOps mailing list<u></u><u></u></p>
</span><p>> <a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank">
<span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p>> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">
<span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p><span class="">
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
</span><p><a href="mailto:VoiceOps@voiceops.org%3cmailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org<mailto:VoiceOps@voiceops.org</span></a>><u></u><u></u></p>
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>-------------- next part --------------<u></u><u></u></p>
<p>An HTML attachment was scrubbed...<u></u><u></u></p>
<p>URL: <<a href="https://puck.nether.net/pipermail/voiceops/attachments/20160316/07612296/attachment.html" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/pipermail/voiceops/attachments/20160316/07612296/attachment.html</span></a>><u></u><u></u></p>
<p><u></u> <u></u></p>
<p>------------------------------<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>Subject: Digest Footer<u></u><u></u></p><span class="">
<p><u></u> <u></u></p>
<p>_______________________________________________<u></u><u></u></p>
<p>VoiceOps mailing list<u></u><u></u></p>
<p><a href="mailto:VoiceOps@voiceops.org" target="_blank"><span style="color:windowtext;text-decoration:none">VoiceOps@voiceops.org</span></a><u></u><u></u></p>
<p><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"><span style="color:windowtext;text-decoration:none">https://puck.nether.net/mailman/listinfo/voiceops</span></a><u></u><u></u></p>
<p><u></u> <u></u></p>
<p><u></u> <u></u></p>
</span><p>------------------------------<u></u><u></u></p>
<p><u></u> <u></u></p>
<p>End of VoiceOps Digest, Vol 81, Issue 15<u></u><u></u></p>
<p>****************************************<u></u><u></u></p>
</div>
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received this message in error, please notify the sender immediately and delete the original. Any other use of the
e-mail by you is prohibited. Unauthorized interception of this e-mail is a violation of federal criminal law. We reserve the right, when permitted by law, to scan electronic communications, including e-mail and instant messaging, for the purposes of security
and compliance with our internal policy.
</div>
<br>_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br></blockquote></div><br></div><img src="https://t.yesware.com/t/eb6d9d6b31412045cb763bde159eae39bedfa8b8/d60715f7a72d044284ff0bfdfb0f93ad/spacer.gif" style="border: 0px; width: 0px; height: 0px; overflow: hidden;" width="0" height="0"><img src="http://t.yesware.com/t/eb6d9d6b31412045cb763bde159eae39bedfa8b8/d60715f7a72d044284ff0bfdfb0f93ad/spacer.gif" style="border: 0px; width: 0px; height: 0px; overflow: hidden;" width="0" height="0"><font face="yw-eb6d9d6b31412045cb763bde159eae39bedfa8b8-d60715f7a72d044284ff0bfdfb0f93ad--to" style="display: none;"></font></div></div>