<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta name="Generator" content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang="EN-CA" link="blue" vlink="purple"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Ahem… <stands on the soap box> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">At a risk of this not being a popular point of view… I view issues with VoIP Innovations as being largely self-inflicted. </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Not only on part of VoIP Innovations (no relationship) who, as it appears, could have designed and communicated the DR better; but also on part of people complaining about the outage. You are using a non-guaranteed service. No SLAs, no promises, no commitments. If stuff like this happens, it is only because it is to be expected. No one can guarantee the quality of Internet between your IP and most other IPs on the Internet. That’s the reality. And if you are relying on a non-guaranteed service, you should at least have contingencies to move traffic quickly (CIC TFNs) or reroute (outbound). </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">And where that’s not possible (DIDs), having a dedicated data link covered by SLAs may be more appropriate. </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">And if you are porting your services to another provider with whom you will also interconnect over the Internet, if I was your client, I would be looking elsewhere. Unless your clients are not looking for a guaranteed 100% (or 99% or whatever) uptime. In which case, staying with VoIP Innovations is just fine… </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">And I know the response will likely be that it isn’t economically feasible to get dedicated connectivity, but by making that argument you are just saying that your business (and your clients’ business) doesn’t require a guaranteed level of uptime. </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">And either your clients have made this decision knowingly, in exchange for cost savings (in which case you are still delivering what you promised) or you made this decision on behalf of your clients unbeknownst to them, in which case you are up a creek… because this is either going to happen again and you will have to justify yet again why their services are down, or you’ll have to pony up the cost of a dedicated connection without being to recover it from your’ clients’ service fees. Either way, it wasn’t a good idea in the first place and it isn’t VoIP Innovation’s fault, unless they mislead you by saying Internet communication is infallible, and you believed it… I have a bridge to sell you. </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><ducks> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Best Regards,</span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Ivan Kovacevic</span></p><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Vice President, Client Services</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Star Telecom | <a href="http://www.startelecom.ca/">www.startelecom.ca</a> | SIP Based Services for Contact Centers </span></p></div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><div><div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> VoiceOps [mailto:<a href="mailto:voiceops-bounces@voiceops.org">voiceops-bounces@voiceops.org</a>] <b>On Behalf Of </b>Anthony Orlando via VoiceOps<br><b>Sent:</b> Thursday, March 17, 2016 8:09 PM<br><b>To:</b> Paul Timmins <<a href="mailto:paul@timmins.net">paul@timmins.net</a>><br><b>Cc:</b> <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br><b>Subject:</b> Re: [VoiceOps] VoIP Innovations reliability</span></p></div></div><p class="MsoNormal"> </p><div><p class="MsoNormal">I get it Paul. It's just a shame the terrorists win. Too bad we can't put a bounty on their sorry asses. <br><br>Sent from my iPhone</p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>On Mar 17, 2016, at 6:55 PM, Paul Timmins <<a href="mailto:paul@timmins.net">paul@timmins.net</a>> wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><div><p class="MsoNormal">Sadly, assassins win if i stop breathing, so if someone starts to choke me, I fight it the best I can and run away at my earliest opportunity.<br><br>On 03/17/2016 07:38 PM, Anthony Orlando via VoiceOps wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal">It's a shame we can't support them. This could be anyone of us. Hackers win if you port away. <br><br>Sent from my iPhone</p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>On Mar 17, 2016, at 5:43 PM, Robert Johnson <<a href="mailto:fasterfourier@gmail.com">fasterfourier@gmail.com</a>> wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><div><p class="MsoNormal">Their primary SBC and DR IP are both in the same IP netblock, so whenever the DDOS hits, both IPs are affected. The past few outages have involved 80% packet loss or so to both hosts, so some calls do make their way through, and plenty of wierdness ensues when an INVITE makes its way through but not the OK on the way back. </p><div><p class="MsoNormal"> </p></div><div><p class="MsoNormal">Can't wait to get our numbers ported out.</p></div></div><div><p class="MsoNormal"> </p><div><p class="MsoNormal">On Thu, Mar 17, 2016 at 6:27 PM, Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>> wrote:</p><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><p class="MsoNormal">Annnd they're down again. </p><div><div><p class="MsoNormal" style="margin-bottom:12.0pt"> </p><div><p class="MsoNormal">On 3/17/2016 5:14 PM, Nate Burke wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><p class="MsoNormal" style="margin-bottom:12.0pt">6 calls from 4 different CID numbers. All within 3 minutes. </p><div><p class="MsoNormal">On 3/17/2016 4:58 PM, Nathan Anderson wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Yesterday I definitely saw calls coming from the DR IP in my logs, but I had not yet added that IP as a peerin our SBC. I'll have to comb through logs today to see if we got any.</span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Are you saying that the multiple calls you saw coming to your desk were all from the same number? If I had to guess, their side probably sent continuous INVITEs to you when it failed to get back an OK for any of them (not that you weren't sending back OK, but that their either didn't reach their SBC or did not reach it in a timely manner).</span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">-- Nathan</span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif"> VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>] <b>On Behalf Of </b>Jeff Waddell<br><b>Sent:</b> Thursday, March 17, 2016 2:53 PM<br><b>To:</b> Nate Burke<br><b>Cc:</b> <a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br><b>Subject:</b> Re: [VoiceOps] VoIP Innovations reliability</span></p></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">That is the issue a lot of our customers are reporting - where multiple calls are sent.... </p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Mar 17, 2016 at 5:49 PM, Nate Burke <<a href="mailto:nate@blastcomm.com">nate@blastcomm.com</a>> wrote:</p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I didn't see any traffic increment on the DR IP Address in my firewall rules, but this was odd. During the 15 minute period, I had probably 5 or 6 simultaneous calls ring into my desk. I normally only take a handfull of calls a day. </p><div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 3/17/2016 4:39 PM, Jeff Waddell wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We implemented it too - I haven't checked to see if any traffic was sent across it </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p></div></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Mar 17, 2016 at 5:20 PM, Nate Burke <<a href="mailto:nate@blastcomm.com">nate@blastcomm.com</a>> wrote:</p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Only 15 Minutes this time though. I had implemented the Disaster Recover Trunk as mentioned previously, but I didn't seem to be getting any calls completed through it. </p><div><div><p class="MsoNormal" style="margin-bottom:12.0pt"> </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 3/17/2016 4:16 PM, Shripal Daphtary wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="margin-bottom:12.0pt">Down again! </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks, </p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Shripal</p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>On Mar 16, 2016, at 9:50 AM, Nate Burke <<a href="mailto:nate@blastcomm.com">nate@blastcomm.com</a>> wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="margin-bottom:12.0pt">Looks like it just came back up for me. Just over 30 min.<br><br>Nate</p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 3/16/2016 8:45 AM, Shripal Daphtary wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="margin-bottom:12.0pt">We are experiencing an outage as well. </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks, </p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> </p></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Shripal</p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>On Mar 16, 2016, at 9:36 AM, Nate Burke <<a href="mailto:nate@blastcomm.com">nate@blastcomm.com</a>> wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="margin-bottom:12.0pt">Problems again this morning? Looks to be acting the same as it has been. </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 3/11/2016 6:00 PM, Alexander Lopez wrote:</p></div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I added them to our monitoring platform, stated getting alarms this past hour or so. </p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Up and down.</p></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br><br>-------- Original message --------<br>From: Nathan Anderson <a href="mailto:nathana@fsr.com"><nathana@fsr.com></a> <br>Date: 3/11/2016 6:31 PM (GMT-05:00) <br>To: 'Nate Burke' <a href="mailto:nate@blastcomm.com"><nate@blastcomm.com></a>, <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a> <br>Subject: Re: [VoiceOps] VoIP Innovations reliability </p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt">...aaaaaand we're back.<br><br>-- Nathan<br><br>-----Original Message-----<br>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org">mailto:voiceops-bounces@voiceops.org</a>] On Behalf Of Nathan Anderson<br>Sent: Friday, March 11, 2016 3:30 PM<br>To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br>Subject: Re: [VoiceOps] VoIP Innovations reliability<br><br>It *feels* like they are under attack again, since I get a response to a ping once every 20 or so.<br><br>-- Nathan<br><br>-----Original Message-----<br>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org">mailto:voiceops-bounces@voiceops.org</a>] On Behalf Of Nathan Anderson<br>Sent: Friday, March 11, 2016 3:28 PM<br>To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br>Subject: Re: [VoiceOps] VoIP Innovations reliability<br><br>Confirmed.<br><br>-- Nathan<br><br>-----Original Message-----<br>From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org">mailto:voiceops-bounces@voiceops.org</a>] On Behalf Of Nate Burke<br>Sent: Friday, March 11, 2016 3:26 PM<br>To: <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br>Subject: Re: [VoiceOps] VoIP Innovations reliability<br><br>Anyone else show them down again right now? My traceroutes aren't even <br>leaving Chicago. Dying at a Chicago hop on Level3.<br><br>Nate<br><br>On 3/6/2016 6:50 PM, Nathan Anderson wrote:<br>> Did anybody else just suffer another 45-minute-ish long outage from about 4:00p PST to 4:45p PST (ending about 5 minutes ago)?<br>><br>> -- Nathan<br>><br>> -----Original Message-----<br>> From: <a href="mailto:frnkblk@iname.com">frnkblk@iname.com</a> [<a href="mailto:frnkblk@iname.com">mailto:frnkblk@iname.com</a>]<br>> Sent: Sunday, March 06, 2016 4:43 PM<br>> To: Nathan Anderson; <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br>> Subject: RE: VoIP Innovations reliability<br>><br>> More here: <a href="http://blog.voipinnovations.com/blog">http://blog.voipinnovations.com/blog</a><br>><br>> Frank<br>><br>> -----Original Message-----<br>> From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org">mailto:voiceops-bounces@voiceops.org</a>] On Behalf Of Nathan<br>> Anderson<br>> Sent: Tuesday, March 01, 2016 8:27 PM<br>> To: <a href="mailto:voiceops@voiceops.org">voiceops@voiceops.org</a><br>> Subject: [VoiceOps] VoIP Innovations reliability<br>><br>> Holy schlamoly. Anybody else use them here and being handed outage after<br>> outage over the last 2 days? Seriously thinking at this point about doing<br>> something else. This is ridiculous.<br>><br>> I desperately need sleep and if my cell goes off one more time...<br>><br>> -- Nathan<br>> _______________________________________________<br>> VoiceOps mailing list<br>> <a href="mailto:VoiceOps@voiceops.org">VoiceOps@voiceops.org</a><br>> <a href="https://puck.nether.net/mailman/listinfo/voiceops">https://puck.nether.net/mailman/listinfo/voiceops</a><br>><br>><br>><br>> _______________________________________________<br>> VoiceOps mailing list<br>> <a href="mailto:VoiceOps@voiceops.org">VoiceOps@voiceops.org</a><br>> <a href="https://puck.nether.net/mailman/listinfo/voiceops">https://puck.nether.net/mailman/listinfo/voiceops</a></span> <a href="https://puck.nether.net/mailman/listinfo/voiceops">https://puck.nether.net/mailman/listinfo/voiceops</a></p></div></blockquote></div></blockquote></blockquote></div></blockquote></blockquote></div></div></div></div></div></blockquote></div></div></div></div></div></div></blockquote></blockquote></div></div></div></blockquote></div></div></div></blockquote></blockquote><p class="MsoNormal"> </p></div></blockquote><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><p class="MsoNormal">_______________________________________________<br>VoiceOps mailing list<br><a href="mailto:VoiceOps@voiceops.org">VoiceOps@voiceops.org</a><br><a href="https://puck.nether.net/mailman/listinfo/voiceops">https://puck.nether.net/mailman/listinfo/voiceops</a></p></div></blockquote></div></body></html>