<div dir="ltr">Their primary SBC and DR IP are both in the same IP netblock, so whenever the DDOS hits, both IPs are affected. The past few outages have involved 80% packet loss or so to both hosts, so some calls do make their way through, and plenty of wierdness ensues when an INVITE makes its way through but not the OK on the way back.<div><br></div><div>Can't wait to get our numbers ported out.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 17, 2016 at 6:27 PM, Nate Burke <span dir="ltr"><<a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Annnd they're down again.<div><div class="h5"><br>
<br>
<div>On 3/17/2016 5:14 PM, Nate Burke wrote:<br>
</div>
<blockquote type="cite">
6 calls from 4 different CID numbers. All within 3 minutes. <br>
<br>
<div>On 3/17/2016 4:58 PM, Nathan Anderson
wrote:<br>
</div>
<blockquote type="cite">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Yesterday
I definitely saw calls coming from the DR IP in my logs,
but I had not yet added that IP as a peerin our SBC. I'll
have to comb through logs today to see if we got any.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Are
you saying that the multiple calls you saw coming to your
desk were all from the same number? If I had to guess,
their side probably sent continuous INVITEs to you when it
failed to get back an OK for any of them (not that you
weren't sending back OK, but that their either didn't
reach their SBC or did not reach it in a timely manner).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--
Nathan<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>]
<b>On Behalf Of </b>Jeff Waddell<br>
<b>Sent:</b> Thursday, March 17, 2016 2:53 PM<br>
<b>To:</b> Nate Burke<br>
<b>Cc:</b> <a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
<b>Subject:</b> Re: [VoiceOps] VoIP Innovations
reliability<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">That is the issue a lot of our
customers are reporting - where multiple calls are
sent.... <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Mar 17, 2016 at 5:49 PM, Nate
Burke <<a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>>
wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">I didn't see any traffic increment
on the DR IP Address in my firewall rules, but this
was odd. During the 15 minute period, I had probably
5 or 6 simultaneous calls ring into my desk. I
normally only take a handfull of calls a day. <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 3/17/2016 4:39 PM, Jeff
Waddell wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">We implemented it too - I
haven't checked to see if any traffic was sent
across it <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Mar 17, 2016 at
5:20 PM, Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank"></a><a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>>
wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">Only 15 Minutes this
time though. I had implemented the
Disaster Recover Trunk as mentioned
previously, but I didn't seem to be
getting any calls completed through it. <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 3/17/2016 4:16
PM, Shripal Daphtary wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Down
again! <u></u><u></u></p>
<div>
<p class="MsoNormal">Thanks, <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Shripal<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On Mar 16, 2016, at 9:50 AM, Nate
Burke <<a href="mailto:nate@blastcomm.com" target="_blank"></a><a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>>
wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Looks
like it just came back up for
me. Just over 30 min.<br>
<br>
Nate<u></u><u></u></p>
<div>
<p class="MsoNormal">On
3/16/2016 8:45 AM, Shripal
Daphtary wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">We
are experiencing an outage
as well. <u></u><u></u></p>
<div>
<p class="MsoNormal">Thanks, <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Shripal<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On Mar 16, 2016, at 9:36 AM,
Nate Burke <<a href="mailto:nate@blastcomm.com" target="_blank"></a><a href="mailto:nate@blastcomm.com" target="_blank">nate@blastcomm.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Problems
again this morning? Looks
to be acting the same as
it has been. <u></u><u></u></p>
<div>
<p class="MsoNormal">On
3/11/2016 6:00 PM,
Alexander Lopez wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">I
added them to our
monitoring platform,
stated getting alarms
this past hour or so.
<u></u><u></u></p>
<div>
<p class="MsoNormal">Up
and down.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
-------- Original
message --------<br>
From: Nathan Anderson
<a href="mailto:nathana@fsr.com" target="_blank"></a><a href="mailto:nathana@fsr.com" target="_blank"><nathana@fsr.com></a>
<br>
Date: 3/11/2016 6:31
PM (GMT-05:00) <br>
To: 'Nate Burke' <a href="mailto:nate@blastcomm.com" target="_blank"></a><a href="mailto:nate@blastcomm.com" target="_blank"><nate@blastcomm.com></a>,
<a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a>
<br>
Subject: Re:
[VoiceOps] VoIP
Innovations
reliability <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt">...aaaaaand we're back.<br>
<br>
-- Nathan<br>
<br>
-----Original
Message-----<br>
From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"></a><a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>]
On Behalf Of Nathan
Anderson<br>
Sent: Friday, March
11, 2016 3:30 PM<br>
To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
Subject: Re:
[VoiceOps] VoIP
Innovations
reliability<br>
<br>
It *feels* like they
are under attack
again, since I get a
response to a ping
once every 20 or so.<br>
<br>
-- Nathan<br>
<br>
-----Original
Message-----<br>
From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"></a><a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>]
On Behalf Of Nathan
Anderson<br>
Sent: Friday, March
11, 2016 3:28 PM<br>
To: 'Nate Burke'; <a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
Subject: Re:
[VoiceOps] VoIP
Innovations
reliability<br>
<br>
Confirmed.<br>
<br>
-- Nathan<br>
<br>
-----Original
Message-----<br>
From: VoiceOps [<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"></a><a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>]
On Behalf Of Nate
Burke<br>
Sent: Friday, March
11, 2016 3:26 PM<br>
To: <a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
Subject: Re:
[VoiceOps] VoIP
Innovations
reliability<br>
<br>
Anyone else show
them down again
right now? My
traceroutes aren't
even <br>
leaving Chicago.
Dying at a Chicago
hop on Level3.<br>
<br>
Nate<br>
<br>
On 3/6/2016 6:50 PM,
Nathan Anderson
wrote:<br>
> Did anybody
else just suffer
another
45-minute-ish long
outage from about
4:00p PST to 4:45p
PST (ending about 5
minutes ago)?<br>
><br>
> -- Nathan<br>
><br>
> -----Original
Message-----<br>
> From: <a href="mailto:frnkblk@iname.com" target="_blank"></a><a href="mailto:frnkblk@iname.com" target="_blank">frnkblk@iname.com</a>
[<a href="mailto:frnkblk@iname.com" target="_blank"></a><a href="mailto:frnkblk@iname.com" target="_blank">mailto:frnkblk@iname.com</a>]<br>
> Sent: Sunday,
March 06, 2016 4:43
PM<br>
> To: Nathan
Anderson; <a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
> Subject: RE:
VoIP Innovations
reliability<br>
><br>
> More here: <a href="http://blog.voipinnovations.com/blog" target="_blank"></a><a href="http://blog.voipinnovations.com/blog" target="_blank">http://blog.voipinnovations.com/blog</a><br>
><br>
> Frank<br>
><br>
> -----Original
Message-----<br>
> From: VoiceOps
[<a href="mailto:voiceops-bounces@voiceops.org" target="_blank"></a><a href="mailto:voiceops-bounces@voiceops.org" target="_blank">mailto:voiceops-bounces@voiceops.org</a>]
On Behalf Of Nathan<br>
> Anderson<br>
> Sent: Tuesday,
March 01, 2016 8:27
PM<br>
> To: <a href="mailto:voiceops@voiceops.org" target="_blank"></a><a href="mailto:voiceops@voiceops.org" target="_blank">voiceops@voiceops.org</a><br>
> Subject:
[VoiceOps] VoIP
Innovations
reliability<br>
><br>
> Holy
schlamoly. Anybody
else use them here
and being handed
outage after<br>
> outage over the
last 2 days?
Seriously thinking
at this point about
doing<br>
> something
else. This is
ridiculous.<br>
><br>
> I desperately
need sleep and if my
cell goes off one
more time...<br>
><br>
> -- Nathan<br>
>
_______________________________________________<br>
> VoiceOps
mailing list<br>
> <a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
><br>
><br>
><br>
>
_______________________________________________<br>
> VoiceOps
mailing list<br>
> <a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
> <a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br>
_______________________________________________<br>
VoiceOps mailing
list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br>
_______________________________________________<br>
VoiceOps mailing
list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br>
_______________________________________________<br>
VoiceOps mailing
list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br>
_______________________________________________<br>
VoiceOps mailing
list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank"></a><a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank"></a><a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><u></u><u></u></span></p>
</div>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><u></u><u></u></p>
</div>
</blockquote>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><u></u><u></u></p>
</div>
</blockquote>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
VoiceOps mailing list
<a href="mailto:VoiceOps@voiceops.org" target="_blank">VoiceOps@voiceops.org</a>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
VoiceOps mailing list<br>
<a href="mailto:VoiceOps@voiceops.org">VoiceOps@voiceops.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/voiceops" rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/voiceops</a><br>
<br></blockquote></div><br></div>