[outages] Neustar UltraDNS sporadic resolution failure

[Paul Hinze]

Looks like it was a large scale DDoS attack:

> Dear UltraDNS Customer,
>
> Beginning at 00:47 GMT on 10/17/12 (please note corrected start time from original notification), customers on the Neustar UltraDNS network were subjected to a large scale DDoS attack. During the early portions of this attack, several node locations across the UltraDNS network experienced high latency and saturation leading to DNS response timeouts and increased drop rates.
>
> At approximately 01:58 GMT, Neustar had restored normal service levels to a majority of the network locations, but continued to experience higher than normal latency in the Western US region. This increased latency continued in that region until approximately 03:19 GMT when all remaining impact was cleared.
>
> While we have fully restored service to all locations at this time, we will maintain proactive mitigation on the network and continue to actively monitor the situation to guard against a renewal of attack activities.
>
> At Neustar we are committed to providing the highest levels of performance and reliability through the products and solutions we deliver. Please feel free to contact our 24x7 UltraDNS Support Team at ultrasupport at Neustar.biz  with any questions or concerns.
>
> Sincerely,
>
> UltraDNS Support

On Tue, Oct 16, 2012 at 10:03 PM, Patrick W. Gilmore <patrick at ianai.net> wrote:
> Works from Comcast in Boston.
>
> Where is it down?  Remember, Ultra is massively anycasted.
>
> --
> TTFN,
> patrick
>
>
> On Oct 16, 2012, at 22:59 , Avleen Vig <avleen at gmail.com> wrote:
>
>> In addition to this, we saw a whole variety of "weird" problems at this time.
>> Internetpulse reported packet loss between multiple ISPs
>> (ATT/Verizon/Sprint/L3 in different directions).
>>
>> We saw packet loss on Abovenet, too.
>>
>> Reaching AWS was sporadic on multiple ISPs...
>>
>> Anyone else see this happen too?
>>
>> On Tue, Oct 16, 2012 at 8:35 PM, Paul Hinze <paul.t.hinze at gmail.com> wrote:
>>> Seeing repeated failures on both udns1.ultradns.net and udns2.ultradns.net.
>>>
>>> Getting some initial corroboration from several folks on twitter, but no
>>> acknowledgement from Neustar.
>>>
>>> ; <<>> DiG 9.8.3-P1 <<>> @udns2.ultradns.net www.braintreegateway.com
>>> ; (1 server found)
>>> ;; global options: +cmd
>>> ;; connection timed out; no servers could be reached
>>> _______________________________________________
>>> Outages mailing list
>>> Outages at outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages