[outages] Godaddy / Premium DNS outage?

[Warren Kumari]

--  
Warren Kumari
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Friday, October 5, 2012 at 2:29 PM, ryanL wrote:

> so, redistributed eBGP into IGP? how does one 210x their routing table? i appreciate the openness, but it doesn't sound like a "perfect storm" to me. it sounds like someone screwed up.

Could be someone screwed up, could have been caused by a bug.

Your assumption that this was redistribution of eBGP -> IGP is not really backed up by the text. if I had to guess, I'd suppose that they have devices (like cisco 4900 series) that don't have enough memory / FIB / TCAM for full tables. Normally they only send partial routes (such as internal only / large aggregates / something) and accidentally or though a bug dropped filter and overflowed these devices…

But, whatever the case, GoDaddy has released a somewhat reasonable post-mortem, sounds like they are / have addressed the issue and os it shouldn't happen again, etc.
As many know, I'm not a real of GoDaddy, but credit where it is due…

Running a large network is non-trivial, sometimes s%#@t happens -- at least GoDaddy was willing to publish something and not just hand wave this away…

We keep kvetching that folk never publish postmortems when they have an outage -- lets not attack them when they do…

W


  
>  
>  
> On Fri, Oct 5, 2012 at 10:18 AM, Jason LeBlanc <jleblanc at godaddy.com (mailto:jleblanc at godaddy.com)> wrote:
> > I made a commitment to get everyone on here the RFO. Its presented in
> > layman's terms:
> >  
> > http://inside.godaddy.com/inside-story-happened-godaddy-com-sept-10-2012/
> >  
> > Regards,
> >  
> > Jason LeBlanc
> > Network Engineering
> > (480)505-8800 x7202 (tel:%28480%29505-8800%20x7202)
> >  
> >  
> > Please contact my direct supervisor mdob at godaddy.com (mailto:mdob at godaddy.com) with any feedback.
> _______________________________________________
> Outages mailing list
> Outages at outages.org (mailto:Outages at outages.org)
> https://puck.nether.net/mailman/listinfo/outages