[Outages-discussion] [outages] HE tunnelbroker.net down?

[Jeremy Chadwick]

There's no speculation in this statement: Vodafone is at the mercy of
HE.

An earlier post of yours said, and I quote: "HE is not a telco, they
will tell you what happened".

I have talked at length on two separate occasions on outages, about HE's
complete and total incompetence on numerous levels, and just as bad,
lack of transparency when it comes to outages of all sorts.  This makes
rant #3.

https://puck.nether.net/pipermail/outages/2009-September/001547.html
http://puck.nether.net/pipermail/outages-discussion/2011-May/000225.html

Both of these posts solicited many responses from list members to me
privately citing their experiences as customers of HE, some of whom had
quit/left and some who remained there but were looking at alternatives
given that their experiences were identical to mine.  I was surprised by
the number of responses I received, but felt relieved to know I was not
the only one amazed by their seat-of-the-pants operations.

I acknowledge it's considered unprofessional to curse on a mailing list,
but with Hurricane Electric Fremont (doesn't matter if it's Facility 1
or 2, *really* it doesn't), this is SSDD (Same Shit Different Day).  I
forget which of their datacenters (FMT1, FMT2, or SJC1) houses their
public EFnet IRC server, which is more or less a DDoS magnet.  I often
wonder if customers are made aware of that fact.

So let's be accurate: blame both vendors for two different things: HE
for horrible and atrocious service, and Vodafone for lack of redundancy.
Shame on them both, as the only ones who suffer are the customers.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |

On Wed, Oct 05, 2011 at 12:54:30AM -0400, Patrick W. Gilmore wrote:
> On Oct 4, 2011, at 12:09 PM, Frank Bulk wrote:
> 
> > Fact, speculation, or educated guess?
> 
> Fact: Vodofone claims their customers could not reach international destination IP addresses because HE was down.
> <quote>
> Spokesman Matt East said the issue was caused by a denial of service attack on Hurricane Electric, a Californian company that provides international connectivity to Vodafone.
> </quote>
> 
> Educated Guess: Sounds to me like they were single homed.  I'm happy to listen to other educated guesses on how to explain the fact above.
> 
> Speculation: Lots of things get lost in translation.  For instance, perhaps they are multi-homed, but had some problems to some locations because one of their transit providers (HE) was being attacked, and stuff.co.nz translated that as 'down'.  Although the peers & customers of HE who talked to me said they lost BGP sessions with HE; so if Vodafone was not in some unusual state during the attack (BGP up, packets not flowing), I don't see how that fits the symptoms above.
> 
> We can speculate more.  Or maybe Vodafone or HE can just tell us?
> 
> -- 
> TTFN,
> patrick
> 
> 
> > -----Original Message-----
> > From: outages-discussion-bounces at outages.org
> > [mailto:outages-discussion-bounces at outages.org] On Behalf Of Patrick W.
> > Gilmore
> > Sent: Tuesday, October 04, 2011 10:20 AM
> > To: outages-discussion at outages.org
> > Subject: Re: [Outages-discussion] [outages] HE tunnelbroker.net down?
> > 
> > On Oct 4, 2011, at 10:57 AM, virendra rode wrote:
> >> On 10/03/2011 07:22 PM, Frank Bulk wrote:
> >>> Vodafone says their mobile phone and broadband customers couldn't access
> >>> international contact because HE was down:
> >>> 
> > http://www.stuff.co.nz/technology/digital-living/5730577/Vodafone-internet-o
> >>> utage
> >>> 
> >>> Single-homed, seriously?
> > 
> > <snip>
> > 
> >> Waiting to see to what went wrong from lesson learned standpoint.
> > 
> > I think we know what went wrong: Vodafone used HE for all their
> > international IP connectivity.
> > 
> > -- 
> > TTFN,
> > patrick 
> > 
> 
> 
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion