[c-nsp] Modern BGP peering border router and DDoS attack defense recommendations?

[Matt Buford]

At my service level, the goal is to keep the victim site operational 
throughout the attack.

In my experience, the Cisco/Riverhead Guard is excellent at protecting 
against packet floods and connection floods.  However, it was not very 
effective against widely distributed application level attacks.

In one case, I had 20,000+ IPs hitting high-cost URLs that included database 
queries.  These zombies seemed to browse around somewhat randomly to sort of 
mimic humans, and they only were doing a few hits per minute so any single 
IP did not seem to be flooding.  However, over a period of time I could pick 
them out by processing the logs and looking at IPs that had hit the site at 
least once in every clock minute for the last 24 hours.  However, even after 
I generated my list, I came back and regenerated it every hour and there 
were always another hundred or so new IPs that hadn't been seen before. 
This steady addition of new IPs made it hard to block at any level.

Additionally, finding somewhere to ACL out 20,000+ IPs was a pain.  The 
Riverhead was not happy when I tried to add the IPs manually, and it also 
did annoying things like attempting to show the currently configured manual 
ACLs would say "Display trimmed at 1000 entries" and I had no way to view 
more.

The PIX also had no interest in allowing a 20,000 line ACL.

I finally blocked them via .htaccess on the web servers themselves.  Apache 
had no complaints with a long list of IPs, and this resulted in "Forbidden" 
responses which were low-cost responses that did not involve database 
lookups.

Since this, I've tried to emphasize to customers that application level 
attacks are application level issues.  The attack should be detected and 
blocked right in the application itself.  Network level scrubbing is for 
connection floods, packet floods, and anti-spoofing.

All that said, at the same time those zombies were hitting, there was a 
couple hundred megs of packet floods.  Some UDP and some SYNflooding.  The 
Riverhead stopped that part of the attack without any issue.

In the past year it seems like things have died down.  Perhaps my customers 
just aren't the DoS magnets they used to be.  However, in past years 1 mPPS 
and 1 gigabit attacks were not uncommon, and the Riverhead was highly 
effective in dealing with these.  Of course, remember that your transit and 
your routers have to be able to push this.

1 mPPS SYNflood = 1 mPPS inbound + 1 mPPS of syncookies outbound.  You 
aren't going to push this on a 7200. :)

The other issue is cost.  Even if you can defend and keep the site running 
smoothly during a gigabit DDoS, can the customer afford the bandwidth bill 
at the end of the month?  Even customers that can afford to pay for our DDoS 
mitigation services are often unable to afford the bandwidth bill when an 
attack hits.

----- Original Message ----- 
From: "Tantsura, Jeff" <jtantsura at ugceurope.com>
To: "'David J. Hughes'" <bambi at hughes.com.au>
Cc: <sam_crooks at yahoo.com>; <cisco-nsp at puck.nether.net>
Sent: Tuesday, June 14, 2005 5:48 AM
Subject: RE: [c-nsp] Modern BGP peering border router and DDoS attack 
defense recommendations?


> I've started with - put your vital web based services in a
> tier 1/2 SP COLO.
>>From my experience decent application based DoS attacks are not about Gigs
> of traffic but exhaustion of Web server farm recourses.
> Again, proper setup web server farm with decent load balancer in front
> + health check on application level will provide you great protection but
> it's very expensive to build it