http://www.zdnet.com/windows/stories/prtfriendly/0,4758,2168256,00.html
How to Survive a Hack Attack
November 23, 1998
Its late, you're sleeping, you get a call waking you . . . "Fred, we're
being attacked, someone is trying to do a denial of service attack against
our web server". You quickly realize that lots of people aren't going to
be very happy, so you leap into action . . . after all, you're the
security GURU, right?
So what do you do about this "alert"?
Well, for many, this has already happened and, for the most part, they've
experienced what I'm going to describe. However, many of you who read
NTBugtraq and follow security alerts on the 'net haven't had it happen to
you yet . . . and it is you I'd like to talk to.
You see the number of times the alert is sounded, versus the number of
times an actual attack occurs, is extremely disproportionate. Usually,
chances are you are not under attack.
Don't get me wrong, I'm not saying attacks don't happen—they do,
especially to ISPs. The key to responding to this type of alert--the key
to determining if you're really under attack--is how you handle the facts.
1. Do you know what's supposed to be running on the box?
Making sure that someone can tell you precisely what's supposed to be
running on the box is crucial to handling an alert. Without this
knowledge, you're at a loss to know whether what's supposed to be there is
there. Has something been placed there by an attacker? Knowing the right
version numbers and names of the various executable programs that are
running (or could be running), when they're supposed to be running, and
what each of them actually does, is crucial to keeping control of your
server.
And of course, knowing all of this will make it easier to search through
Knowledgebase articles and/or vendor's support sites . . . but don't go
there yet.
2. Do you have a backup?
Before you go trying to stop an attack, you need to be sure you can
recover from it once you do. Since you don't know what the attack may or
may not have done to your system, figuring out when your last backup was
made, and more importantly how useful it might be, is crucial to
determining how to handle the alert. If you know you have a backup from
just prior to any indications of an alert you could, if necessary,
completely replace the box. That makes attacking the problem easier:
Should you have to delete or alter something sensitive in the process, you
know you can go to the backup if all else fails. If you don't have a
reasonably up-to-date backup, then you'll have to be a whole lot more
cautious in your approach. And you'll have to work harder to determine
exactly what might have changed due to the alert—trashing the system and
doing a full "restore" won't be an option.
3. Who discovered the alert? What exactly did they discover?
Who discovered the alert is important in diagnosing the problem because
their knowledge and expertise could greatly affect the validity of their
observations. Not to be harsh, but: Do they know what they're talking
about? Should they have been able to even see the alert? If not, what
were they doing on the box that allowed them to discover it? The answer to
that question has solved more than a few alerts I've dealt with (um, they
were just poking around the registry on that box and then all of a sudden
they noticed the CPU went to 100% . . . ;-])
Additionally, what the alert actually is can be highly subjective. That is
to say that there may be a difference between what the person thinks the
problem is and the problem itself. We've all likely heard the claim "its
running much slower than it usually does . . ." only to find out it
happens to be in the middle of a full system backup . . . ;-] Performance
counters are complex, and can easily be mis-read. Telling someone that the
number of "HTTP Connection Attempts" is way above normal doesn't
necessarily mean you're under attack. Especially if they take that Perfmon
value and compare it with a web statistics program's "Hits" value (when
you consider that a connection attempt doesn't necessarily get logged as a
hit if the transfer was unsuccessful).
Too many people spend far too little time analyzing the alert and
verifying it by multiple mechanisms—sad but true. Do it: analyze & verify.
If you think you're under a DOS attack, you should:
* run Netmon and see whether network utilization is being sustained at a
higher than normal level. And remember to compare your LAN bandwidth
with that of your ISP connection. Your utilization shouldn't be able to
sustain rates higher than your Internet connection allows. If it is,
then the attack is either coming from another machine on your LAN, or
you've got some other problem causing the saturation.
* look at "netstat -n 1" and see whether or not you have a lot of
connection attempts from the same, or similar, IP addresses. Chances
are a DOS attack is going to come from a single IP address, or a range
of addresses from a single subnet. When the Teardrop2 attack happened
earlier this year, the source address was always the same. Despite it
being spoofed (i.e. not from the address you actually saw), it was
still the same address repeatedly.
* Have a look at your web logs. Do they reflect the traffic you're
seeing? The Net is a funny place. I know—I had a spike on my web site
once that seriously made me consider it an alert. Turns out something I
wrote had just been linked from Microsoft's web site and the viewers
rose exponentially in a matter of minutes.
If the results of the above verifications show some inconsistencies, then
you can assume (as I usually do) that someone has changed something on the
box, and that it's causing a problem. When did the alert start? Did
someone add or remove something from the box around that time? What's new
on the box? Can you revert back to what you had before, and if you do,
does the problem still occur? I can't stress this enough: chances are
something you (or your team) did is the cause of your alert. More often
than not, it's a change to the box that's caused things to run afoul.
If, on the other hand, all verifications check out and you're convinced
your being attacked, then what?
4. Contact your ISP.
If you can identify a particular IP address (or range of IP addresses) as
the root cause of the alert, you should contact your ISP and ask them to
filter out that address at their routers. Of course you could filter out
the offending address at your routers, but that'll amount to a hill o'
beans most likely. Remember, the bandwidth from your ISP to you is your
bandwidth—it will still be consumed by the attacking traffic as it comes
down to your router to be rejected! Getting your ISP involved should help
avoid this bandwidth loss, as well as set their security folks in motion.
An attack against you is also an attack against your ISP! Once you and
your ISP have cut off the attack, you're looking at fixing what ever was
damage (say hello to your backup) and patching you box.
Of course it goes without sayting that you and your ISP should have a plan
in place before an attack occurs. Talk to your ISP now and find out what
they will do in such a situation . . . if they don't have an answer:
change your ISP!
5. Log everything you can.
While actually going to court against an attacker is extremely rare, you
won't be able to do anything without logs of the attack. Further,
companies like Network Flight Recorder (http://www.nfr.net) and ISS Real
Secure (http://www.iss.net), who make products that can detect attack
signatures on the wire, would greatly appreciate detailed logs of actual
attack events. The more detailed the information you can provide to such
vendors, the more they'll be able to build products to combat such
attacks.
6. Let me know about it!
NTBugtraq is here to keep NT folk informed about such attacks. The damage
done by Teardrop2, while devastating to quite a few people for a short
period of time, was limited at least in part due to our ability to
coordinate lots of different people onto the same problem . . . virtually
in real time.
Chances are, if you follow these steps, your problem will be resolved.
Remember:
1. know what's running on your box
2. keep current backups
3. verify the attack as real
4. cut it off at the ISP level
5. log it
6. report it
Bottom line to all of this is to make sure you have a plan. Make sure you
know what you're going to do when/if you should ever get that call in the
middle of the night. Trust me, when it happens, your heart is going to
start pumping and you're going to be scrambling. Having a plan, a list of
questions to ask, a list of things to do, a list of people to call (with
numbers!) will help calm you down and let you take the right actions to
resolve it quickly.
-o-
Subscribe: mail majordomo@repsec.com with "subscribe isn".
Today's ISN Sponsor: Repent Security Incorporated [www.repsec.com]
Received on Tue Dec 8 08:58:54 1998