O’Reilly Network: Scanning for Rootkits

“Usually, the first sign that a server might be
compromised is simple anomalies in the behavior of the server. One
of the more common anomalies one might notice is a change in how
one or more of the core system utilities behave. For instance, a
command-line switch to ‘netstat’ or ‘ps’, which you used to use
without a problem everyday, might start returning an error message.
The reason for this is that intruders replace these utilities with
versions designed to hide their malicious activities. The utility
they replace your original one with might be a different version,
or it could have been compiled with different options, and as a
result, it does not have the same options you are used to.

Another anomaly that should raise a big red flag is a change in
your bandwidth-usage patterns. If you or your hosting company
routinely monitors your bandwidth usage, you might notice an
increase in the amount of traffic your server is pushing compared
with your normal traffic patterns. This is usually caused by
intruders using your server to distribute copyrighted software,
commonly known as ‘warez’. Remember that you might be the target of
a legal action as a result of such activities on your servers.

Ideally, a server administrator should not wait until all the
alarms go off before a server is checked for signs of compromise,
because the less time an intruder has the opportunity to spend on a
server, the less damage he or she will be able to inflict.
(Although a very malicious intruder can potentially wipe out the
whole system seconds after gaining root-level access.) For this
reason, it is important to conduct server-security audits
periodically, and to know as soon as possible when a server is
compromised.”

Complete
Story