issues with security

[cliff at cliffmeyers.com]

Hi Everyone,


Apparently my first message didn't go through, so here I go again:


I've been away from the list for a little while, but been having a major problem
here at the office so I figured I'd post to see if you guys had any ideas...

On the 22nd we had an issue with one of our systems that I thought had to do with
some kind of hard drive error.  The system is a Red Hat Linux box, running
primarily Apache and PHP to serve web sites.  I typically compile these things
from source so I can have a little more control over configurability.

Anyways, as it turned out, I noticed late last week that there were processes
running that shouldn't be there.  After I killed the processes I noticed files in
the /tmp directory, where PHP stores most of the session files (unless I tell it
to store them somewhere else).  There was a 'blackhole.c' file and some other
things which had been compiled to run on my system.

I talked to my other programmer and we were going to come in Saturday to do a
full re-install, but the hacker struck against Thursday night around 11 PM and
defaced all of our sites.  I came into the office and spent the next 8 hours
formatting, installing Red Hat 9, download all the newest source code for Apache
and PHP, and getting everything set up.

Well, I get into work today, and guess what?  Another bad process and more files
in the /tmp folder.  I killed them all again, and am going to do *another*
reinstall tonight.  I was e-mailing a colleague asking for his input so I'll post
a few of the ideas I had for how the hacker got back in.  Here they are:

(1)  I used the latest stable version of PHP, 4.3.4, when in fact there is a new
version, 4.3.5RC1.  I wanted to avoid a release candidate version but that would
be my first guess.

(2)  Some other vulnerability I don't know about.  I installed the latest version
of every other package so that's probably unlikely. Every other service is
firewalled, so...

(3)  I used the web backup from the morning of the 30th so as to not loose any
changes - perhaps there was something in there that it allowing the hacker to
gain access again.

(4)  A problem with some of our PHP code.  Again, not sure how that's possible or
what the issue might be.

Does anyone have any other ideas?  Can anyone direct me (or offer) security
consulting services to help take a look?  Is there any other information I can
provide?  This is the first time I've really dealt with this and my blood
pressure is through the roof... thanks very much guys.


-Cliff Meyers