issues with security

cliff at cliffmeyers.com cliff at cliffmeyers.com
Mon Feb 2 13:57:55 EST 2004


real quick, here's what I re-installed with after the hack:

- Apache 1.3.29
- mod_ssl 2.8.16-1.3.29
- mm 1.1.3
- OpenSSL 0.9.7c
- PHP 4.3.4 (didn't use 4.3.5RC1 since it's not "stable", but I'm going to use it
in my re-install tonight)
- and a bunch of other latest versions of source code, including zlib, libpng,
libjpeg, freetype, mysql and pgsql

The box is firewalled by a Sonicwall hardware firewall.  The only ports open to
the WAN are 80 and 443.  The only other services running on the box are sendmail
(bound to localhost, used by PHP) and FTP which we IP restrict.  Right now there
are only 4 known IPs allowed to connect via FTP and they're all static IPs that
our clients use.

Now, the stupid choice I made was probably to use a backup of the webroot that
was made after the system was compromised.  I suppose it's possible they left
something in the web root that was allowing them to hack their way in.  So
tonight, I will be re-installing again and making the following changes:

(1)  Using PHP 4.3.5RC1
(2)  Using a backup about 3 weeks old, before the system was compromised

Someone suggested that I run "chkrootkit" from chkrootkit.org but that didn't
flag anything as being bad.  I changed the root password and root passwords on
all other machines as soon as this happened.  As far as I can tell, the other 2
web servers I'm running (Apache with ColdFusion) haven't been touched.  This
leads me to believe it was a PHP exploit.

Mark, what kind of "bad code" in PHP might open this server up for attack? I
don't have any CGIs or PHP code doing any kind of suexec type stuff, although a
bunch of scripts do allow file uploads (although most of them are behind
HTTP-auth or PHP session-based authentication anyways).

Thanks again everyone for your help and comments!  Any other ideas would be
*greatly* appreciated.


-Cliff



On Mon, 2 Feb 2004 13:20:17 -0500, "Mark Musone" wrote:

> 
> There is an exploit for apache handling chunks, and also for mod_ssl.
> 
> Depending on whats running under php, there could be some bad
> programming, however it's most likely it's an apache thing. What else
> are you allowing through your firewall?
> 
> Although doing security audits and security monitoring for our clients
> is a core part of our NOC group, we generally don’t do it as an outside
> professional service, however if you are interested, feel free to email
> me and I can see if we can set something up.
> 
> 
> 
> -----Original Message-----
> From: owner-nflug at nflug.org [mailto:owner-nflug at nflug.org] On Behalf Of
> cliff at cliffmeyers.com
> Sent: Monday, February 02, 2004 12:46 PM
> To: nflug at nflug.org
> Cc: Darin.Perusich at cognigencorp.com
> Subject: issues with security
> 
> Hi Everyone,
> 
> 
> Apparently my first message didn't go through, so here I go again:
> 
> 
> I've been away from the list for a little while, but been having a major
> problem
> here at the office so I figured I'd post to see if you guys had any
> ideas...
> 
> On the 22nd we had an issue with one of our systems that I thought had
> to do with
> some kind of hard drive error.  The system is a Red Hat Linux box,
> running
> primarily Apache and PHP to serve web sites.  I typically compile these
> things
> from source so I can have a little more control over configurability.
> 
> Anyways, as it turned out, I noticed late last week that there were
> processes
> running that shouldn't be there.  After I killed the processes I noticed
> files in
> the /tmp directory, where PHP stores most of the session files (unless I
> tell it
> to store them somewhere else).  There was a 'blackhole.c' file and some
> other
> things which had been compiled to run on my system.
> 
> I talked to my other programmer and we were going to come in Saturday to
> do a
> full re-install, but the hacker struck against Thursday night around 11
> PM and
> defaced all of our sites.  I came into the office and spent the next 8
> hours
> formatting, installing Red Hat 9, download all the newest source code
> for Apache
> and PHP, and getting everything set up.
> 
> Well, I get into work today, and guess what?  Another bad process and
> more files
> in the /tmp folder.  I killed them all again, and am going to do
> *another*
> reinstall tonight.  I was e-mailing a colleague asking for his input so
> I'll post
> a few of the ideas I had for how the hacker got back in.  Here they are:
> 
> (1)  I used the latest stable version of PHP, 4.3.4, when in fact there
> is a new
> version, 4.3.5RC1.  I wanted to avoid a release candidate version but
> that would
> be my first guess.
> 
> (2)  Some other vulnerability I don't know about.  I installed the
> latest version
> of every other package so that's probably unlikely. Every other service
> is
> firewalled, so...
> 
> (3)  I used the web backup from the morning of the 30th so as to not
> loose any
> changes - perhaps there was something in there that it allowing the
> hacker to
> gain access again.
> 
> (4)  A problem with some of our PHP code.  Again, not sure how that's
> possible or
> what the issue might be.
> 
> Does anyone have any other ideas?  Can anyone direct me (or offer)
> security
> consulting services to help take a look?  Is there any other information
> I can
> provide?  This is the first time I've really dealt with this and my
> blood
> pressure is through the roof... thanks very much guys.
> 
> 
> -Cliff Meyers
> 




More information about the nflug mailing list