Hacked!

Posted by jerry as Hacking, Security

I don’t think a lot of people who run sites like to admit this, but I was just hacked.  It was extrodinarily interesting… I had the advantage of watching it happen. 

 I have a server at a colo running Freebsd and cpanel.  I host a hand full of sites, and keep telling myself “someday, I’m going to get serious about it”, but life keeps getting in the way.  I personally run a bunch of sites on that server, including this one. 

So anyhow, after I wake up, I grab my laptop from the side of my bed and check email, weather and the usual stuff.  As noted in a previous post here, I’ve been trying to increase the number of visitors to one of my sites – syslog.org, so I had a tail -f running on the apache log file.  Nothing out of the ordinary at first.  Then, if I had hair, it would have been standing up – I saw requests to part of the site that clearly had filesystem path’s in them.  Paths for other accounts on my server.

So, sure enough, when I looked more closely, the requests were going to a file called “public_html/forum/avatars/.php”.  The site is running the Simplemachines forum software.  I copied one of the requested urls into my browser, and BAM! a really nice control panel for my server, the main part of which is a filesystem browser.  Yikes! 

I first moved the .php file to a directory outside of the web path then I logged into WHM and suspended all of the accounts and then started looking through logs to see what had been done.  It was clear that the attacker was going through the public_html directories for all of the sites and replacing files with “Hacked by XXX”.   But only a few files.  I was able to find each file that had been replaced by referencing the logs.  A check of date/time stamps and a comparison against a backup verified that. 

I went back to the avatars directory on the syslog.org site, and found a file called erne.txt.  When I open it up, it’s a file used to bypass PHP safe mode.

I want back, changed passwords, replaced files from backup, and whatnot.  I did find a suspicious user account “courier” in /etc/password.  What made it suspicious is was at the bottom of the file.  I deleted the account and searched for any files owned by that account.  I logged into my dev server and found that the same account existed in the same place, so that was a false alarm.  Thankfully.

So, I went back to the monthly web logs and decided to see how long .php has been on the system.  I found that it was first accessed on Oct 20.  I grepped  through the log for the IP that first accessed and saw that it had come to my site via a google search for “powered by SMF”.  Then I remembered (!) that on Oct 20th, I had been looking at my logs as well, and saw that very request, which prompted me to make sure I was up to date with SMF.  Of course I wasn’t – I was runnig 1.1.2 and 1.1.4 has recently been released.  So, I did the upgrade and didn’t look back.

After looking at the logs a little more, it became evident that they were linking from the site http://turk-h.org.  That site is a defacement content tracking site.  The few sites that they were able to register show that the hack was “political”.  The group that is responsible for the attack has a site here: WwW.BilisimSecurity.Com.  Why is the middle “w” always lowercase on hacker sites??? 

So, here are the lessons I’ve learned (so far):

1. ALWAYS ALWAYS ALWAYS keep your software up to date
2. A correllary to #1 above – a consistent approach to verifying that all software is up to date is desperately needed.
3. Open source web apps really really really need to stop adding a “powered by” line at the bottom of websites that use their software.  This makes it incredibly easy and efficient to find sites to hack.
4. Keeping your logs is really important.  They can in really handy today.
5. Using phpsuexec would have drastically limited the impact of the attack.
6. An automated method to determine if something has changed or been added in a directory tree would have provided me with almost 2 weeks of notice to resolve the issue.
7. The hacker tools are quite sophisticated.  I have retained copies of their tools.  I will not be making them available as I don’t want to spread them any more than they already are.
8. A tool that parses the apache logs for filesystem components (outside of the public_html tree) would have provided a notice that something has happened, though it would really have been too late to stop the attack.