Strange happenings at Microsoft’s Search Engine

Posted by jerry as Uncategorized

So, with all of the looking at logs I’ve been doing lately, I noticed something really unusual.  Visits from hosts in the same range as the normal MSN bot that appears to be downloading whole pages (css, images, etc), which are referred from a search that looks like this:

http://search.live.com/results.aspx?q=myself&mrt=en-us&FORM=LIVSOP

 The keywords keep changing, and they are landing at different sites.  My theory is that MSN is going through an validating the search results for different terms, and analyzing the whole page - probably in an attempt to stop spam.  I suspect that it is running through the entire index of terms that are associated with the site and “viewing” each in turn.

I’m seeing this activity for all of the sites I own and manage.

 This appears to have been going on for about a week now.  Hopefully MSN is on to something that will help reduce the amount of spam in their search results.

Insight into web attacks

Posted by jerry as Hacking, Security

I’ve been fairly attentive to my web server logs lately.  I noticed something strange this morning in the logs for syslog.org:

200.88.114.166 - - [12/Nov/2007:10:21:21 -0500] “POST /forum/index.php?action=quickmod2;topic=249.0 HTTP/1.1″ 200 11257 “http://www.syslog.org/forum/index.php?PHPSESSID=5537d4fd7661fd20c7aa6aa8b3f05a05&topic=249.msg2124” “Opera/9.0 (Windows NT 5.1; U; en)”

Now, all day long, bots and sites are trying to submit stuff and run automated hacking tools.  This entry caught my eye because it had a session ID.  Not sure why it caught my eye, but it did.  So, I grep for the session id and get this other line:

69.46.16.2 - - [12/Nov/2007:10:21:14 -0500] “GET /forum/index.php?PHPSESSID=5537d4fd7661fd20c7aa6aa8b3f05a05&topic=249.msg2124 HTTP/1.1″ 200 22114 “-” “-”

Grepping the log files for both IPs only gives those two lines.  What I find particularly interesting is that they are clearly different sources, but obviously working together to somehow do something.  Then I decided to look at a longer term - 3 months.  There were a total of 801 hits that included that session ID, going all the way back to the start of the logs.  The sources where from hundreds of different IP addresses.  Some of them running through a battery of tests, but most of them only occuring once.

A search of google for the string turns up nothing - though it probably will now that this article is up.  So, what does that tell me?  At some point long ago, someone probably logged into SMF and included the resulting URL into a database that is used by bots to launch attacks from. 

More on why Google will probably not buy Sprint

Posted by jerry as Uncategorized

So, I thought of a few more issues with this.  The report that prompted the acquisition rumors of Sprint, notes a cancelled partnership with ClearWire.  Why would Sprint back out?  Google is going to buy them, of course. 

Well, not quite.  They may well back out because they were being ACQUIRED, but not necessarily by Google.  BT apparently has it’s sights set on Sprint as well.  So, it’s possible that the source of the rumor is a tie up between BT and Sprint. 

At the very least, this is likely to result in a bidding war for Sprint between Google and BT.  In that event, the $50B that I cited below is probably more like $55B or $60B.  That just makes the deal much less likely to go in Google’s favor.

Can Ron Paul be elected?

Posted by jerry as politics

The answer is: I don’t think so.  I’ve grown to equate the presidential selection process with the top 3 videos on each week’s edition of “America’s Funniest Home Videos”.  You sit and watch the show, and at the end, the producers pick three videos which will be voted on to determine the winner of $10,000, but the videos that are picked are typically lame and definitely not the funniest of the show, but the audience is forced to vote among that set.

The presidential race is not entirely different.  We sit and watch a lot of really interesting candidates, but in the end, we’ll end up having to choose from a select few that the “producers” have hand picked for us - Guiliani &  Romney on one side, Clinton and Obama on the other side.  The media is really the group that determines who gets to play and who doesn’t.  I don’t believe that they have a hidden agenda, are working as part of some covert society, or the like, just that they all want to focus on who they think are going to be the top contenders. In the process, they impact the outcome of the races.

 I was looking at the Ron Paul Money Bomb, and they have a great list of Ron Paul “commandments”:

01. End the Patriot Act Immediately.
02. End the IRS.
03. End the Homeland Security Department.
04. End the Iraq Police Action.
05. End the private ownership of the Federal Reserve.
06. End the national ID Card program.
07. End the sending of jobs overseas.
08. End the flood of illegal immigrants.
09. End preemptive military strikes.
10. End the “future” police state.(a)(b)
11. End nation building & defend OUR nation, not others.
12. End executive war powers. Congress alone can declare War.
13. End the North American Union. Preserve US sovereignty.
14. End the War on terror. Terror is not tangible. Therefore it cannot be won [EVER].
15. End the onslaught of our freedoms and liberties by the Federal Government.

 I definitely want to see most of these put into place.  They are all noble and sensible, but there is so much inertia behind almost all of them, that even if Paul were to be elected, I have no confidence that any of them could be accomplished.  However, there is value in slowing down the course that we’re on, so it might not be all bad. 

In the end, I believe the contortion of Paul’s views by opponents and media, such as abortion and affirmative action, will be his undoing.  I share them, but they will inevitably come across as “minority hating”, “women hating”, etc, even if that’s not the case.

Growing dissent of Ron Paul

Posted by jerry as politics

It’s been interesting watching the groundswell of support for Paul over the past year.  His republican affiliation and anti-war, pro-liberty beliefs are very appealing to many who feel the current administration has been assaulting all that is American.  But now, people are starting to realize that he is a conservative libertarian.  As a libertarian, he has views that the federal government should not be in the education business, it should not be mandating racial quotas or bailing out municipalities after disasters.

I find it really funny that the author of the article in question is clearly anti-Iraq war, yet wants the US to go in and kick some butt in Darfur. Ron Paul’s position is that we don’t go to war unless the threat is against us.  See, it’s a consistent view.

It should also not be suprising that he attempts to pass legislation to stop abortions.  He is, after all, a conservative.  Protecting the lives of a countries citizens is one of the core tenants of a libertarian, so that would appear to be in the perview of the federal government.  You can debate the issue of whether or not a fetus is a life, but in his eyes, at least, it’s clear and he is fighting for what he believes is right.  I’m personally pro-women’s choice, but he isn’t.  His philosophy aligns with mine in many other key areas, though. 

Paul’s desire to do away with the Department of Education is being taken as a anti-education stance.  I can’t believe that for a minute.  I do believe that his stance is that education is a State and local affair, and that the Federal government only screws things up.  History, since the creation of the DoEd, seems to back up his view.

Likewise, hiw view that affirmative action should be abolished is taken as an anti-minority stance.  Again, I can’t believe that is his intent.  Like me, I believe his intent is that discrimination should not be tolerated, and when found, should be prosecuted, but federally mandated minority quotas are not appropriate.

Paul’s view that the Federal government should not have provided aid for Katrina victims is also a hot button. Again, his view is likely not that the people should be assisted at all, just that the federal government is not the right group to do so.  The states  themselves should be doing this sort of thing.

So, in short, it seems that many people are awakening to Paul’s conservative views, and “jumping off the Ron Paul bandwagon”, which will ensure that we continue to have many more years of the same kind of crap that we’ve been experiencing up to now - continued erosion of liberty, increasing our military activities abroad, bigger deficits, more taxes, lower currency value, and on and on.  Good luck with that.  Glad you’re love of the federal programs is worth the complete destruction of the US.

Google to buy Sprint?

Posted by jerry as Uncategorized

Google’s open wireless phone initiative has been discussed frequently, but apparently now there is rumor of Google wanting to buy Sprint.  Certainly it seems to make sense as a way to jumpstart things, but I see a couple of critical flaws that will almost certainly render this just a rumor:

• Sprint (S) has a market cap of about $46B USD.  Google has about $18B USD of assets, and a market cap about over $200B USD.  Sprint would cost more that Google can afford.  Since Google is not in the business, there won’t be synergies similar to other telecom rollups.
• Financing of what would likely by a $50B deal will be very hard to come by in the post apocolyptic credit market. 
• Organizationally, Google and Sprint would tear each other apart, if combined.  The old telecom culture of Sprint would likely be untenable in Google’s fast paced world.
• The bits that I’ve read about what Google wants to do with their phone service leave me wondering if owning Sprint really buys them anything.  If all of the infrastructure needs to be replaced to support some fundamentally new technology that costs say $20B to implement, why would you first spent $50B on a phone company then immediately dump $20B more into gutting and replacing it’s infrastructure. 

So, to me at least, this doesn’t add up.  Given that the stock market is in the toilet, I suspect that if Google did announce such a deal, it would be a boost to the overall market, but I suspect that Google would suffer a beating on Wall Street if they did.  In fact, I would expect rumors of the acquisition to have a negative impact too.

Bad deal.  Buy a start up with the tech you want.  Hire away the talent needed from ATT, Sprint, etc.  Just don’t buy a dinosaur.

My deer crash pictures are apparently pretty popular

Posted by jerry as Uncategorized

This blog doesn’t get a lot of traffic, but I noticed that nearly all of it is to look at the pictures of my car accident.  Many of them are referrals from google for people searching for images of car and deer accidents.  That’s pretty morbid.  My picture is on the second page of nearly all the search results, which means that there are LOT of people searching for pictures of car crashes with deer.  It’s also apparently going around in an email.  To be honest, it’s not really that impressive, but since it seems to be what people want to see and I’m all about delivering, here you go:

Web Site Security Guide

Posted by jerry as Security, Uncategorized

So, in the aftermath of my little incident, I decided to write an article for NetworkStrike on how to keep your website secure.  I’ve included it below: 

Background

The “cost of entry” to run a website is very low now, and we have seen an explosion of small sites on the Internet. Supporting this trend has been the availability of free and open source applications that allow site owners to do nearly anything they want, from blogs to wikis to forums. 

Unfortunately, nearly all of the software has inherent bugs that can be exploited by bad people. The exploitation of such bugs has matured in a similar fashion to the applications themselves.  Compromised web sites are a foundation to many threats currently on the Internet. Site owners are unknowingly supporting illegal activities such as:

• Hosting warez
• Hosting copyrighted files
• Hosting tools to compromise other sites
• Sending massive amounts of spam
• Host phishing sites to steal money
• Become part of command and control for botnets

Running a web site properly is truly becoming a social responsibility. If you are not part of the solution, you are likely part of the problem. 

This guide outlines some general and basic security measures that you MUST take if you want to make sure your site is not defaced or otherwise compromised.

Key elements needed in a web hosting platform

Try as you might, if another site on the server you share does not follow prudent security measures, then your site it still vulnerable. Suphp executes php scripts as the user-id and group-id of the account – not the userid/group id of the web server.

This will allow more restrictive permissions on your directories and will prevent an attacked from being able to read your php files.

Some hosts will not want to use suphp, as it can cause performance problems.

Build a relationship with provider on dealing with security issues

Firewall outbound connections

 This is not always practical or possible, but firewalling traffic originating from the server – other than what is expected – can prevent backdoor tools from phoning home.

Run suhosin patch for php

Suhosin can stop the exploitation of many different types of coding weaknesses and some vulnerabilities in php itself.

Keeping up with the web software you use

• Subscribe to mailing list for any application software used as part of your site.  Nearly all such applications have an “announcement” list that is low traffic.
• Attempt to participate regularly in the forums or discussion lists for the software you use. You will often get advanced warning of security issues and interim fixes.  You will also know if the software is at risk of being abandoned and not updated.
• Upgrade after a security fix release within 3 days.  Vulnerabilities are often exploited soon after they are discovered.
• If an application you use becomes abandon, plan to migrate your site to a new tool as fast as possible.
• Try not to customize software.  This will slow down your ability to patch or upgrade, because you’ll have to rewrite code, test, etc, and that can be difficult to do when it’s not on your timeline.
• Attempt to remove ‘powered by’ and version number references.  Nearly all open source web applications proudly, but this has turned out to be an extrodinarily efficient way to systematically identify sites running vulnerable software. 
  
• Create a file with a random name in your html directory. Do not reference the file in your site. Grep your web logs daily for the file being accessed. Any hits may indicate that your site has been compromised.
  
• Scan for file differences and notify of differences
  

Your responsibilities as a site owner

• Perform frequent off-site backups.  Even if you have a local backup strategy, such as to another server, it is imperative to maintain an offsite backup.  While it doesn’t happen often, datacenters are broken into and servers are phyically stolen.
• Do not store sensitive data like credit card numbers.  Just don’t do it.
• Change account passwords regularly

Nukes…

Posted by jerry as World, politics

Just read this article on reddit, and my head is about to explode.  The author clearly writes with an authoritative air about military operations, and seems to have his shit straight with a believable story.  Trouble is, it’s got holes.

The author makes a reference to “central command” several times.  Anyone at all intimate with US military strategy understands that central command is located in Doha, Quatar, not in the Pentagon, and would not be governing the movement of nuclear weapons in the US.  That job would either fall to United States Transportation Command or to United States Strategic Command.  I really am not sure which in this particular case.

What concerns me is that this is a really good story written with a seemingly good handle on military protocol.  Yet, a basic fact like that is wrong, so what does that say about the rest of the article?  Probably not much if you really really hate Dick Cheney.  Speaking of which, I have yet to understand how good ole Dick has obtained so much control over the military as to be able to single handedly oder the shipment of nuclear weapons to a war front.  Maybe he does, but call me skeptical.

When I heard the story originally, my first thought was that of propoganda - not a mistake or any kind of internal struggle.  This administration is extrodinarily effective at managing the media, if you can say nothing else for them.  I have to believe that whatever the “real” plan is, it involved letting Iran/Syria/whoever else know that we had nuclear weapons en-route to their back door at one point, and that was just on accident.  “Just wait and see what happens when we get mad!”.  I still think that’s a pretty likely scenario. 

Hacked!

Posted by jerry as Hacking, Security

I don’t think a lot of people who run sites like to admit this, but I was just hacked.  It was extrodinarily interesting… I had the advantage of watching it happen. 

 I have a server at a colo running Freebsd and cpanel.  I host a hand full of sites, and keep telling myself “someday, I’m going to get serious about it”, but life keeps getting in the way.  I personally run a bunch of sites on that server, including this one. 

So anyhow, after I wake up, I grab my laptop from the side of my bed and check email, weather and the usual stuff.  As noted in a previous post here, I’ve been trying to increase the number of visitors to one of my sites - syslog.org, so I had a tail -f running on the apache log file.  Nothing out of the ordinary at first.  Then, if I had hair, it would have been standing up - I saw requests to part of the site that clearly had filesystem path’s in them.  Paths for other accounts on my server.

So, sure enough, when I looked more closely, the requests were going to a file called “public_html/forum/avatars/.php”.  The site is running the Simplemachines forum software.  I copied one of the requested urls into my browser, and BAM! a really nice control panel for my server, the main part of which is a filesystem browser.  Yikes! 

I first moved the .php file to a directory outside of the web path then I logged into WHM and suspended all of the accounts and then started looking through logs to see what had been done.  It was clear that the attacker was going through the public_html directories for all of the sites and replacing files with “Hacked by XXX”.   But only a few files.  I was able to find each file that had been replaced by referencing the logs.  A check of date/time stamps and a comparison against a backup verified that. 

I went back to the avatars directory on the syslog.org site, and found a file called erne.txt.  When I open it up, it’s a file used to bypass PHP safe mode.

I want back, changed passwords, replaced files from backup, and whatnot.  I did find a suspicious user account “courier” in /etc/password.  What made it suspicious is was at the bottom of the file.  I deleted the account and searched for any files owned by that account.  I logged into my dev server and found that the same account existed in the same place, so that was a false alarm.  Thankfully.

So, I went back to the monthly web logs and decided to see how long .php has been on the system.  I found that it was first accessed on Oct 20.  I grepped  through the log for the IP that first accessed and saw that it had come to my site via a google search for “powered by SMF”.  Then I remembered (!) that on Oct 20th, I had been looking at my logs as well, and saw that very request, which prompted me to make sure I was up to date with SMF.  Of course I wasn’t - I was runnig 1.1.2 and 1.1.4 has recently been released.  So, I did the upgrade and didn’t look back.

After looking at the logs a little more, it became evident that they were linking from the site http://turk-h.org.  That site is a defacement content tracking site.  The few sites that they were able to register show that the hack was “political”.  The group that is responsible for the attack has a site here: WwW.BilisimSecurity.Com.  Why is the middle “w” always lowercase on hacker sites??? 

So, here are the lessons I’ve learned (so far):

1. ALWAYS ALWAYS ALWAYS keep your software up to date
2. A correllary to #1 above - a consistent approach to verifying that all software is up to date is desperately needed.
3. Open source web apps really really really need to stop adding a “powered by” line at the bottom of websites that use their software.  This makes it incredibly easy and efficient to find sites to hack.
4. Keeping your logs is really important.  They can in really handy today.
5. Using phpsuexec would have drastically limited the impact of the attack.
6. An automated method to determine if something has changed or been added in a directory tree would have provided me with almost 2 weeks of notice to resolve the issue.
7. The hacker tools are quite sophisticated.  I have retained copies of their tools.  I will not be making them available as I don’t want to spread them any more than they already are.
8. A tool that parses the apache logs for filesystem components (outside of the public_html tree) would have provided a notice that something has happened, though it would really have been too late to stop the attack.