script to analyze web logs for intrusion attempts?

[Anna Fong]

I'm looking for a perl module or script that can analyze web logs. In particular, I want to analyze the number of 404 errors generated from any particular host in my web access log. For example, if example.com does 300 accesses to my website and 200 of those are 404, I want to know that. Either the user from example.com is finding a bunch of broken links (highly unlikely) or is attempting a hack (more likely).

I want to capture each unique IP/domain, the number of accesses for each in the period defined by the web access log, and the number of 404 errors for the particular IP/domain. If the ratio of 404 errors to total accesses per IP/domain is too high, then I'll block the suspect IP/domain.

Is there something like this already available?

TIA,
Anna

[Steven E. Protter]

It is hard to be certain that someone is trying to hack your web servers.

There are telltale signs and there is software to help you check:

http://www.thefreecountry.com/webmaster/loganalyzers.shtml

Here is a potentially helpful blog.

http://www.sysinternals.com/blog/2005/10/sony-rootkits-and-digital-rights.html

You can probably do a good job with open source tools.

Attempting to access a broken link or file that does not exist is not necessarily a hack attempt.

A non-hacking attack example is manipulating the httpd server to send out browser like sessions to websites outside the domain of the httpd server. This seems to be an attempt to make money off pay per click vendors.

SEP