Centralized extendable secure logging application?

[TwoProc]

We've got logs everywhere of all kinds of things, the more merrier I always say. It really gets unwieldly to manage them, keep them secure, know who's reading them, recording access activity, and differentiating that from the recording of the new data itself, etc.

Requirements now state that I need a log of who accesses audit logs, who changes audit logs, who initializes audit logs, etc. Rather than writing such for each and every type of log for each and every type of area (system, network, database) - which is what we've got now, with SOME centralization, I'd like an extensible, centralized, secure, auditable piece of software that I can send data to, that organizes collected data by type and subtype, provides time stamping, CRC certification, update protection, as well as purging and archiving, and review.

Whew, I *know* that's asking a lot - but SURELY someone has done this, or most of this already.

Suggestions for all or part of solution, as well as commercial and open source are appreciated.

Thanks!

P.S. Plz don't just suggest a syslog server, I've got that already, and it misses way too many of the requirements.

[Bernie Vande Griend · Accepted solution]

We have that same scenario. But we use syslog services to a specific syslog server that we lock down access to bigtime. We run just the minimum services on that server and only allow just those that need access to their logs. Then we have a bunch of scripts that moves the files to the appropriate locations on the server so we can lock down those filesystem to only those who need to read them. We also have a read-only type device on the server to write logs for the critical apps where we absolutely have to have a "legal" copy of them. We also write cron jobs to handle the archive, compression, rotation of each apps log files. Sometimes it can be tricky separating the apps as they come into syslog, but we've been able to work that out with the delegating the types to certain apps and using script to cut things out into their own logs once on the syslog server. There might be something better out there but we didn't see anything that met our needs so we used syslog and tweaked it.

[Ian Kidd_1]

There's a whole class of software that was initially designed to be a security tool, but has changed rolls with the advent of HIPAA and SOX - security event management tools (SEMs). Most of them, like RSA Envision or Intellitactics NSM, include log retention functionality, including log encryption, hashing to prove that the log data was untampered, separating by OS and IP, backup and archiving, ability to look at past logs, etc. They will also allow sending of email alerts/snmp traps etc in the event something is seen in the logs and the creation of custom reports on the log data. HOWEVER
*most SEMs' parsing is still weak since the parsing wasn't developed by admins who know an os/app well but by developers on limited data
*EXPEN$IVE (6 figures minimum and can run over a mil for a large enterprise
*requires a whole new set of indepth training to use
*usually wont work in a DHCP environment
*may require a substantial extra charge to have a vendor engineer come onsite to assist with log collection (especially with apps)

depending on your needs, this might be overkill (especially if you don't need the alerting functionality). you didn't want me to suggest a syslog server, but a syslog server running a slightly better syslog application than native syslogd, such as syslog-ng, coupled with a LOT of scripting might be cheaper and can be tailored to your environment quicker.

If you're interested in SEMs, you might want to attend the RSA conference coming up real soon and get product demos and speak with a lot of the vendors. But be prepared for sticker shock!

[TwoProc]

Sounds like the real solution here is to write a nice one myself, and go to the show and sell it for big $$$ and let others have the sticker shock! :-)

Of course, we won't tell anyone that it would be written in MS Access... :-)

[TwoProc]

Wow, so everyone going through compliance is hacking up their own?

Sigh...

OK, how did you accomplish the requirement that all changes (as in edits) to the audit log be visible, yet not show up on reports when the file just grows?

Tripwire? Can you specify in it that it should alert for changes, but not for file size growth?

I'm losing my little mind over the requirements for review and security of the audit logs themselves.

I've tried just drinking lots of beer, and while fun and all, it's just not working - at least for compliance.

[VK2COT]

Hello,

a) As part of SOX compliance, I worked on
project to introduce Symantec Enterprise
Security Manager for several fortune 500 companies.

I do not work for Symantec but it seems the
customers are quite happy with the solution:

http://www.symantec.com/business/products/overview.jsp?pcid=2242&pvid=855_1

Albeit, these solutions are not cheap!

b) SOX is quite vague and therefore,
solutions to met its requirements can do
(or not do) many things.

That is typical for any legal document :)
Their Fog Index is high on purpose so that
everyone else gets puzzled when trying to
interpret the results.

c) I heard about several companies who could
not afford high cost of vendor-based
solutions and ended up writing home-grown
scripts and tools.

d) From my colleagues, I hear Tripwire is
very good too.

e) I have also heard of these solutions:

Enterprise Configuration Manager (ECM):
http://www.configuresoft.com/ecm.aspx

MasterControl SOX:
http://www.mastercontrol.com/solutions/sox_fb.html

Openpages SOX Express:
http://www.openpages.com/

SOXLab:
http://www.soxlab.com/

... and there are many others...

Of course, everyone has a different opinion :)

An interesting read:

http://www.controlcase.com/Downloads/Why_SOX_Tools_Fail.pdf

Cheers and good luck,

VK2COT

[Don Mallory]

www.loglogic.com

I haven't bought one of these yet, but it's a linux 'appliance' that apparently builds (including racking and getting it online and logging) in under an hour.

I saw a pretty good webinar on it a couple of years ago hosted through www.sans.org (what works in log management). I've read numerous whitepapers since and it's quite good.

It also grabs windows event logs and converts them to syslog form using snare (www.snare.org), which has agents for almost any other OS and you could hack yourself, but the Log Logic tool is probably cheaper considering the time you'd spend to build a system.

I believe I've also seen a Snare VMWare appliance (on www.vmware.com, which is now EMC).

The log logic one also covers compliance. Your biggest problem will be sizing to ensure you are getting all the messages if there is an issue, network bursting gets too high and you lost packets, since they're all UDP, you won't know what you've lost.