Important system log files and key words

[Robert Milne]

Hi all,

I am just trying to compile a definitive list of all the important log files to monitor (like syslog.log), together with their respective standard locations necessary to warn of impending system problems or critical events.

Also the necessary keywords to select by a regex for important system messages warning of doom within the logs respectively, so I can create an automated monitoring script with Perl or whatever to make our sysadmin monitoring more effective. Of course if anyone already has any tools like this it could be helpful too !

The system is an L1000 running HP-UX 11i.

Any info., links, guidelines, advice or whatever appreciated.

All the best.

Rob.

[RAC_1]

We use sendmail to send us the last 200 lines of syslog file at defined intervals.

In addition to that you can also grep for errors like vmunix, LPMC, HPMC etc. and mail it to you.

Dmesg is also important. you can append dmesg output in syslog.

[john korterman]

Hi Robert,
SAM has a function for trimming system logfiles (Routine Tasks - System Log Files): this very detailed list is kept in /var/sam/rt/.log
Depending on what else you have on the system, there might be logfiles also under /opt

regards,
John K.

[Chris Wilshaw]

check for these [not an exhaustive list]

/var/adm/syslog/syslog.log

lbolt (indication of disk/tape drive problems)
SCSI: Reset (as above)
fork: Not enough space (out of processes)
table is full (nfile/ninode exceeded)
AC POWER FAILURE (UPS monitoring)
EMS Event Notification (error from EMS system)
Temperature exceeded (obvious)
cmcld (Service Guard errors)

/var/adm/syslog/mail.log

unable to qualify my own domain name (system name resolution problem)
WARNING: dangerous write permissions (permissions risk on file specified)
Name server: your.host.net : host not found (system unable to resolve mail target address)


/var/opt/resmon/log/event.log (EMS monitoring)

Hardware failure (failure of listed component)
power supply has failed (obvious)
Drive is not responding (listed disk problem)

/var/adm/rc.log (system boot)

ERROR
FAILED
EXIT CODE:

All 3 of the above can indicate failure of a startup script.

Any application logs - errors would depend on the application.

[Trever Furnish]

Rather than write your own script, why not adapt the logwatch script commonly used on Linux systems? It's just a perl script, very flexible, very nice, very modular, and it's easy to write your own modules for a specific type of log file or to modify the modules it comes with.

If you have a redhat linux system handy, it's installed by default: rpm -ql logwatch

Main site is:
http://www.logwatch.org

For the most part it works not by listing what to watch for, but rather by listing what NOT to watch for, which is much more likely to catch problems.