System Panic or Crash

[YLTan]

Does the HPUX system have a chance to execute any of the /sbin/rc*.d scripts when a system panic or crash happen?

I am interest to know if it is possible to execute a scripts in /sbin/rc*.d to email an alert should this happen. If this is not possible any suggestion would be appreciated.

thnks

[Patrick Wallek · Accepted solution]

No. Typically when a system Panics or crashes it just reboots and NONE of the rc?.d scripts are executed.

The only things I can think of is to have something that monitors the system and page you whenever it goes down. HP Network Node Manager or Vantage Point Operations can do this.

[Michael Steele_2]

System alerts may be what you're looking for. Since panics are displayed on the console and console alarms can be rerouted over the network to another server, use the alert configuration option in /etc/syslog.conf.

Change:

*.alert /dev/console

-to-

*.alert @server_name.domain.com

[Steven E. Protter]

After kernel load(boot) crash dumps are configured as follows.

/etc/rc.config.d/savecrash

Default has the variable not set to 1, you should uncomment that line and the default locatoin, /var/adm/crash

Save, restart system.

Then if you have crashes after kernel load, crash dumps will be created.

I'm attaching a procedure to show you how to analyze a crash dump with the q4 analysis, which would be useful.

SEP

[Eugeny Brychkov]

You can accomplish your goal by modifying script which saves crash dump after crash. So the scenario will be: server panics, saving crash dump into swap area, then reboots. During boot system detects that swap area contains crash dump and saves this dump into /var/adm/crash. This is a point where you can insert your command which will do something - sends email (I'm in doubt system can send email at this run level) or sets some flag which will be read later at run level 3 and email can be sent then
Eugeny

[melvyn burnard]

When a system panic or crash occurs, it goes down, simple as that.
It cannot run any of the rc scripts, so htere is no point in trying to look for something like that.
However, some crashes do sometimes have symptoms appearing in the syslog file BEFORE the crash, so you could monitor this, but the variations that you may have to cater for are enormous.
Another way would be to have a script run at boot time that check to see whether a crash dump has been taken, and email an alert at this time.
Again, there are occasions when no crash dump might be saved, so it does not cater for 100% of failures.
You could also have something that checks hte /etc/shutdownlog when hte system reboots, looking for anything other than a reboot command

[George Liu_2]

If system is in panic, it cannot do anything in init level. However, it still put mesg in the mesg buff which is a ring, which allows error mesgs to be displayed after reboot. Setting up a remote syslog may allow you to get some info in a remote syslogd server before the local machine entering panic.

[Frank Slootweg]

If you want to be notified if the system rebooted without you knowing it, then you can make a crontab(1) (or other method) invoked script which monitors the output of uptime(1) (or some similar piece of information).

This is a snippet from my script (bad, but working, code):

uptime_info=`uptime`
contains_day=`echo $uptime_info | grep day`
no_of_days=`echo $uptime_info | cut -f3 -d' '`
if [ -z "$contains_day" ]
then
no_of_days=0
fi
if [ -z "$contains_day" -o "$no_of_days" -lt 1 ]
then
cat <<-E@O@F | mailx -s 'desired subject' desired_user
[Desired text]
E@O@F
fi