Why did this C3700 Reboot!!

[Govind]

Hey Everyone
This is really bugging me. Because I cant resolv why this Hp-visualize C3700 just rebooted it self, there wasnt any scheduled cron job or anything doing that. And the user confesses he didnt push the power button either, the syslog.log file doesnt have anyinfo. Is there anyway I can get a snap shot of the processes before the reboot occurred. Lot of inportant work was lost because of this, and I want to stop it from happenning again. Any suggestions will be greatly appreciated.
Thanks
Govind

[Jeff Schussele · Accepted solution]

Hi Govind,

Check

1) The /etc/shutdownlog
2) If there's an appropriately dated tombstone in /var/tombstones
3) If there a crash image in /var/adm/syslog

Any one or all of these will help clue you in.

Rgds,
Jeff

[Pete Randall]

Check the shutdown log, look for a crash dump in /var/adm/crash, tombstones, and anything the old syslog.

Pete

[Jeff Schussele]

Dooohhh...that should have been /var/adm/crash for the dump location.

Cheers,
Jeff

[Marco Santerre]

Govind,

if I were you, i'd start by checking /etc/shutdownlog to see if there was a Panic message or something similar. You might even find if someone actually rebooted the server.

then you can check to see if you have any ts99 file dated of the day and time of the reboot in the /var/tombstones.

i'd also check to see if you have any crash directory in /var/adm/crash dated of the day of the reboot.

You may want to check your ioscan and dmesg to see if you have the same amount of CPUs and memory, cause if the server found some problems, it may have rebooted itself to deconfigure a CPU or some memory.

[Govind]

Do I have to enable the shutdown Logging? Because I cannot find the shutdownlog. The /var/adm/crash is empty. There is no /etc/shutdownlog nor /var/tombstones.
I appreciate ur quick replies but I am still in the having no luck with this issue.
Thanks
Govind

[Paul Sperry]

My shutdownlog is here.
/var/adm/shutdownlog
system shutdown and reboot log

[Pete Randall]

The shutdownlog in /var/adm should be a link:

/var/adm/shutdownlog -> /etc/shutdownlog

You don't need to do anything special to enable it.


Pete

[Govind]

For some strange reason the SYMBOLIC link /var/adm/shutdownlog pointing to /etc/shutdown is a hanging out nowhere as there is no /etc/shutdownlog, In anycase,Thanks everyonefor your replies. I really appreciate it.
Have a Great Day.
Regards
Govind

[Bill Hassell]

shutdownlog will not exist by default on earlier versions of HP-UX and the default for taking a crash dump is NO. When HP-UX reboots, it is almost always a system crash. However, there will be NO log entries since the crash halts HP-UX before there is time to log anything (and possibly do serious damage to files, etc).

So edit the file /etc/rc.config.d/savecrash and remove the comment in front of # SAVECRASH=1 so it reads:

SAVECRASH=1

Next, determine where you'd like to store the crash dump (apx = the size of your RAM). Uncimment this line: # SAVECRASH_DIR=/var/adm/crash and change it to a directory with enough space. Some sysadmins prefer to leave /var/adm/crash as the destination and simply create a symlink for /var/adm/crash that points to the preferred location.

Then create /etc/shutdownlog with:

touch /etc/shutdownlog
chmod 644 /etc/shutdownlog
chown bin:bin /etc/shutdownlog

Now shutdownlog will only tell that a crash occurred (it will say PANIC) but the reason is nothing that you can use (ie, data page fault, etc). It might also state that it was due to an HPMC (high priority machine check) which is a hardware failure.

Now you are ready for the next crash. NOTE: Most crashes are due to lack of patches or an occasional hardware failure. Analyzing a crash dump to determine the right patch is very tedious and requires a lot of information not publicly available (ie, HP-UX source code). If you have a software contract with HP, open a call at the next crash and have them go through the steps to analyze your crash dump. Otherwise, download the Support Plus bundles called HWE and QPK and load thos at your next opportunity. Support Plus is available from:

http://www.software.hp.com/SUPPORT_PLUS/

[Govind]

Well, Thanks for all that rich Information, Bill. I really appreciate you sharing it with me in such detail. Kudos to you. I wish I could give you more than 10 points(Not That points would matter much, but just to show my appreciation for your help!!!) Thanks again Sir.Have a Fantastic Day.
Regards
Govind