Why system reboot?

[yyghp]

One of our servers ( rp5470 ) rebooted automatically about 1 hours ago at Dec 12 16:57, I checked syslog and noticed that nothing special in OLDsyslog.log ( the last updated time is Dec 12 16:11, more than half an hour before the reboot ), and syslog.log looks fine:

Dec 12 16:57:13 srs075 syslogd: restart
Dec 12 16:57:14 srs075 vmunix: gate64: sysvec_vaddr = 0xc0002000 for 2 pages
Dec 12 16:57:14 srs075 vmunix: NOTICE: autofs_link(): File system was registered at index 3.
Dec 12 16:57:14 srs075 vmunix: NOTICE: cachefs_link(): File system was registered at index 5.
Dec 12 16:57:14 srs075 vmunix: NOTICE: nfs3_link(): File system was registered at index 6.
Dec 12 16:57:14 srs075 vmunix: 255/255 swspBus
Dec 12 16:57:14 srs075 vmunix: 0 sba
Dec 12 16:57:14 srs075 vmunix: 0/0 lba
Dec 12 16:57:14 srs075 vmunix: 0/0/0/0 btlan
Dec 12 16:57:14 srs075 vmunix: 0/0/1/0 c720
Dec 12 16:57:14 srs075 vmunix: 0/0/1/0.7 tgt
Dec 12 16:57:14 srs075 vmunix: 0/0/1/0.7.0 sctl
Dec 12 16:57:14 srs075 vmunix: 0/0/1/1 c720

And I also checked GSP log, couldn't find any symptom about such reboot ( using "SL" ).
So, how can I know what made such reboot?
Thanks!

[Dave Olker]

Hi,

Did the system panic or simply reboot? If it paniced there should be a dump in the /var/adm/crash directory with the details of the panic. Please check the /var/adm/crash directory to see if there is a crash dump to analyze.

Regards,

Dave

[yyghp]

it was simply reboot.
no files under /var/adm/crash

Thanks!

[Denver Osborn]

Have you looked at the,,,

/etc/shutdownlog
/etc/rc.log.old
/var/adm/syslog/OLDsyslog.log

Also from the GSP, the server logs... (sl)


-denver

[Denver Osborn]

Have you looked at the,,,

/etc/shutdownlog
/etc/rc.log.old
/var/adm/syslog/OLDsyslog.log

Also from the GSP, the server logs... (ctrl + b, sl)


-denver

[Sameer_Nirmal]

Hi,

The system could reboot on account of software panic , mechine check, HPMC/TOC.

You need check following files as well.

/etc/shutdownlog
/var/adm/syslog/OLDsyslog.log
/var/tombstones/ts99 & ts98 ( Look for HPMC infomation matching the timestamp when the reboot occured )

Connect to GSP using either serial console or lan console and check the log entries using
GSP > SL > E > n
GSP > CL

[yyghp]

As I have mentioned in my previous post, I have checked /var/adm/syslog/OLDsyslog.log, nothing special ( the last record in it was something about ftp login, more than one hour before the reboot ).
And I have checked GSP, couldn't find any useful symptom for the reboot.


/etc/shutdownlog:

...
...
16:57 Wed Sep 7, 2005

( Why there is no record here for the reboot of Dec 12 ? )


/etc/rc.log.old:

Old /etc/rc.log moved to /etc/rc.log.old

**************************************************
HP-UX Start-up in progress
Wed Sep 7 17:03:36 EDT 2005
**************************************************

( This log file only has the info. for the reboot of Sep 7, can't find something special )



/etc/rc.log:

**************************************************
HP-UX Start-up in progress
Mon Dec 12 16:57:02 EST 2005
**************************************************

...
...

**************************************************
HP-UX run-level transition completed
Mon Dec 12 17:00:06 EST 2005
**************************************************

( This log file has the info. for the reboot of Dec 12, can't find something special )



# pwd
/var/tombstones
...
-rw-r--r-- 1 root root 14718 Sep 7 17:03 ts98
-rw-r--r-- 1 root root 14718 Dec 12 16:57 ts99

I have attached "ts99" within this ticket, please help me to take a look, thanks!

[Fabio Ettore]

Hi,

since no messages anywhere and no dump can be analyzed it is hard to say what happened exactly. Anyway since ts99 is dated on yesterday (Dec 12nd 16:57) just before of reboot then I should think it is the only one possibility to know why system rebooted. I cannot help so much with it...I hope other people can help better on that, otherwise I would suggest you to open an HW call by HP to analyze ts99.

Best regards,
Fabio

[Patrick Wallek]

If /etc/shutdownlog does not show anything, and there is nothing in /var/adm/crash then you are pretty much out of luck.

One thing comes to mind in this situation though -- POWER.

It sounds like your machine completely lost power. That would result in a HARD shutdown with no chance to log any messages anywhere. If you have a UPS attached to this machine, check its logs if possible and see if there were any interruptions in power.

[Sameer_Nirmal]

Hi again,

The ts99 file doesn't show any HPMC chassis codes logged or any other error.

If the error could not be found anywhere , then the possible cause could be "power" as Patrick said.

If you want you can place a h/w call to HP to make sure everything is OK. It may be a case for the system board (A14323) you have, which doesn't log any h/w error anywhere.

Although it may not be related to current issue, but you need to upgrade the system firmware to version 44.12 and GSP firmware ( if not to latest level ) as well.

[Rory R Hammond]

Everyone mentioned the usual places.
1. OLDsyslog
2. OLDmessages (if you Keep them I have the crontab entry
5,15,25,35,45,55 * * * * /usr/sbin/dmesg - >>/var/adm/messages )
3. /var/tombstones
4. /var/adm/crash
5. /etc/shutdownlog

Lastly you might look at your accounting files.
/var/adm/pacct?? You might get a hint on what program were running at the time.

Rory

[dirk dierickx]

cehck with the 'last' command if there is anything of use. (you never know)