HP 9000 L-Class rebooting at will

[cnb]

Hi Gary,

Can you capture and post the GSP Error Log and Activity events from before and after the update?

Connect to GSP & CTRL ^B
GSP> SL > E (hit return enough to get all of the events), then Q

GSP> SL > A (same as above), then Q

Rgds,

[Gary_O]

Hello cnb,
With my new system firmware I wonder if I am getting more info in my gsp logs.
I replaced the power monitor card a week or two ago. I wonder if my power monitor interface has issues, or could it be the power supply itself? Not sure what the symptoms of that are.

In any case I have include the GSP error log.

Thanks,
Gary

[Gary_O]

Here is the activity GSP log.
Thanks

[cnb]

Hi Gary,

The system fans reporting issues as if the power might have browned-out. Is this system on a UPS?

The fan failures were followed by the GSP not responding, then platform (power) monitor failures. Then I/O failures and machine checks.


Before you swap anymore components like another platform module, power modules, GSP or the system board, you should update the GSP firmware to A.01.12 to eliminate the infamous GSP reset issue. Although the date isn't clear on when this happened, the entry event and codes do indicate (among other things; disk errors, fan failures, machine checks & lba failures) a GSP self-test failure and this is a known platform issue.

See this link for some great troubleshooting help on these systems:

http://h30499.www3.hp.com/hpeb/attachments/hpeb/hpsc-46/8120/1/Lclassgsptsflowchart.pdf

Check dmesg and /var/opt/resmon/log/event.log


Since the system reported a machine check it should have generated a tombstone (ts99) file in the O/S at /var/tombstones/

Might be time to log a service call after the GSP firmware upgrade.

HTH

Rgds,

[Gary_O]

OK firmware is now latest and greatest, just update GSP A01.12.

We havent had any power hits in awhile, so it
shouldnt be losing power. Seems to me I see those fan errors every time it reboots...

I just rebooted the machine myself, for a clean start: Apr 20 13:08:36 acp /sbin/init.d/sendmail[1177]: #### rebooted ####

Thanks for all your help, we'll see if she behaves...

[Gary_O]

Hello All,
New set of reboots. I've included the GSP error logs. Could this be a symptom of a bad power supply?
Thanks,
Gary

[Gary_O]

I currently have 2 disks with mirroring between the two. For the heck of it, I moved the one
disk from c2t2d0 to be in c1t2d0.
The primary disk is c2t0d0. So far no reboots.

# uptime
1:32pm up 21:46, 2 users, load average: 0.04, 0.05, 0.05

[Robert_Jewell]

Its interesting to me that you are noting the reboots by the sendmail message. Is this from the syslog files?

I would ask you attach a copy of the /var/adm/syslog/OLDsyslog.

Also run the following:

# ll /var/stm/logs/os

Look at the timestamps for each of the log.xx.raw files. If there are many with dates close to each other then the system has been logging lots of diagnostic messages that could be reviewed. They are reviewed using the logtool utility from within the CSTM tool.

-Bob

[Gary_O]

you are correct. The only way I know how many times it has rebooted is by looking in /var/adm/syslog/mail.log

there is nothing of interest in the OLDsyslogs.log except for maybe this:
OLDsyslog.log8:Apr 17 16:22:08 acp sfd[1800]: recovering from previous daemon crash


how do you invoke the log utility to look at the logs in /var/stm/logs/os?

I do have some:

-rw-r--r-- 1 root root 18424 Apr 28 12:55 ccbootlog
-rw-r--r-- 1 root root 69196 Apr 28 12:48 memlog
-rw-r--r-- 1 root root 161816 Apr 25 16:48 log56.raw.cur
-rw-r--r-- 1 root sys 124040 Apr 25 15:48 ccerrlog
-rw-r--r-- 1 root root 21016 Apr 25 15:48 ccbootlog.426
-rw-r--r-- 1 root root 8424 Apr 25 13:57 ccbootlog.425
-rw-r--r-- 1 root root 11112 Apr 25 06:08 ccbootlog.424
-rw-r--r-- 1 root root 16864 Apr 24 07:51 ccbootlog.423
-rw-r--r-- 1 root root 7528 Apr 24 03:59 ccbootlog.422
-rw-r--r-- 1 root root 7504 Apr 24 03:00 ccbootlog.421
-rw-r--r-- 1 root root 18928 Apr 24 02:18 ccbootlog.420
-rw-r--r-- 1 root root 11424 Apr 24 01:59 ccbootlog.419
-rw-r--r-- 1 root root 7416 Apr 24 01:33 ccbootlog.418
-rw-r--r-- 1 root root 7440 Apr 24 01:11 ccbootlog.417

[Robert_Jewell]

> how do you invoke the log utility to look at the logs in /var/stm/logs/os?

Start CSTM (#cstm) and then use 'ru' to list and select the logtool utility. You will see a description of how to view a log on the screen. I beleive the commands are along the lines of:
sr (select raw file)
fr (format raw file)
fl (display formatted log)

The only log here that is worth reviewing is log56.raw.cur. Check it out and see if you see anything that may help you.

-Bob