L2000 error during boot

[Aviad Twina]

Gurus,
My L2000 (HPUX 11.00 ) behaves strange lately.
During the last week, the machine halted itself without reason.

I've looked at the machine GSP (I used the SL command ), and I found the following error:
LERT LEVEL: 6 = Boot possible, pending failure - action required

SOURCE: 8 = I/O
SOURCE DETAIL: 9 = serial SOURCE ID: 0
PROBLEM DETAIL: 3 = functional failure

Is it hardware related problem or what?
Where can I found an official HP documentation about the GSP errors?

Thanks,
Aviad

[Massimo Bianchi]

Hi,
usually you can find more info looking in /var/tombstones and /var/adm/crash/crash.*, if after the halt there was a reboot or if you did a TOC.

in the /var/adm/syslog/OLDsyslog.log, is there any usefull information ?

HTH,
Massimo

[Lior Mishkovsky]

Hi,

Take a look at this one : (It might lead to other stuff...)

http://docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/A5191-96018/A5191-96018_top.html&con=/hpux/onlinedocs/A5191-96018/00/00/21-con.html&toc=/hpux/onlinedocs/A5191-96018/00/00/21-toc.html&searchterms=GSP|messages&queryid=20030328-221741

Lior./

[Enrico P.]

Hi,
if now the server is up and running check also the files:

/etc/shutdownlog

and

/var/adm/syslog/OLDsyslog.log

for further information.

Enrico

[Bruno Vidal]

Hi,
A good thing to know, is to take a look at the console log. With a GSP, if you are not working on the console, you can look at the CL command, it will give you the last message on the console. So you will be normally able to see 2 or 3 boot, and in your case, you should be able to see if you had a panic (data page fault/HPMC). In standart case, you should have a crash under /var/adm/crash, if it is an hardware problem you should have also a file named ts99 under /var/tombstone (but take care of the timestamp in this file, it should be around your reboot). But in both case, you should contact HP for further analyse (but if software crash, i suggest last patch bundle).

Cheers.

[Aviad Twina]

Thank you for all your answers.
There is on file under /var named as tombstone...
I didn't see nothing special under /var/adm/syslog/OLDsyslog.log.
/var/adm/crash shows me 9 core files.
I looked at the latest one( used strings comamnd ), but i w]couldn't find anything bombastic...

However I found the following error under /etc/shutdownlog " Reboot after panic: callout table overflow".

Thanks,
Aviad

[Paula J Frazer-Campbell]

Hi
Can you post ts99 (tombstone file)

Paula

[Massimo Bianchi]

hi,
if you error is "callout table overflow" you may consider a kernel recompilation, chaning hte parameter ncallout, but i didn't think that this can cause a panic.

Can you also post a swlist -l product, to see your path bundle level ?

Maybe you are missing a critical patch.

HTH,
Massimo

[Jean-Louis Phelix]

Hi,

You could try 2 things :

- install latest SCSI patch (PHKL_29049) to prevent panics from being triggerred by underlying SCSI hardware errors
- increase nproc and ncallout
- check your hardware (have a look at ts99)

Regards.

[Helen French]

I would say this error is because of a mismatch between GSP revision and your server's PDC version. What revision of GSP are you using? What version of PDC do you have?

If you have Rev B GSP card, you need to have a PDC version 41.38 or higher installed. Check the ITRC patch page for updated versions of PDC and install it on your system. That will solve the problem.

[Steven E. Protter]

The symptoms you report are likely associated with a crash of some kind.

cd /var/adm/crash

If there are subdirectories, there are crashes.

Use the attached instructions to analyze the crash.

You may not have crash dumps configured.

There is an intuitive configration file called /etc/rc.config.d/savecrash

Makes sure the
SAVECRASH variable looks like this
SAVECRASH=1

Regards and Good Luck,

SEP

[twang]

I have same experience, I have done the following:
- check /var/tombstones after reboot
- check /var/adm/crash/
- check /var/adm/syslog/syslog.log
- check /var/adm/syslog/OLDsyslog.log
Finally, I called hp support hotline, and it was resolved by replacing one of harddisks of vg00.

[Aviad Twina]

Hi again,
March Quality pack is already installed on my system.
I'll install the PHKL_29049 SCSI patch, and I hope it might solve the problem.
In addition I removed the comment from the SAVECRASH varibale.
Thanks,
Aviad

[Elena Leontieva]

This is regarding kernel parameter:
Default value of the kernel parameter (ncallout) is too small. DocId: KBRC00007212 Updated: 6/14/01 2:55:00 AM

PROBLEM
Problem Text :

The default value of kernel tunable parameter "ncallout" is
too small and may cause system panic ("callout table overflow")
if many multi-threaded applications are running on HP-UX 11.00.

It is defined with NPROC in
/usr/conf/master.d/core-hpux file as,

ncallout NCALLOUT (16+NPROC)RESOLUTION
The workaround of this phenomenon is to modify the
/usr/conf/master.d/core-hpux file by hand as follows
and recreate new kernel.

from
ncallout NCALLOUT (16+NPROC)
to
ncallout NCALLOUT (16+NKTHREAD)
(or larger).


This has been fixed on HP-UX 11i.

Also, you probably have another issue (Document id: pamLC200111121):
A7 - The chassis code below appears because the Rev B
GSP Card (A6696B) is not fully supported on the
revisions of PDC (L2000 (PDC 40.48) and L3000
(PDC 40.43)) installed on the system. You must
have PDC 41.36 for L3000 and PDC 41.38 for
L1000/L2000.

-------------------------------------------------------------------
***** EARLY BOOT VFP : SYSTEM ALERT *****
SYSTEM NAME: octopus
DATE: 08/30/2001 TIME: 10:44:07
ALERT LEVEL: 6 = Boot possible, pending failure - action required

REASON FOR ALERT
SOURCE: 8 = I/O
SOURCE DETAIL: 9 = serial SOURCE ID: 0
PROBLEM DETAIL: 3 = functional failure

LEDs: RUN ATTENTION FAULT REMOTE POWER
FLASH OFF OFF ON ON
LED State: Running non-OS code. (i.e. Boot or Diagnostics)

0x00000063890014C0 00000000 00000000- type 0 = Data Field Unused
0x58000863890014C0 00006507 1E0A2C07- type 11 = Timestamp
08/30/2001 10:44:07
A/a: ack read of this entry - Q/q: quit Virtual Front Panel Display
Anything else redisplay the log entry
->Choice:

[Aviad Twina]

Elena ,
How can I determine the GSP rev ?
Thanks,
Aviad

[Helen French]

For the last question:

To find GSP revision, go to your system console or LAN console. Enter "Ctrl+B", hit enter twice to get the GSP> prompt. There enter command "HE" to get your GSP revision.

[Elena Leontieva]

Aviad,

To get the GSP version:
at the console prompt :
CTRL-B
return
return
GSP> he -> see the version
GSP:HELP> q
CSP> co
enter

[Aviad Twina]

Thanks.
So far, after i've changed the kernel parameter it seems to be working fine.

Aviad