System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

n4000 system crash - alert level:12 = software failure

 
SOLVED
Go to solution
KiddA
Advisor

n4000 system crash - alert level:12 = software failure

Hi people, server is not coming up after applying latest qpk.

We already checked gsp, interacted with isl, ma, tried to boot from pri and alt boot path, to see what might be the reason for this error but server still crash.

I would like to know if there's a way to save a crash dump at this point or how to troubleshoot this alert and have server up.

Screenshot attached.

Thanks all for time&help.
7 REPLIES
Michael Steele_2
Honored Contributor
Solution

Re: n4000 system crash - alert level:12 = software failure

Hi

Boot off of the vmunic.prev kernel, and roll back the QPK with swremove. Do you need the steps? What do you do after you interrupt the boot up at the 10 second prompt?
Support Fatherhood - Stop Family Law
lbseraph
Advisor

Re: n4000 system crash - alert level:12 = software failure

If you still can't bootup, the crash dump file won't save to OS(/var/adm/crash). I am no sure if have other method to save crash dump files before OS up.

Could you bootup to single mode? Have better to remove the QPK firstly, if fail, may need to restore OS from your backup tape.
KiddA
Advisor

Re: n4000 system crash - alert level:12 = software failure

Hi Michael ... if you could provide the steps, it would be more than appreciated.

lbseraph, we tried to boot into single user mode, but server crash before getting prompt.

Thanks again guys.
Michael Steele_2
Honored Contributor

Re: n4000 system crash - alert level:12 = software failure

Hi

1) reboot
You will see a prompt that says "10 seconds to interrupt boot up - press any key "
2) Press space bar and interrupt
You will see another prompt that says "Do you want to interact with IPL?"
3) yes
4) ISL> hpux /stand/vmunix.prev

Note: Now, you said your box crashed before seeing 2), but you also said "...We already checked gsp, interacted with isl...." - If you can't get to the ISL then you're made.

However, crashing before seeing 2) indicates a HW problem, and this is not consistent with ALERT LEVEL 12 - Software. ALERT LEVEL 12 - Software is an O/S alarm. Not a HW alarm.

Write back with your results.
Support Fatherhood - Stop Family Law
Michael Steele_2
Honored Contributor

Re: n4000 system crash - alert level:12 = software failure

Ooops - Type -

"...If you CAN get to the ISL..."

:-)
Support Fatherhood - Stop Family Law
KiddA
Advisor

Re: n4000 system crash - alert level:12 = software failure

Hi Michael, as you mentioned, and since we weren't able to get prompt we started troubleshooting hardware.

First thing we did, we unplugged fibers that go to SAN, rebooted server and it came up normally.

I'll assign points and leave post open to see if this behavior is familiar to someone else.

Thanks Michael for your help.
Michael Steele_2
Honored Contributor

Re: n4000 system crash - alert level:12 = software failure

Hi

Bad HBA card or, no HBA driver. The HBA cards fit into PCI I/O slots. The fibre attach to them on the server side.

Now that you are up, check these logs.

/etc/shutdownlog
/etc/opt/resmong/log/*
/var/adm/syslog/OLDsyslog.log
GSP Event Logs
/var/adm/crash/ - Find the dump from the first day

Get the HBA product numbers and verify the drivers installed and newest offered.
Support Fatherhood - Stop Family Law