topic Re: Alert 13 in Operating System - HP-UX

Alert 13

Ben Wood_2 — Tue, 19 Nov 2002 20:05:22 GMT

My system crashed and the errorlog showed an Alert 13:
"System hang detected via timer popping". I am new to the HP-UX environment and am looking for guidance on how to troubleshoot this. Does this type of error warrant making a service call to HP?

Thanks, Ben

Re: Alert 13

Ken Hubnik_2 — Tue, 19 Nov 2002 20:19:33 GMT

One of my servers paniced last night and I received the same alert. I just got off the phone with HP support and basically that alert indicates your system paniced due to a hang condition and it is rebooting. They suggested to me that it was not hardware related and either the O/S or third party software created a hang condition causing the reboot. I am looking into my application logs for errors around the reboot time.

Re: Alert 13

Jeff Schussele — Tue, 19 Nov 2002 20:46:38 GMT

Hi Ben,

Depends - this can be caused by a CPU going bad & not releasing a resource that another CPU is waiting on.
OR it could be caused by bad code.

You need to look at several things:
1) The /etc/shutdownlog to see what actually initiated the shutdown
2) The GSP error log for any events that hint at HW failure.
3) The crash dump to see exactly what caused the "hang". You may need HP's expertise to decipher the dump.
4) The tombstone created by this event - it may spotlight a bad CPU.

Either way I'd log a call to the Responce Ctr & let them decide whether to open a HW or SW ticket. But you definitely to investigate root cause & monitor this system closely.

Rgds,
Jeff

Re: Alert 13

Ken Hubnik_2 — Tue, 19 Nov 2002 20:49:01 GMT

Pertaining to Jeffs response. I logged a call with HP support and we went through all 4 of those steps and everything was clean (meaning no errors). So if you check these and they are clean then could be the software cause the panic.

Re: Alert 13

Jeff Schussele — Tue, 19 Nov 2002 21:00:13 GMT

Again that depends.
First question should be - What's changed?
Any new code installed - be it OS or application?
If not & this code has been running w/o incident for some time, then I'd still lean towards a HW scenario.
Does the SW log anything? Have you checked those logs?

Ken - did HP have you ftp the entire dump to them - tombstone, shutdownlog & swlist -l patch output?
If not I'd insist on it. A quick look at the dump & tombstone can easily miss the problem.

Rgds,
Jeff

Re: Alert 13

Ken Hubnik_2 — Tue, 19 Nov 2002 21:04:29 GMT

I sent them everything but the swlist. FYI check this link out that has just been posted this afternoon on the server board.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xe05d0fe6d0f7d61190050090279cd0f9,00.html

Re: Alert 13

Jeff Schussele — Tue, 19 Nov 2002 21:30:14 GMT

Yea - I saw that.

I had an N reboot like that several weeks ago, that looked all the world to be a SW hang. But we did spot "trace" residue in the GSP of a HW problem - but VERY unspecific msgs in all. The tombstone was blank - literally all zeros. But the dump held the clue that CPU 1 had hung up causing CPU 5 - that was waiting on a resource that CPU1 was NEVER gonna release - to pop the timer. But it took HP a day to find it in the dump.

Rgds,
Jeff

Re: Alert 13

Ken Hubnik_2 — Tue, 19 Nov 2002 21:51:07 GMT

Jeff what did HP recommend as a fix. Did they take any action?

Re: Alert 13

Jeff Schussele — Tue, 19 Nov 2002 22:40:55 GMT

Yes - CPU 1 was replaced.
No trouble since.

Rgds,
Jeff

Re: Alert 13

Ben Wood_2 — Thu, 21 Nov 2002 13:23:41 GMT

I checked the suggested logs and /var/tombstones. There was not any relevant info in either. I talked to HP and they are going to bring out a CE. Thanks for pointing me in the right direction.

--Ben