HPE 9000 and HPE e3000 Servers
cancel
Showing results for 
Search instead for 
Did you mean: 

RP5470 Hung?

 

RP5470 Hung?

I have a new RP5470 that for the past two mornings appears to be hung. I can not log into the console via hp terminal or lan console. I can log into gsp and check chassis log all is ok and all indicators on system are ok. Cannot telnet to system either, ping does work. With not being able to log into system not sure what else to look at before simply reset the system via the gsp. Any ideas?
15 REPLIES 15
DGH_2
Valued Contributor

Re: RP5470 Hung?

hi
i had the same problem, but i had error in GSP like processor time out....
if you have like these messages in GSP, can you please post it?

Regards
G. Vrijhoeven
Honored Contributor

Re: RP5470 Hung?

Hi,

Can you do an ioscan -fn and give the output?

I like to see if all the hardware is claimed.


Gideon
Bernhard Mueller
Honored Contributor

Re: RP5470 Hung?

Did you involve HP already?

Does ping show any drops or longer response times?

You say it is a new server, so it should have
PHKL_28695 - 11.11 Cumulative VM, Psets, Preemption, PRM, MRG
PHKL_28410 - 11.11 vm preemption point, pdc, vhand performance
PHKL_25212 - 11.11 vm preemption point, mlock/async_io
PHKL_28529 - 11.11 VxFS mmap(2) performance improvement; vhand

or superseding patches installed. If not install them and open a call with HP.

Regards,
Bernhard

Re: RP5470 Hung?

Sorry should have given more info....it is running HPUX 11.0 (application required) and has the September patches. Cannot issue any commands as I can not get into the system via either a hardwired console or lan console. Nor can I telnet of ftp the system. I can ping the system with no packets lost an ping times less than 1ms.
G. Vrijhoeven
Honored Contributor

Re: RP5470 Hung?

Hi James,

I had the same kind of problems with a rp5470 and this patch ( PHNE_27788 )solved it.
Take a look at this:

http://www5.itrc.hp.com/service/patch/patchDetail.do?BC=patch.breadcrumb.main|patch.breadcrumb.search|&patchid=PHNE_27788&context=hpux:800:11:00

Bernard is right you should open a call.

Gideon
Bryan D. Quinn
Respected Contributor

Re: RP5470 Hung?

Hello James,

I am not sure this is exactly the same problem that you are having, but let me throw this at you. It might be something that you can investigate.

I had a similar problem with an N-class box. The only thing I could do was GSP, I placed a call with HP. I was told to power cycle the box and allow it to boot fully. When it came up to log on and remove the following file:

/etc/ioctl.syscon

Then reboot the system.

It might be something you can investigate. For some reason though, I am thinking this had something to do with not being able to get a command line in single-user mode. I don't have that in my notes, but for some reason that comes to mind.

Anyways, I hope this helps!

-Bryan

Re: RP5470 Hung?

thanks all for inputs....reset and reboot the system now getting:



Leaving Console Mode - you may lose write access.
When Console Mode returns, type ^Ecf to get console write access.

GSP Host Name: l3000gsp
GSP>


GSP Host Name: l3000gsp
GSP>


GSP Host Name: l3000gsp
GSP>


GSP Host Name: l3000gsp
GSP> rs


RS

Execution of this command irrecoverably halts all system processing and
I/O activity and restarts the computer system.

Type Y to confirm your intention to restart the system: (Y/[N]) y
y
-> SPU hardware was successfully issued a reset.


GSP Host Name: l3000gsp
GSP>
********** VIRTUAL FRONT PANEL **********
System Boot detected
*****************************************
LEDs: RUN ATTENTION FAULT REMOTE POWER
FLASH OFF OFF ON ON
LED State: Running non-OS code. (i.e. Boot or Diagnostics)

platform config 626F
o
*****************************************
Invalid choice. Q/q: quit EARLY BOOT VFP
*****************************************
processor slave rendezvous 1C17
processor test 1142
processor test 1100

*****************************************
Invalid choice. Q/q: quit EARLY BOOT VFP
*****************************************
processor test 1100
processor test 1100
processor test 1100
processor test 1100
processor test 1100
processor test 1100
processor test 1100
processor test 1100
PDH config 322F
PDH test 3149
PDH test 3160
platform test 616A
processor test 1146
processor INIT 1701
processor INIT 1701
processor test 1110
processor test 1110
processor test 1110
processor test 1110
processor INIT 1701
processor INIT 1701
processor test 1110
processor test 1110
processor slave rendezvous 1C17
processor test 1142
processor slave rendezvous 1C10
processor test 1142
PDH test 316E
PDH test 316E
PDH test 316E
PDH test 316E
PDH test 316E
PDH test 316E
PDH test 316E
memory config 7210
memory INIT 7702
memory INIT 771D
memory config 7230
memory config 7231
memory config 7231
memory test 7150
memory config 7213
memory config 7213
memory config 7214
memory config 7214
memory config 7213
memory config 7214
memory config 7213
memory config 7214
memory config 7213
memory config 7214
memory config 7214
memory config 7213
memory config 7214
memory config 7214
memory config 7213
memory config 7214
memory config 7213
memory config 7214
memory config 7213
memory config 7214
memory config 7214
memory config 7213
memory config 7215
memory config 7216
memory config 72A0
memory test 71A1
memory test 71A2
memory test 71A4
memory test 71A5
memory test 71A6
memory test 71A3
memory test 71A4
memory test 71A4
memory test 71A5
memory test 71A5
memory test 71A6
memory test 71A6
memory config 7210
I/O INIT 8701
I/O test 8118
I/O test 8118
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
I/O INIT 8701
memory config 7240
memory INIT 7702
memory config 72A0
memory test 71A1
memory config 72A1
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory INIT 77A9
memory config 7240
I/O config 8204
I/O config 8204
I/O config 8204
I/O config 8204
I/O config 8204
I/O config 8204
I/O config 8204
I/O test 8119
I/O test 8119
I/O test 8119
I/O test 8119
I/O test 8119
I/O test 8119
I/O test 8119
I/O test 8119
I/O config 8204
processor test 1120
processor test 1120
processor INIT 1701
processor test 1120
processor test 1120
processor test 1120
processor INIT 1701
processor test 1120
processor slave rendezvous 1C40
processor test 1142
platform test 612A
I/O config 8238

*****************************************

************ EARLY BOOT VFP *************
End of early boot detected
*****************************************

Firmware Version 42.19

Duplex Console IO Dependent Code (IODC) revision 1

------------------------------------------------------------------------------
(c) Copyright 1995-2000, Hewlett-Packard Company, All rights reserved
------------------------------------------------------------------------------

Processor Speed State CoProcessor State Cache Size
Number State Inst Data
--------- -------- --------------------- ----------------- ------------
0 750 MHz Active Functional 750 KB 1.5 MB
2 750 MHz Idle Functional 750 KB 1.5 MB

Central Bus Speed (in MHz) : 133
Available Memory : 6291456 KB
Good Memory Required : 17444 KB

Primary boot path: 0/0/1/1.2
Alternate boot path: 0/0/2/0.2
Console path: 0/0/4/1.0
Keyboard path: 0/0/4/0.0

WARNING: The non-destructive test bit was set, so memory was not tested
destructively. Information only, no action required.


Processor is booting from first available device.

To discontinue, press any key within 10 seconds.

10 seconds expired.
Proceeding...

Trying Primary Boot Path
------------------------
Booting...
Boot IO Dependent Code (IODC) revision 1


HARD Booted.

ISL Revision A.00.38 OCT 26, 1994

ISL booting hpux

Boot
: disk(0/0/1/1.2.0.0.0.0.0;0)/stand/vmunix
7950336 + 900512 + 4793064 start 0x23fce8


alloc_pdc_pages: Relocating PDC from 0xffff800000 to 0x7fa03000.
gate64: sysvec_vaddr = 0xc0002000 for 1 pages
NOTICE: autofs_link(): File system was registered at index 3.
NOTICE: nfs3_link(): File system was registered at index 5.
td: claimed Tachyon XL2 Fibre Channel Mass Storage card at 0/3/0/0
td: claimed Tachyon XL2 Fibre Channel Mass Storage card at 0/9/0/0

System Console is on the Built-In Serial Interface

************* SYSTEM ALERT **************
SYSTEM NAME: l3000gsp
DATE: 11/19/2003 TIME: 13:34:19
ALERT LEVEL: 12 = Software failure

REASON FOR ALERT
SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

LEDs: RUN ATTENTION FAULT REMOTE POWER
FLASH OFF OFF ON ON
LED State: Running non-OS code. (i.e. Boot or Diagnostics)

0xF8E000C01100B800 00000000 0000B800 - type 31 = legacy PA HEX chassis-code
0x58E008C01100B800 0000670A 130D2213 - type 11 = Timestamp 11/19/2003 13:34:19
A: ack read of this entry - X: Disable all future alert messages
Anything else skip redisplay the log entry
->Choice:Timeout!
*****************************************

************* SYSTEM ALERT **************
SYSTEM NAME: l3000gsp
DATE: 11/19/2003 TIME: 13:34:19
ALERT LEVEL: 12 = Software failure

REASON FOR ALERT
SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

LEDs: RUN ATTENTION FAULT REMOTE POWER
FLASH FLASH OFF ON ON
LED State: Running non-OS code. Non-critical error detected.
Check Chassis and Console Logs for error messages.

0xA0E000C01100B000 00000000 000005E9 - type 20 = major change in system state
0x58E008C01100B000 0000670A 130D2213 - type 11 = Timestamp 11/19/2003 13:34:19
A: ack read of this entry - X: Disable all future alert messages
Anything else skip redisplay the log entry
->Choice:

I will continue to troubleshoot for a bit as this is not yet a production server. Software Failure hmm .... bad boot disk?
G. Vrijhoeven
Honored Contributor

Re: RP5470 Hung?

Hi,

Looks like you hava a Bad CPU.

REASON FOR ALERT
SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

Call HP....



Gideon
Bryan D. Quinn
Respected Contributor

Re: RP5470 Hung?

Hello James,

Looking at your errors, I would call HP and open a ticket. These are the hex codes in your GSP message. Give them these hex codes from your GSP message and tell them what is happening. They should be able to decode these and let you know what is going on.

0xF8E000C01100B800

0x58E008C01100B800

Hope this helps!
-Bryan

Re: RP5470 Hung?

just an update to those who may be following this thread. I am using this as a learning experience and will call HP if needed. So far I have now been able to reset the system and interupt the autoboot and do a search for potential boot drives and the 18gb drive at 0/0/1/1.0 does not show up this drive is one of 2 drives in vg00 and i a, able to boot from the install cd with no errors.

ran ioscan and neither drive on the 0/0/1/1 bus showed up. Reseated both drives and now both show in ioscan.

Rebooted system and did a search for boot devices an the drive at 0/0/1/1.0 now shows up as a potential boot device.

booted system from promary boot device and system booted without errors.

OLDsyslog.log had following errors:



Nov 18 11:44:22 larry EMS [2182]: ------ EMS Event Notification ------ Value:
"MAJORWARNING (3)" for Resource: "/storage/events/disks/default/0_0_1_1.0.0"
(Threshold: >= " 3") Execute the following command to obtain event details:
/opt/resmon/bin/resdata -R 142999554 -r /storage/events/disks/default/0_0_1_1
.0.0 -n 142999553 -a
Nov 18 11:44:23 larry EMS [2182]: ------ EMS Event Notification ------ Value:
"MAJORWARNING (3)" for Resource: "/storage/events/disks/default/0_0_1_1.2.0"
(Threshold: >= " 3") Execute the following command to obtain event details:
/opt/resmon/bin/resdata -R 142999557 -r /storage/events/disks/default/0_0_1_1
.2.0 -n 142999554 -a
Nov 18 15:45:53 larry : su : + 1 root-oracle
Nov 18 15:41:54 larry vmunix: SCSI: Target detected parity error -- lbolt: 14877
91, dev: 1f010000
Nov 18 18:32:50 larry vmunix: SCSI: Unexpected Disconnect -- lbolt: 2513351, dev
: 1f010000, io_id: 1015400
Nov 18 18:33:17 larry vmunix: SCSI: Resetting SCSI -- lbolt: 2515609, bus: 1
Nov 18 18:33:17 larry vmunix: SCSI: Reset detected -- lbolt: 2515609, bus: 1
Nov 18 18:33:17 larry vmunix: LVM: Recovered Path (device 0x1f010000) to PV 1 in
VG 0.
So i would have to conclude that the disk at 0/0/1/1.0 is bad.

Any thing else to add or differing opinions?

Thanks so much for the help and i must say using these forums for is such a great help!
Bryan D. Quinn
Respected Contributor

Re: RP5470 Hung?

Hello James,

Looks like you nailed it. I would definately still consider having HP swap out that drive.

-Bryan
Bernhard Mueller
Honored Contributor

Re: RP5470 Hung?

James,

good you got it up again, however, you got EMS warnings at about the same time from *two* disks.

Unlikley two drives fail at the same moment, maybe both were not seated properly (did you by chance experience any earthquakes lately ;-)

Still to be on the safe side regarding the disk drives, do the following.

run diskinfo /dev/rdsk/cxtYd0
it should return immediately, longer pausing indicates a problem even if ioscan shows CLAIMED.

even if diskinfo is ok, run a full
dd if=/dev/rdsk/cXtYd0 of=/dev/null
this should take a while and exit saying
# records in
# records out

If you get an I/O error from dd replace the drive.

Regards,
Bernhard
Bernhard Mueller
Honored Contributor

Re: RP5470 Hung?

oops

to speed it up a bit use
dd if=/dev/rdsk/cXtYd0 of=/dev/null bs=1024k

Though as mentioned, I would assume the drives to be good and rather suspect a backplane problem.

Regards,
Bernhard

Re: RP5470 Hung?

Thanks Bernhard, had another drive and put it same position so that it would be at 0/0/1/1.0 and issued a seach for boot devices and new drive did not show up as a potential boot device. So I like believe it is not a disk drive. Calling HP! Thanks!
Michael Tully
Honored Contributor

Re: RP5470 Hung?

While your on the job, this would be a very oppotune time to update your GSP and CPU firmware. Having the latest patch bundle loaded should have been a pre-requisite to doing these as well.
Anyone for a Mutiny ?