rx2600 boot failure

[Rob1n]

I'm having problems with one of our rx2600 servers. After a recent power-down it will no longer boot the OS.

I've run all the diagnostics tests, which show no issues, and have updated the firmware to the latest version. I'm also unable to run the EFI shell - it just enumerates the disks & partitions and then hangs.

I'm suspecting a hardware failure, but don't have enough familiarity with these systems to be sure. Any suggestions?

[Torsten.]

If you updated the firmware, you can reach EFI, right?

Disconnect all external cables and run "reconnect -r" and "map -r" from EFI.

Does this work?

[Rob1n]

I can't get to the EFI shell, no - I updated the firmware via the management card (which seems to be working fine) and can boot from CD without any issues, but both the EFI shell and OS boot hang.

[Torsten. · Accepted solution]

Disconnect external cables and try to get into EFI.
If this doesn't work, pull out the internal disks. If still doesn't work, remove PCI cards.

[Rob1n]

Ah, many thanks! Pulling both the disks allows me to get into the EFI shell anyway. I can then hotplug them and run "reconnect -r" and "map -r" and all the partitions show up okay, but trying to boot with them in still results in a hang.

So that looks like partition/filesystem corruption. Is there anything I can do from the EFI shell to check for either of these? Or do you think there's something else wrong?

[Torsten.]

If only a partition is bad this should not prevent to reach EFI, but a bad/failed disk could.

Insert one disk at a time and try to run the "reconnect -r" "map -r" again.

Does it hang with one of the disks?

If not, post the result of "map -r".

[Torsten.]

BTW, if you updated the firmware from MP, you probably don't have the latest.

MP:CM> sysrev

MP FW : E.03.30
BMC FW : 01.53
EFI FW : 01.22
System FW : 02.31

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=447335&swItem=ux-46489-1&prodNameId=88839&swEnvOID=54&swLang=13&taskId=135&mode=4&idx=1

[Rob1n]

Yep, only fails with one of the disks. Gets as far as:

fs0 : Acpi(HWP0002,0)/Pci(2|0)/Ata(Primary,Master)/CDROM(Entry0)
fs1 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig6EF545E8-FFC5-48EF-AE28-495F465D47AB)
blk0 : Acpi(HWP0002,0)/Pci(2|0)/Ata(Primary,Master)
blk1 : Acpi(HWP0002,0)/Pci(2|0)/Ata(Primary,Master)/CDROM(Entry0)
blk2 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)
blk3 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig6EF545E8-FFC5-48EF-AE28-495F465D47AB)
blk4 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part2,SigF8AF6D94-EB25-4DA4-BCFE-FCF757150830)

So the disk's kaput?

And, yes - the firmware is up to date. I had to do the first couple via the MP (it was on 02.10) and then the final update was from CD.

[Torsten.]

Well, this disk has a valid file system:

fs1 : Acpi(HWP0002,100)/Pci(1|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig6EF545E8-FFC5-48EF-AE28-495F465D47AB)


Try

Shell> fs1:

fs1:\> cd efi

fs1:\EFI> cd hpux

fs1:\EFI\HPUX> hpux

Does it boot?

[Rob1n]

I managed to track down the problem. I removed the problem drive, installed it into another system and ran some checks (SMART tests, full disk copy, fsck, etc). Between that and the fstab data on the working drive I figured out the problem was that the initial partition on the disk was the wrong type. It was marked as FAT32, but a dosfsck reported corrupt FAT tables and the fstab entry showed it should be mounted as swap, so it clearly wasn't actually FAT32. I changed the partition type and ran mkswap on it and it's all working now and boots without issues.

Must say that I'm not impressed that the EFI system tries to read (all?) FAT32 partitions and hangs if it fails - I'd expect it to just produce a warning/error and move on.

Anyway, many thanks for helping me track this one down.

[Torsten.]

Do you have hp-ux on that disk?

Not sure if I understand what you did.

This is a part of the output of the "devices" command in EFI:

Shell> devices
...
A0 B X X 1 2 5 LSILOGICLogical Volume 3000
A1 D - - 1 2 - FAT File System [FAT32] 100 MB
A2 D - - 1 2 - FAT File System [FAT32] 400 MB
...

This is a logical drive (hardware RAID), the first partition is FAT32 (EFI partition), the second (not showed) is for hp-ux, the third is FAT32 (HP Service partition) - this is the default disk layout.

[Rob1n]

No, the system's running Redhat Enterprise 3.

The first (boot) disk was working fine - it has a FAT32 (EFI) partition and a number of linux partitions. The second disk was showing as having a FAT32 partition and a linux partition (though the OS settings indicated it should have been a linux swap partition, not FAT32).

The EFI Shell (and boot loader) were trying to read from the FAT32 partition on the second disk (not sure why) and hanging because it wasn't actually formatted as FAT32 (i.e. the partition type for it was incorrect).

Correcting the partition type stopped the system from trying to read from the partition and allowed it to boot normally.

[MALIK ADEEL IMTIAZ]

Dear Torsten,
I am facing the same problem i have hp server rx2600 and it got corrupted.
first off all it has 146G and 320 10K
HDD and when i reboot it with bootable 64 bit Itanium cd. it shows me no CDROM options from where i select CDROM to make fresh installation it hangs.

Kindly help me in this regard

Thanks
Adeel Imtiaz

[Torsten.]

Please open your own thread and attach the output of

Shell> reconnect -r
Shell> map -r
Shell> devices

[Rob1n]

All's still working okay so I'm closing this thread. Looks like the final solution is to check that all partitions marked as FAT32 actually are, and that there's no filesystem corruption.