L2000

[John Gunnis]

I am an HP Reseller and one of our customers
has an L2000. It would not boot up and the GSP
log would just say that a disk caused the fatal
error. Upon trying to reboot, the system would
stop booting at different spots. I then booted to single user mode. I checked a few things out then did an init 2. The system
fsck'ed several partitions then all of a sudden
we got the message that dev_t=0x1F012000 has
power failed. I was able to get back to single
user mode and did a ls on /dev/dsk. c1t2d0 has
brw-r----- 1 bin sys 31 0x012000 Feb 1 2000 c1t2do. After finally booting up all the way after several attempts, doing a vgdisplay -v on vg00 shows this device as /dev/dsk/c1t2d0.
Hence, I believe the root drive is failing.
My problem is, what are the best way or ways to
replace the root drive?
At present, we just completed a full backup of vg00 using fbackup.
The drive is not mirrored and vg00 is only on one disk drive. We do have another drive in the system that could become empty. Would doing a dd help? If so, do I need to boot from
a recover cd? The drive is a 9.1 GB how long
would this copy take?
Another thought I had was to create an Ignite-UX image to tape, replace the drive and boot to the tape and restore vg00.

Thanks in advance,

John Gunnis

[Iain Ashley · Accepted solution]

Hi John,
If you can get through an ignite session, this would be a good thing to do. Make sure you use the -A option though to get everything. If you can mirror the disk (you will need a mirrordiskUX license installed), that would be an excellent option also and the quickest. You need to be careful that you follow the correct procedure to make it bootable. I will outline the minimum to get it bootable below:

pvcreate -B -f /dev/rdsk/
mkboot /dev/dsk/
mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/
(repeat for current disk)
vgextend vg00 /dev/dsk/

The lvols must be mirrored in the correct order, it /stand is not in the right place, the system will not boot.

lvextend -m 1 /dev/vg00/lvol1
lvextend -m 1 /dev/vg00/lvol2
lvextend -m 1 /dev/vg00/lvol3
.
etc.

lvlnboot -Rv

The disk should then be mirrored, make a note of it's hardware path, then reboot to test it.

Regards
Iain Ashley

[Stefan Stechemesser]

Hi John,

you could both try dd or make_tape_recovery (Ignite), but if the root disk is really bad, then this may fail. Another question is: Is the root disk bad or do you have an internal SCSI bus problem (bad cable/controller etc.).
If you have a spare disk, I would recomend to replace the bad disk and install a minimal HPUX system. If this works, there is a good chance that the disk was the error cause. You can then replay the full backup with frecover.

Best regards

Stefan

[John Gunnis]

Just a quick update of what is happening.

We are experiencing another disk failure
on a different controller and volume group.
I am currently running a make_tape_recovery
-x inc_entire=vg00 -I -v -a /dev/rmt/1mn

The other disk powerfailed. After a few shutdowns the drive is now active. After the
tape recovery is made, I plan to run some diagnostics. My client is also in the process of moving the data to a windows based
system.

Thanks for the help so far.

[Ted Buis]

Are these internal disks? Do you have good clean power to the L2000? Does the L2000 have the redundant power supply (3rd), in case you are losing the power supply?

[Shaikh Imran]

Hi,
There may be many reasons of the disk failing so frequently.You have to check some
1) Please check the Power coming to the server == Are you receving over voltage from what is required ?
2) Please check the earthing whether you are getting voltage on earthing ?
4) Also have a look to the above post of TED
regarding the power supplies
3)Are you using very Old Disks in L class or they are somewhat new ?
Please do a proper ignite backup without wasting any time and i will advise you to take 2 copies if the situation allows.


Regards,

[John Gunnis]

Well,

I was able to get an Ignite backup, I replaced the root drive and installed the
OS (HP-UX 11.0) by the tape backup. Now,
I have problems with my other volume groups.

Some background. The L2000 has 4 Internal drives. These are set up in 4 volume groups
VG00,VG01,VG02,VG03. All these volume groups
appeared with no errors. My client also has
8 disk drives with 2 volume groups. VG04 has
5 disks and VG05 has 3 disks.

Also, they
have 8 more disks with 2 controllers, so that
there are 2 paths to the same disks. Sam thinks there are 16 drives but there are only 8. This is VG06.

The message I am getting is that the drives
do not match the volumes.

When I did a vgscan -p I seemed as if the drives for VG06 were listed as VG04 and the rest just seemed all mixed up. I restored the lvmtab file from frecover as well as the lvmconf directory. I also compared the all files in /stand and changed only one file. Then I thought that the device files were created differently but they check out ok.

So,
I put back in the original root drive and rebooted. All drives and volume groups mounted ok.

Question? What is happening? Is my tape backup missing something. I still believe
I have 2 iffy drives. The new root drive
worked well all day. I left my client, and we were copying VG04 to the Windows server.
Last night we copied VG06. VG01 thru VG06
are on a tape library backup using networker
software. VG06 is 146GB, VG04 is 46GB and VG05 is 26GB VG01, VG02, and VG03 are each
9GB.

To answer a few of the questions about power,
the server is powered on a ups circuit. I don't think they have the third power supply. In the past, they have had other drives power fail. I replaced them recreated the volume group and restored the data with backup tapes. This has happened about every 9 monthes for the last 4 years.
This is the first time it happened to the root drive.

Thanks for all of the suggestions so far.

John Gunnis

[Shaikh Imran]

hi,
I slightly confused by your above big post.
pls tell me points:
1) Pls Post your lvmtab,/etc/fstab & ioscan -fnC disk outputs
2) You are ok with the internal disks Y/N.
3) External Disks VG's report a mismatch with the corresponding PV's ?
4) By any chance while taking backup one of your external disk went offline ?
5) Have you received any errors while backup ?

Regards,

[John Gunnis]

Hi Shaikh,

To answer a few of your questions,


2) I have put back in the original root
drive that drops it's connection at random
times.

3) I replaced the bad external drive in VG04
and I was restoring the data, 42GB as of last night.

4) Yes this was possible.
5) I did not receive any error while creating
a backup or restoring VG00


My next plan of action is to place the Restored VG00 back in the system and do vgexports of vg04,vg05,vg06, saving the mapfiles and then doing vgimports.
I was told this is common when restoring VG00
using ignite that the disk layouts get mixed up.
Your point about question 1 will be forth coming. I plan to capture the disk layouts
before I replace the root drive VG00.

Thanks again.

[John Gunnis]

Hi,

I'm back. Our data restore failed last night. We are trying it again. It takes 10
hours to restore 42GB.

The reason my volume groups won't mount
after an ignite tape restore is the the
disk device files change. 8 drives have different nodes with different device file names. I hope I have included the original
devices. The new devices are:

brw-r----- 1 bin sys 31 0x04a000 c4t10d0
brw-r----- 1 bin sys 31 0x04b000 c4t11d0
brw-r----- 1 bin sys 31 0x048000 c4t8d0
brw-r----- 1 bin sys 31 0x049000 c4t9d0
brw-r----- 1 bin sys 31 0x05a000 c5t10d0
brw-r----- 1 bin sys 31 0x05b000 c5t11d0
brw-r----- 1 bin sys 31 0x058000 c5t8d0
brw-r----- 1 bin sys 31 0x059000 c5t9d0

all the other devices remained the same.
not included were:
31 0x06a000 c6t10d0
31 0x06b000 c6t11d0
31 0x068000 c6t8d0
31 0x069000 c6t9d0
31 0x07a000 c7t10d0
31 0x07b000 c7t11d0
31 0x078000 c7t8d0
31 0x079000 c7t9d0

One group of disks is in a surestore E cabinet.

I have make a new ignite tape.
If I replace the root drive and restore from this tape and the devices show up like above,
can I do a vgexport, then vgimport specifying
the new devices will it work or corrupt the data?

John Gunnis