harddisk replacement

[Ionut Grigorescu_2]

Hi gurus!

I have a HP-UX11.00 system with 2 k-class servers sharing the same disk array - I use also disk mirroring. I have rebooted one server from the Ignite tape then I have tried to rebuild the mirroring for the root disk. I have received an error "harddisk stale" after lvreduce command. Is that mean that I have to replace the disk? How can I check the integrity of a harddisk?
If the disk is bad and I replace it, do I have to rebuild mirroring also for the other server (the root disk is shared)?

Thank you in advance,
Ionu

[G. Vrijhoeven]

Hi,

you can use dd to check the integrity of the disk:

diskinfo -v /dev/rdsk/c?t?d?

output ok?

dd if=/dev/rdsk/c?t?d? of=/dev/null bs=4096k

If this command fails replace the disk, and reenable mirroring.

Gideon

[Ionut Grigorescu_2]

dd - invalid number specification in dd option

I understood that I have to rebuild mirroring, but also for the other server?

[G. Vrijhoeven]

Hu again,

Im am not sure. If you enable mirroring with MirrorUx on all the vg00 lvols ( lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c?t?d?)
Both disks shoud be idenical and if both the disks are accessable by both servers you are in business. But i do not know how to update the lvminfo on the other servers kernel.


Gideon

[Eugeny Brychkov]

Please check if you entered lowercase 'k' in the block size option 'bs=4096k'.
If this dd command will complete successfully (w/o errors like 'I/O error') then disk surface is physically accessible and you need to troubleshoot LVM structures
Eugeny

[Michael Duthie]

When we encounter stale PEs, it could be an indication of the hard disk
consists of bad segment. Use 'dd' test on the disk hard disk:

# dd if=/dev/rdsk/cXtYdZ of=/dev/null bs=64k

Take note of the number returns, multiply it with 64. The product of the
multiplication should be about the size of hard disk which is reported in
'diskinfo':

# diskinfo /dev/rdsk/cXtYdZ

Where are the VG00 disks? In the array?

[Ionut Grigorescu_2]

Yes, the vg00 disks are in the array.
I gave
#dd if=/dev/rdsk/c3t6d0 of=/dev/null bs=64k and I don't receive the prompt back!

So, I want to be sure:
- Shutdown both servers
- Replace disk
- Startup the server and - rebuild mirroring (for both?)

[Ionut Grigorescu_2]

it tooks some time for the output:

138930 + 1 records in
138930 + 1 records out

Does it mean that my harddisk is OK?
Mike, say something to give you the points for the command :-)

[Pete Randall]

Ionu,

With a small block size like 64K, dd will take a VERY long time to scan any decently sized disk - several hours at least. Try again with a block size of 1024k (or more).

Pete

[Ionut Grigorescu_2]

I forgot something:
with lvdisplay -v /dev/vg00/lvol3 (root filesystem)
on the first server the PE are in 'current' state but on the second (the one with the problem) the same command show me the PE's in 'stale' state..

[Michael Duthie · Accepted solution]

Ionu,

If the disk is 8.8gb, it looks ok.

What kind of array?

[Ionut Grigorescu_2]

The model of rack enclosure is A3312A, I don't know more, it has 2 power supplies on the left side and 2 columns each with 4 disks on the right.
All the disks are the same size - 8891556 kbytes - from diskinfo

[Pete Randall]

FWIW, the A3312A is commonly known as a Jamaica or HASS (High Availability Storage System). It's basically a JBOD with hot swappable drives, power supplies, and fans - no RAID capabilities.

How is this being shared between the two systems?

Pete

[T G Manikandan]

check this document for hte procedure to rebuild the mirror after replacing the bad hard disk

http://www1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&admit=-682735245+1037886324162+28353475&docId=200000062909815

[John Palmer]

It would be useful if you posted the output from the following command (both systems):

ioscan -funC disk
vgdisplay -v
strings /etc/lvmtab

To clarify an earlier reply, the time difference between running dd with a 64k against a 1024k blocksize (at least in HP-UX 11.00) appears from experiment to be negligible. Provided the blocksize is 8k or greater, a SEAGATE ST39173WC provides just under 15Mb/sec, SEAGATE ST39175LC's are better at about 19Mb/sec.

Regards,
John

[Ionut Grigorescu_2]

That's the output of the commands. I had problems only on /dev/vg00

[Pete Randall]

I'll ask the dumb question: don't stale lvol's need to be sync'd?

Pete

[Michael Duthie]

Ionut,

OK, so your internal root disk is mirrored to /dev/dsk/c3t6d0 which is on a FW HPPB card. If this disk is in a Jamaica disk enclosure it should be terminated and not on a shared bus.

Why did you boot from your ignite tape in the 1st place?
Did either /dev/dsk/c2t6d0 or /dev/dsk/c3t6d0 fail?
Are the extends still stale ?
Have you tried lvsync or vgsnyc?

[Ionut Grigorescu_2]

15 minutes before your question I started a vgsync vg00, Pete - it looks OK so far, I think you're the winner.
The story: On my servers is running a third vendor application. If something is wrong to the operating system or the application I have to call them and they try to solve the problem. If they cannot do it , they open a Helpdesk request to HP and HP call me back - sounds stupid but this is the contract. Rebuilding the mirroring was part of a faulty installation they have performed last week - it is not a live system, so it was no emergency. After redoing the mirroring the guy from the thrd party helpdesk realized that he forgot to make the mirror disk bootable, so he performed mkboot at the end, I have tried to reboot from ALT path- the system failed so the guy tried to undo mirroring and told me that he received after lvreduce this 'stale' state of the mirror, so they called HP and I received this morning a brand-new disk. But I don't want to use it if my old one is OK. That's all.
thanks Pete, it wasn't a dumb question at all!
Thank yo

[Ionut Grigorescu_2]

Mike:
I have rebooted from Ignite because swinstall has failed and I had as result a corrupted kernel, that was the only reason.

[Pete Randall]

OK, I'm going a little off topic here, but John is absolutely right about the blocksize difference being inconsequential (a couple of seconds difference for 8.8GB). The real difference comes if you don't specify a blocksize at all and dd defaults to 512 byte blocks - now that takes forever!

Pete

[John Palmer]

I note from your vgdisplay that c2t6d0 has 2169 total PE's whereas c3t6d0 has 2170. This sounds as though c3t6d0 hasn't been pvcreated with the -b flag, so you won't be able to boot from it.

Could you also post the output from lvdisplay -v /dev/vg00/lvol1 so that we can determine which disk is stale, and the output from setboot.

Regards,
John