topic Re: Problem with the mirror in Operating System - HP-UX

Problem with the mirror

JakobK — Mon, 21 Dec 2009 10:04:27 GMT

Hello,

i changed the /dev/dsk/c5t1d0. After the changing, I have put on the mirror for all LV's again.

If I liked to mirror LV /dev/vg00/lvora10g, I get in LE only one mirror stale.
00342 /dev/dsk/c4t1d0 02646 current /dev/dsk/c5t1d0 00831 stale
The /dev/dsk/c5t1d0 is new disk.
I have run a fsck on /dev/vg00/lvora10g and there was okay. What can I do, to get this stale LE away?
Many thanks.
Yours sincerely
K. Jakob

Re: Problem with the mirror

Torsten. — Mon, 21 Dec 2009 10:07:20 GMT

You can check the disk with "dd" in order to verify the disk. Maybe it is bad again? Another thing to try would be to un- and remirror this LVOL.

Re: Problem with the mirror

Johnson Punniyalingam — Mon, 21 Dec 2009 10:22:52 GMT

>>What can I do, to get this stale LE away?<<

stale extends refer to sometimes "Bad disk"
if you can ioscan , diskinfo and dd a drive normally than there is no hardware problem

Please check disk really bad by "dd"

sample :-

dd if=/dev/dsk/c2t1d0 of=/dev/null bs=128k

Re: Problem with the mirror

Johnson Punniyalingam — Mon, 21 Dec 2009 10:25:23 GMT

ITRC member since: June 08, 2004
Last contribution date: December 21, 2009
I have assigned points to 0 of 15 responses to my questions.

Finally can you assign points,?? its been very long time your never assign points.

Its kind of giving thanks for your people support and respond your Question and problem

Re: Problem with the mirror

JakobK — Mon, 21 Dec 2009 10:30:12 GMT

dd is okay. dd if=/dev/dsk/c5t1d0 of=/dev/null bs=64.
I mirrored the LV already several times, after beboot, after fsck, but the error is the same and on the same LE (00342 /dev/dsk/c4t1d0 current and the matching LE from c5t1d0 is stale).

Re: Problem with the mirror

Johnson Punniyalingam — Mon, 21 Dec 2009 10:36:11 GMT

so its confirmed - no hardware . so you can try vgcfgrestore.

vgcfgrestore -n /dev/vg00 /dev/rdsk/c5t1d0
vgchange -a y vg00
vgsync vg00

Re: Problem with the mirror

JakobK — Mon, 21 Dec 2009 10:44:38 GMT

Hello Johnson,

thanks, but the vgrestore reports the message:
Cannot restore Physical Volumes /dev/dsk/c5t1d0,
Detach the PV ..
The c5t1d0 is already successfully integrated into other LV as a mirror.

Yours sincerely
K. Jakob

Re: Problem with the mirror

sujit kumar singh — Mon, 21 Dec 2009 11:14:21 GMT

Hi

seems as if the source PE is not being read,
try using dd to read the source disk.
might be that LE is not having some filesystem data in that,

regards
sujit

Re: Problem with the mirror

JakobK — Mon, 21 Dec 2009 13:07:12 GMT

Hello Sujit,

i have make a dd on c4t1d0 source and dd on c5t1d0 target with bs=256k and i get the same result: 280028+1 records in and 280028+1 records out

Regard Kati

Re: Problem with the mirror

Viktor Balogh — Tue, 22 Dec 2009 13:44:23 GMT

>i have make a dd on c4t1d0 source and dd on c5t1d0 target with bs=256k and i get the same result: 280028+1 records in and 280028+1 records out

instead of using another disk, you could test the drive by using /dev/null and /dev/zero like this:

read test:

# dd if=/dev/dsk/c5t1d0 of=/dev/null bs=256k

write test:

# dd if=/dev/zero of=/dev/dsk/c5t1d0 bs=256k

this way you can eliminate other error sources and the test goes much faster.

Re: Problem with the mirror

R.K. # — Tue, 22 Dec 2009 14:13:02 GMT

Hi Jakob,

>> Cannot restore Physical Volumes /dev/dsk/c5t1d0
Need to do "pvchange -a n "

>> same and on the same LE
If this is the case, why not perform write test as mentioned by Viktor. (You already did read test that came up fine). But that will destroy other mirrors.

Another experiment that you can do is un-mirror all LVs in c5t1d0 and re-mirror them, changing ORDER of mirroring this time. So we can see if the same LE (xyz) gets stale or same LV (lvora10g) gets satle extent.

As far as I know, sometimes in rare cases LVM looks as if stale extent is in disk1 (from lvdisplay) whereas it comes out to be in disk2 (not shown in lvdisplay).

All the best..

Re: Problem with the mirror

R.K. # — Tue, 22 Dec 2009 14:22:49 GMT

Hi Again..

Take an example of two mirrored disks c4t0d0 and c5t0d0..

There are some stale PE in one disk c4t0d0, but it's not necessary that the disk c4t0d0 is faulty. It's only mean LVM can't sync the data in the mirrored PEs, the 'current' PE has the latest data, the 'stale' PE has old data. The 'current' PE maybe bad such as can't be read from it, then sync failed.

To find the fault disk, we can use dd on each disk, if the disk has I/O error, it's the faulty disk.

# dd if=/dev/dsk/c4t0d0 of=/dev/null bs=1024k
# dd if=/dev/dsk/c5t0d0 of=/dev/null bs=1024k

And some hardware log such as EMS will show error message. In this case, Media failure happened in the 'current' disk.
Check /var/opt/resmon/log/event.log file:

Now, we know that the disk c5t0d0 with 'currect' status is bad. If we use normal procedure to replace disk c5t0d0 in mirror, lvreduce c5t0d0 will fail because c4t0d0 has stale PE and c5t0d0 is current.

There is a tool named lvunstale which can help us to change the stale state.

It will switch which mirror copy is stale. Now, we can replace the bad disk as per normal process.

Re: Problem with the mirror

JakobK — Tue, 22 Dec 2009 17:10:13 GMT

Hello,

i have reduce the mirror of all LV and then
i have lvextend the mirror new,started with lvora10g. The result was successful.
Before, i have made the dd of c4t0d0 and this dd did not come back and can't kill -9.
--> reboot and all okay.

(only Infomation: on the oldsyslog i has found the mesg 037 vx_metaioerr on lvora10g)

Many thanks. Merry Christmas!

Yours sincerely
Kati

Re: Problem with the mirror

JakobK — Tue, 22 Dec 2009 17:10:54 GMT

Thanks