A disk from one vg is "NO_HW" from "ioscan -funC disk"

[Mohammad Jeffry]

here is my lvmtab
root@myhpux[/]#strings /etc/lvmtab
.
.
.
.
/dev/vg05
/dev/dsk/c0t9d0
/dev/dsk/c0t11d0
/dev/dsk/c0t12d0
/dev/dsk/c0t13d0
/dev/dsk/c0t14d0
/dev/dsk/c0t15d0

and here is my ioscan -funC disk
.
.
.
disk 9 10/0.14.0 sdisk CLAIMED DEVICE SEAGATE ST318275LC
/dev/dsk/c0t14d0 /dev/rdsk/c0t14d0
disk 10 10/0.15.0 sdisk NO_HW DEVICE SEAGATE ST318436LC
/dev/dsk/c0t15d0 /dev/rdsk/c0t15d0
disk 6 10/12/5.2.0 sdisk CLAIMED DEVICE TOSHIBA CD-ROM XM-5701TA
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0

So the vg in question is vg05 and the missing disk is /dev/dsk/c0t15d0. So here is my vgdisplay -v for vg05

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t15d0":
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
--- Volume groups ---
VG Name /dev/vg05
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 18
Open LV 18
Max PV 16
Cur PV 6
Act PV 5
Max PE per PV 4342
VGDA 10
PE Size (Mbytes) 4
Total PE 8432
Alloc PE 8432
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg05/lvol1
LV Status available/syncd
LV Size (Mbytes) 2700
Current LE 675
Allocated PE 675
Used PV 1

LV Name /dev/vg05/lvol2
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 2

LV Name /dev/vg05/lvol3
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol4
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 2

LV Name /dev/vg05/lvol5
LV Status available/syncd
LV Size (Mbytes) 2700
Current LE 675
Allocated PE 675
Used PV 2

LV Name /dev/vg05/lvol6
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol7
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 2

LV Name /dev/vg05/lvol8
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol9
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol10
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol11
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol12
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol13
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol14
LV Status available/syncd
LV Size (Mbytes) 2400
Current LE 600
Allocated PE 600
Used PV 1

LV Name /dev/vg05/lvol15
LV Status available/syncd
LV Size (Mbytes) 4752
Current LE 1188
Allocated PE 1188
Used PV 0

LV Name /dev/vg05/lvol16
LV Status available/syncd
LV Size (Mbytes) 3500
Current LE 875
Allocated PE 875
Used PV 0

LV Name /dev/vg05/lvol17
LV Status available/syncd
LV Size (Mbytes) 3500
Current LE 875
Allocated PE 875
Used PV 0

LV Name /dev/vg05/lvol18
LV Status available/syncd
LV Size (Mbytes) 5136
Current LE 1284
Allocated PE 1284
Used PV 0


--- Physical volumes ---
PV Name /dev/dsk/c0t9d0
PV Status available
Total PE 1023
Free PE 0

PV Name /dev/dsk/c0t11d0
PV Status available
Total PE 1023
Free PE 0

PV Name /dev/dsk/c0t12d0
PV Status available
Total PE 1023
Free PE 0

PV Name /dev/dsk/c0t13d0
PV Status available
Total PE 1023
Free PE 0

PV Name /dev/dsk/c0t14d0
PV Status available
Total PE 4340
Free PE 0

I'm doomed ain't I? How can I salvage the data in /dev/dsk/c0t15d0? Is the other data in other lvm still usable?

[Devender Khatana · Accepted solution]

Hi,

All other data is still very well usable but as the disk is showing NO_HW in ioscan it will be not possible to recover data unless the LVOLs in it were mirrored. This you can confirm by

#lvdisplay -v /dev/vg05/lvol*

HTH,
Devender

[MarkSyder]

Is the disc hot-swappable? Taking it out and reseating it may cure the problem.

If not, do you have a good backup?

Mark Syder (like the drink but spelt different)

[Sivakumar TS]

Hi Mohammad,

normally NO_HW means the disk in not responding at all, as suggested you can refix if it is a hotswap and run #ioscan again. If it gets claimed you can deactivate and activate the vg.

Regards,

Siva.

[Fabio Ettore]

Hi Mohammed,

welcome to ITRC forum.
From the man ioscan:

NO_HW the hardware at this address is no longer responding

So the disk should be checked (connection, terminator and so on...)

Hope you have a good backup.

When you solve your problem don't forget to assign points...this is a great feature of this forum. Thanks for your collaboration!

Regards,
Fabio

[Prashant Zanwar_4]

NO_HW is state where it means there is no hardware present at that particular hardware address on host..

Go and reach the host, and check that particular address if feasible.. That disk would have LED not blinking or contineously on, you can take disk off if it is hot swappable and push it back and see what it does (It might start working again)..

On the host you can check
dmesh & see what errors are shown, check for pvdisplay of this device, it will show not available, but it can show what VG it belonged too..

If it is part of mirror, you are safe, or else some file system is screwed...
run mount -a & check for cannot mount errors..

Call HP CE if you come to know that disk is bad..

Good luk
Thanks
Prashant

[Patrice Le Guyader]

Hi Mohammed,

welcome to ITRC forum.

Sivakumar I wonder if once the disk is back at CLAIMED state you just only need to do a vgchange -a y .

Hope this helps
Pat

[Shahul]

Hi,

I think once you replace the hard disk, then you may need to restore the volume group information to that particular PV. You can restore either from the default location /etc/lvmconf/vgname or you can specify alternate location.

See vgcfgrestore manual page for more details.

Best of luck
Shahul

[Torsten.]

Once the disk is replaced and the config is restored (vgcfgrestore), run a

pvdisplay -v /dev/dsk/c0t15d0

and look for

--- Distribution of physical volume ---
LV Name LE of LV PE for LV

Make a new filesystem on all listed LVOLs and restore the data from your backup, because your disks are not mirrored.

[Mohammad Jeffry]

Thanks for the tips. The disk is blinking and I think there is nothing else to do for /dev/dsk/c0t15d0 except replacing it. This is concluded from the photos of the disk, vgdisplay result and lvdisplay result. I don't want to flood the pix and the output here. So I put it at my blog.
http://linuxlah.blogspot.com/2006/03/one-disk-that-gone.html

So how can I replace the disk without destroying the vg05 and other lv on that vg that is not affected by the /dev/dsk/c0t15d0 corruption?

p/s: I got the spare disk already in hand. taken from another server ;)
p/ss:Yes I have backup.

[Devender Khatana]

Hi,

The disk at which the LED is glowing continuously as supected to be the defective one, but it is not the case always. If you are not sure that the best way to do is to physically remove it and see in ioscan if something else do not show starting NO_HW. Once you are confirm that this is the disk then you can go ahead in replacing it.

Plug in new disk in same slot. and use vgcfgrestore to make it accessible to the VG.
#vgcfgrestore -n vg05 /dev/rdsk/c0t15d0

Now create new file systems on all the LVOLs which were there on this disk and were not mirroed. From the output I found these to be lvol4,5,6,7,8. Confirm this once and then mount the file systems and restore backup.
#newfs -F vxfs /dev/vg05/rlvol4

Repeat for other LVOLs and
#mountall

#bdf to confirm.

This will not disturb other LVOLs in the VG.

Here is official disk replacement guide for reference.
http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf

HTH,
Devender

[Mohammad Jeffry]

tq. you guys rocks!

[Josiah Henline]

Please note the usage of "ioscan".

If you reseat a drive, running "ioscan -funCdisk" will not scan the hardware. The "-u" option scans the kernel for a list of usable devices. It does not physically scan the hardware.

Try "ioscan -fnCdisk" instead. It will scan the hardware an see if the disk is responding after a reseat.