HELP! How do I get a disk back into a VG?

[Ray Humpage]

We lost power to our data center. After the crash I can't activate a VG because it says the disk can't be queried. Is there a way to add it back in without losing the data on it?
It says it's used by DLVM. Not sure why. See attachment.

nmhhrprd(/etc)# vgchange -a y /dev/vg06
vgchange: Warning: couldn't query physical volume "/dev/dsk/c6t6d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg06":
Quorum not present, or some physical volume(s) are missing.

[Patrick Wallek]

What does a 'diskinfo' show for the disk? Does 'ioscan' show it as NO_HW or CLAIMED? It could be that the disk did not come up after the crash and has gone bad. It may need to be replaced.

[Ray Humpage]

It shows claimed.

SCSI describe of /dev/rdsk/c6t6d0:
vendor: HP 36.4G
product id: ST336752FC
type: direct access
size: 35566480 Kbytes
bytes per sector: 512
rev level: HP03
blocks per disk: 71132960
ISO version: 0
ECMA version: 0
ANSI version: 3
removable media: no
response format: 2

[Ray Humpage]

I can't activate the VG to do a vgextend. Should I export it and then import it?

[Denver Osborn]

I wouldn't try to export/import yet... 1st try to vgcfgrestore the lvm header to the disk.


vgcfgrestore -l -f /etc/lvmconf/vg06.conf

with the -l you should see c6t6d0... if so, restore the header to it.

vgcfgrestore -n vg06 /dev/rdsk/c6t6d0

then try to activate the vg... if you still have probs, try posting the output of vgcfgretore -l -f /etc/lvmconf/vg06.conf and strings /etc/lvmtab showing the PV's for vg06.

hope this helps,
-denver

[Ray Humpage]

Seems to have worked but not worked.

See attachment.

[Denver Osborn]

Is this node part of a cluster or activated from any other node?

-denver

[Denver Osborn]

meant to ask if it's possible that the VG could be activated on another node.

form the looks of it vg06 has two PVs. Primary and Mirror. Problems are with c6t6d0.

Are there any other disks on that HBA that are showing imilar problems? Just trying to rule out a h/w issue w/ the hba, gbic, disk, etc that may have happened from your outage.

The vgcfgrestore worked fine -> it completed w/ success. Then vgchange was good and let lvm know the disk was there, but it went away again... we're just not sure why yet. :)

Any SCSI errors, lvm powerfail messages logged in the syslog... how about EMS alerts?

[Sandman!]

Stale extents on a PV indicate that the disk might have gone bad. Check for errors in the syslog.log file related to that disk.

[Adisuria Wangsadinata_1]

Hi Ray,

Looks like /dev/dsk/c6t6d0 have a stale status on the disk, this means the disk have an error. That's why you cannot activate the vg06.

It's a good way to prepare your vg06's backup, this one needed when you restore the data.

The other way around, is to check whether the disk c5t6d0 is a mirror with c6t6d0. If this happened, you just need to change the harddisk and the data is 99% save.

# lvdisplay -v /dev/vg06/ps*

Hope this information can help.

Cheers,
AW

[Raj D.]

Ray,

Check any error in syslog and dmesg, possibly you will have some SCSI error or lbot error in syslog,

use,
# cat /var/adm/syslog.log | grep -i vmunix

# cat /var/adm/OLDsyslog.log | grep -i lbot


And also check for stale PE on the disk c6t6d0,
# lvdisplay -v /dev/vg06/pstemp01 | grep stale | wc -l

You may need to replace the disk and restore the lvm configuration and to restore data of the 2 lvol.

Cheers,
Raj.