Problem removing or adding vg01

[John Jimenez]

I am having issues with my future disaster recovery box off site. The power went out and now vg01 is messed up. It seems like information is in the /etc/lvmtab but no where else preventing me from removing or adding it. VG01 does not show up in SAM, yet it will not let me add it because it is already in my lvmtab, but I cannot remove it from my lvmtab.

here are some of the messages.

# lvdisplay /dev/vg01/lvol1
lvdisplay: Couldn't query logical volume "/dev/vg01/lvol1":
Volume group not activated.

lvdisplay: Cannot display logical volume "/dev/vg01/lvol1".
K-380:root /etc
# vgdisplay /dev/vg01
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "/dev/vg01".
# strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t6d0
/dev/dsk/c2t5d0
/dev/vg01
/dev/dsk/c3t0d0
/dev/dsk/c3t1d0
/dev/dsk/c3t2d0
/dev/dsk/c3t3d0
/dev/dsk/c9t0d0
/dev/dsk/c9t1d0
/dev/dsk/c9t2d0
# vgremove vg01
vgremove: Volume group not activated.

[Chris Watkins_1]

What's the response of:
#vgchange -a y vg01

???

[John Jimenez]

# vgchange -a y vg01
vgchange: Couldn't activate volume group "vg01":
I/O error while reading the VGDA.

[Joseph Loo]

hi,

any problem issue pvdisplay for the disk in vg01:

#pvdisplay -v /dev/dsk/c3tXd0
#pvdisplay -v /dev/dsk/c9tXd0

regards.

[Bill Hassell]

The I/O error is at the beginning of the disk. This error is what has caused everything else to fail. The volume group vg01 cannot be activated because the information cannot be read. Most likely there are error messages in /var/adm/syslog/syslog.log concerning the disk with the problem. The power failure may have been very bad (ie, down/up/down/up lots of times per second) which will easily damage computers and disks. That's probably the most important reason to have a UPS, to filter out bad power failures.

[Joseph Loo]

hi,

another thing, does the volume group, vg01 contain alternate links?

if so, refer to this doc:

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000063235797

regards.

[Chris Watkins_1 · Accepted solution]

Did you try an export? Grab a mapfile first, just in case it works.

# vgexport -p -v -s -m /etc/lvmconf/vg01mapfile
If that worked ok, try simply exporting and importing:

#vgexport vg01
(If that succeeded)
#vgimport -s -v -m /etc/lvmconf/vg01mapfile


If that doesn't work... try this:
# vgscan -v -p
See if the disks for vg01 match what you're expecting.

If so, I'd try dd'ing from each, to see if all are "OK".
If so, you may have to do a vgcfgrestore on each to get it right.

(Though I'd try moving /etc/lvmtab to /etc/lvmtab.old,
and doing a "vgscan" before I went that route.)

It's also possible you simply have a bad disk/disks.

[T. M. Louah]

Also troubleshoot the disks with ioscan & diskinfo check the size is well reported by each disk, a size of 0:zero is a sign of bad drive. An exaustif test may be required to proove that disks are good by:
# dd if=/dev/rdsk/cXtYdZ of=/dev/null
this should report same record IN=OUT
Now you may try to re-play LVM info on disks by:
# vgcfgrestore -n vg01 /dev/rdsk/cXtYdZ ..
# vgchange -a y vg01

Good luck,

[John Jimenez]

Joseph,
here is what I get
# pvdisplay -v /dev/dsk/c3t1d0
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t0d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t1d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t2d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t0d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t1d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t2d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t3d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg01".
pvdisplay: Cannot display physical volume "/dev/dsk/c3t1d0".

[Pete Randall]

John,

As Bill Hassell pointed out, it sounds like you have a problem with the disk. Try exercising it with the dd command:

dd if=/dev/rdsk/c3t0d0 of=/dev/null bs=1024k

If you get an I/O error, you've got a bad disk.


Pete

[Sridhar Bhaskarla]

Hi John,

Do the following to see these disks are really hosed up.

1. DO a 'vgexport -p -v -s -m /tmp/vg01.map vg01'. (do not forget -p option in the above)

This will put a vgid string in /tmp/vg01.map.

2. On the disks corresponding to vg01 (from strings /etc/lvmtab, do

echo 0x2010?2X|adb /dev/dsk/cxtydz|sed -e 's/2010://' -e 's/[ ]*//g'

[ ] is [] in the next -e

If you get errors, then most likely that your PVs got messed up. You may have to restore the data from your backups.

-Sri

[John Jimenez]

Bill,
This system has a UPS on it, but we recently moved it off site to be used for disaster recovery. The room has limited access, but still someone looked like they stepped on the UPS power chord. I have rebooted serveral times since this has happend, but this is something that keep showing up every 15 or 20 minutes on the syslog


Jan 17 06:03:17 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/4/0/0.1.0.255.0.0.0;
Jan 17 06:03:30 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/1/0/0/1/1.6.0;
Jan 17 06:03:43 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/0/3/0.6.0;

[John Jimenez]

# dd if=/dev/rdsk/c3t0d0 of=/dev/null bs=1024k
dd read error: I/O error
0+0 records in
0+0 records out
K-380:root /var/adm/syslog

[John Jimenez]

I had Ioscaned it and it always comes up claimed

disk 12 10/4/4.0.0 disc3 CLAIMED DEVICE SEAGATE ST34371W
/dev/dsk/c3t0d0 /dev/rdsk/c3t0d0
/dev/floppy/c3t0d0 /dev/rfloppy/c3t0d0

[John Jimenez]

The data is not important. I need to create another volume to mount /home into. The ioscan shows that these disk are still claimed. So do you think that these disks are totally hosed, or is there a way to get rid of vg01 all together and start all over. Also, the lvmtab is set to read only. Can I chmod it and then remove these lines?

[John Jimenez]

You are right The first disk is bad. This second one looks good. I can run the DD command and see which disks are still good. But is there any way to remove the bad disks from the lvmtab?

# dd if=/dev/rdsk/c3t1d0 of=/dev/null bs=1024k
4095+1 records in
4095+1 records out
K-380:root /etc

[Sridhar Bhaskarla]

Hi,

To get rid of the volume group, you simply do "vgexport vg01". This will take out vg01. It should not give any errors as the volume group is in deactivated state.

You could do a vgcfgrestore after replacing the bad disks. Since you want to destroy the data anyway, I would suggest you go ahead do a vgexport.

You will need to run 'pvcreate -f' on the good disks later whiel configuring them into a new volume group.

-Sri

[John Jimenez]

Thanks you guys for helping me identify the bad disk, and vgexporting the info in the lvmtab...