topic Re: Problem removing or adding vg01 in Operating System - HP-UX

Problem removing or adding vg01

John Jimenez — Fri, 16 Jan 2004 20:16:41 GMT

I am having issues with my future disaster recovery box off site. The power went out and now vg01 is messed up. It seems like information is in the /etc/lvmtab but no where else preventing me from removing or adding it. VG01 does not show up in SAM, yet it will not let me add it because it is already in my lvmtab, but I cannot remove it from my lvmtab.

here are some of the messages.

# lvdisplay /dev/vg01/lvol1
lvdisplay: Couldn't query logical volume "/dev/vg01/lvol1":
Volume group not activated.

lvdisplay: Cannot display logical volume "/dev/vg01/lvol1".
K-380:root /etc
# vgdisplay /dev/vg01
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "/dev/vg01".
# strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t6d0
/dev/dsk/c2t5d0
/dev/vg01
/dev/dsk/c3t0d0
/dev/dsk/c3t1d0
/dev/dsk/c3t2d0
/dev/dsk/c3t3d0
/dev/dsk/c9t0d0
/dev/dsk/c9t1d0
/dev/dsk/c9t2d0
# vgremove vg01
vgremove: Volume group not activated.

Re: Problem removing or adding vg01

Chris Watkins_1 — Fri, 16 Jan 2004 20:26:36 GMT

What's the response of:
#vgchange -a y vg01

???

Re: Problem removing or adding vg01

John Jimenez — Fri, 16 Jan 2004 20:29:46 GMT

# vgchange -a y vg01
vgchange: Couldn't activate volume group "vg01":
I/O error while reading the VGDA.

Re: Problem removing or adding vg01

Joseph Loo — Fri, 16 Jan 2004 20:51:01 GMT

hi,

any problem issue pvdisplay for the disk in vg01:

#pvdisplay -v /dev/dsk/c3tXd0
#pvdisplay -v /dev/dsk/c9tXd0

regards.

Re: Problem removing or adding vg01

Bill Hassell — Fri, 16 Jan 2004 20:52:22 GMT

The I/O error is at the beginning of the disk. This error is what has caused everything else to fail. The volume group vg01 cannot be activated because the information cannot be read. Most likely there are error messages in /var/adm/syslog/syslog.log concerning the disk with the problem. The power failure may have been very bad (ie, down/up/down/up lots of times per second) which will easily damage computers and disks. That's probably the most important reason to have a UPS, to filter out bad power failures.

Re: Problem removing or adding vg01

Joseph Loo — Fri, 16 Jan 2004 20:54:56 GMT

hi,

another thing, does the volume group, vg01 contain alternate links?

if so, refer to this doc:

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000063235797

regards.

Re: Problem removing or adding vg01

Chris Watkins_1 — Fri, 16 Jan 2004 20:59:36 GMT

Did you try an export? Grab a mapfile first, just in case it works.

# vgexport -p -v -s -m /etc/lvmconf/vg01mapfile
If that worked ok, try simply exporting and importing:

#vgexport vg01
(If that succeeded)
#vgimport -s -v -m /etc/lvmconf/vg01mapfile

If that doesn't work... try this:
# vgscan -v -p
See if the disks for vg01 match what you're expecting.

If so, I'd try dd'ing from each, to see if all are "OK".
If so, you may have to do a vgcfgrestore on each to get it right.

(Though I'd try moving /etc/lvmtab to /etc/lvmtab.old,
and doing a "vgscan" before I went that route.)

It's also possible you simply have a bad disk/disks.

Re: Problem removing or adding vg01

T. M. Louah — Sun, 18 Jan 2004 19:30:53 GMT

Also troubleshoot the disks with ioscan & diskinfo check the size is well reported by each disk, a size of 0:zero is a sign of bad drive. An exaustif test may be required to proove that disks are good by:
# dd if=/dev/rdsk/cXtYdZ of=/dev/null
this should report same record IN=OUT
Now you may try to re-play LVM info on disks by:
# vgcfgrestore -n vg01 /dev/rdsk/cXtYdZ ..
# vgchange -a y vg01

Good luck,

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 12:46:32 GMT

Joseph,
here is what I get
# pvdisplay -v /dev/dsk/c3t1d0
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t0d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t1d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t2d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t0d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t1d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t2d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query physical volume "/dev/dsk/c9t3d0":
The specified path does not correspond to physical volume attached to
this volume group
pvdisplay: Warning: couldn't query all of the physical volumes.
pvdisplay: Couldn't retrieve the names of the physical volumes
belonging to volume group "/dev/vg01".
pvdisplay: Cannot display physical volume "/dev/dsk/c3t1d0".

Re: Problem removing or adding vg01

Pete Randall — Tue, 20 Jan 2004 12:53:16 GMT

John,

As Bill Hassell pointed out, it sounds like you have a problem with the disk. Try exercising it with the dd command:

dd if=/dev/rdsk/c3t0d0 of=/dev/null bs=1024k

If you get an I/O error, you've got a bad disk.

Pete

Re: Problem removing or adding vg01

Sridhar Bhaskarla — Tue, 20 Jan 2004 12:58:25 GMT

Hi John,

Do the following to see these disks are really hosed up.

1. DO a 'vgexport -p -v -s -m /tmp/vg01.map vg01'. (do not forget -p option in the above)

This will put a vgid string in /tmp/vg01.map.

2. On the disks corresponding to vg01 (from strings /etc/lvmtab, do

echo 0x2010?2X|adb /dev/dsk/cxtydz|sed -e 's/2010://' -e 's/[ ]*//g'

[ ] is [] in the next -e

If you get errors, then most likely that your PVs got messed up. You may have to restore the data from your backups.

-Sri

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 13:07:32 GMT

Bill,
This system has a UPS on it, but we recently moved it off site to be used for disaster recovery. The room has limited access, but still someone looked like they stepped on the UPS power chord. I have rebooted serveral times since this has happend, but this is something that keep showing up every 15 or 20 minutes on the syslog

Jan 17 06:03:17 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/4/0/0.1.0.255.0.0.0;
Jan 17 06:03:30 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/1/0/0/1/1.6.0;
Jan 17 06:03:43 diamk380 syslog: CVSDM; DEVICE ADDED Event Code=198; Description
=Device Added; Hardware Address=1/0/0/3/0.6.0;

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 13:13:37 GMT

# dd if=/dev/rdsk/c3t0d0 of=/dev/null bs=1024k
dd read error: I/O error
0+0 records in
0+0 records out
K-380:root /var/adm/syslog

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 13:16:33 GMT

I had Ioscaned it and it always comes up claimed

disk 12 10/4/4.0.0 disc3 CLAIMED DEVICE SEAGATE ST34371W
/dev/dsk/c3t0d0 /dev/rdsk/c3t0d0
/dev/floppy/c3t0d0 /dev/rfloppy/c3t0d0

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 13:27:42 GMT

The data is not important. I need to create another volume to mount /home into. The ioscan shows that these disk are still claimed. So do you think that these disks are totally hosed, or is there a way to get rid of vg01 all together and start all over. Also, the lvmtab is set to read only. Can I chmod it and then remove these lines?

Re: Problem removing or adding vg01

John Jimenez — Tue, 20 Jan 2004 14:00:03 GMT

You are right The first disk is bad. This second one looks good. I can run the DD command and see which disks are still good. But is there any way to remove the bad disks from the lvmtab?

# dd if=/dev/rdsk/c3t1d0 of=/dev/null bs=1024k
4095+1 records in
4095+1 records out
K-380:root /etc

Re: Problem removing or adding vg01

Sridhar Bhaskarla — Tue, 20 Jan 2004 14:25:59 GMT

Hi,

To get rid of the volume group, you simply do "vgexport vg01". This will take out vg01. It should not give any errors as the volume group is in deactivated state.

You could do a vgcfgrestore after replacing the bad disks. Since you want to destroy the data anyway, I would suggest you go ahead do a vgexport.

You will need to run 'pvcreate -f' on the good disks later whiel configuring them into a new volume group.

-Sri

Re: Problem removing or adding vg01

John Jimenez — Wed, 21 Jan 2004 16:05:40 GMT

Thanks you guys for helping me identify the bad disk, and vgexporting the info in the lvmtab...