transferring volume group map not working..

[Mark Henry_1]

Hi All,

I have a raid shared by 4 servers - different volume groups are activated on the different machines through serviceguard in exclusive mode.
When I update a volume group (add new PV's) I typically export a map, copy it to the other servers, export the 2nd servers map and import the new mapfile.

However it's not working - I export the map on one machine and get..

vgexport -p -v -s -m /tmp/vg_one.map vg_one
Beginning the export process on Volume Group "vg_vob".
/dev/dsk/c12t0d2
/dev/dsk/c12t0d3
/dev/dsk/c10t0d2
/dev/dsk/c10t0d3
/dev/dsk/c10t1d2
/dev/dsk/c12t1d2
/dev/dsk/c10t1d6
/dev/dsk/c12t1d6
/dev/dsk/c10t1d7
/dev/dsk/c12t1d7

Copy it to the other, run:
vgexport vg_one
mkdir /dev/vg_one
mknod /dev/vg_one/group c 64 0x020000
vgimport -v -s -m vg_one.map /dev/vg_one

And when the package fails over I get an error that the filesystem is greater than the device...
When I check the vg contents running a vgexport similar to the first one, I only get 6 devices, not 10 (2 x redunandant controllers)..

Why is the map not transferring the info?

Thx,

Mark

[Sanjay_6]

Hi Mark,

This is what you have to do,

Do a vgexport on one node. copy the map file to the other node. vgexport the current vg info from the other node. Now vgimport the new map file,

on node 1

vgexport -p -s -m /dev/vg_name /tmp/vg_name.map

rcp /tmp/vg_name.map node2:/tmp

on node 2

vgexport /dev/vg_name
mkdir /dev/vg_name
mknod /dev/vg_name/group c 64 0x0n0000
vgimport -s -m /tmp/vg_name.map /dev/vg_name

Or you can do this,

on node 1 print a list of all the device files under /dev/vg_name. Go to node 2 and create the device files for the new lv's added to this vg. Don't forget to create the raw device files too. And make sure the device files minor nos are same on both systems.

You should also make sure that you have replicated the updated package control file from node 1 to node2 since you would have added the new lv's to the package control file on node1 but they need to be added to the package control file on other nodes too.

Hope this helps.

Regds

[Mark Henry_1]

Sanjay,

Step one is exactly as I described it - I think though in the vgexport, the map file should come before the vg device file name?

This I have done but it's not working.

I did an insf -e to make sure the device files are present for the other machine and they are there.

Step 2 shows the lvol names, not the disk devices..

No lvols have changed though just increased the size of the existing lvols..

Mark

[Sanjay_6]

Hi Mark,

Yes you are correct, the map file name should come first in the vgexport command before the name of the vg.

Can you attach your package log file from /etc/cmcluster/pkg_name/pkh_control_file.log

Thanks

[Roger Baptiste]

Mark,

Does the VGIMPORT command complete without any errors? If it does, then the VG setup part is considered succesful.

What is the exact error you are getting? Can it be with your Package control files not being set correctly on the other nodes?

The less number of devices can be due to lack of "alternate pv paths" on the other nodes. It should't be a issue, since as long as there is atleast one link available for each of the disks in the VG, the VG should be accessible.

-raj

[Mark Henry_1]

Sanjay,

Thanks for your help..

Please find attached part of the control log file that results when trying to start the package. From the log, it appears that the machine doesn't know about the recently expanded vg - even though I have imported it (several times).

Raj,
Yes, the import appears to succeed ok..

Beginning the import process on Volume Group "/dev/vg_vob".
Logical volume "/dev/vg_vob/lvol1" has been successfully created
with lv number 1.
Logical volume "/dev/vg_vob/lvol3" has been successfully created
with lv number 3.
Volume group "/dev/vg_vob" has been successfully created.

Note that there is a 2nd volume group associated with the package but I've verified that it's map has been imported successfully too..

Many thanks guys,

Mark

[Sanjay_6]

Hi Mark,

What are the filesystem types, vxfs or hfs. If you are using vxfs filesystem make sure the entry in /etc/default/fs says

LOCAL=vxfs

Hope this helps.

Regds

[James R. Ferguson]

Hi Mark:

Verify your '/etc/fstab' entires for the filesystems in question. Make sure that the type (e.g. vxfs) is applicatable. That is, make sure that the structure of the filesystem matches that declared in /etc/fstab. If not, correct it and try again.

Regards!

...JRF...

[Luc Bussieres]

Mark,

I look at your package log, but since this is a problem with LVM, inside a package I would suggest you to debug the LVM part of it outside the package first and when you are sure this is working perfectly then you. try within the package.

To do this:

1) stop the package if it's currently running.
2) "declusterized" the VG:
vgchange -c n /dev/vg_vob
3) On the first node :
a) activate the VG:
vgchange -a y /dev/vg_vob
b) verify that all the disk are present:
vgdisplay /dev/vg_vob
and look for Act PV and Cur PV they
should match
c) fsck all filesystems
If you have an error message at this
point what is it?
d) when done deactivate the VG
vgchange -a n /dev/vg_vob
4) repeat step 3 with the other node(s)
5) when completed sucessufully "clusterize" the VG:
vgchange -c y /dev/vg_vob


Now it should work within the package. If it fail at this point could you attach both the package.cntl and package.cntl.log file.


Luc

[Rita C Workman]

The two errors I make most often are:
1. I entered the wrong minor number when re-creating the vg on the second node !
2. I copied the wrong (older version) of the mapfile.

Node 1
Make note of minor # 0x0n0000
vgexport -pvs -m /etc/lvmconf/vg.map /dev/vg
rcp /etc/lvmconf/vg.map node2_server:/etc/lvmconf

Node 2
vgexport /dev/vg
mkdir /dev/vg
mknod /dev/vg/group c 64 0x0n0000
..(same unique # from above-make sure it's not duplicated anywhere on any node in cluster !)
vgimport -vs -m /etc/lvmconf/vg.map /dev/vg

..to check run strings /etc/lvmtab and ensure everything is there..

Just a thought,
Rita

[Mark Henry_1]

Guys,

The /etc/default/fs entry is LOCAL=vxfs

There's nothing in /etc/fstab, but it's SG controllled.. (where in control script)?? Either way this machine has worked fine in the past, until I expanded the VG - I just cant get the VG info into the second machine - again, if I do a verbose vgexport on the 2nd machine (into which I recently imported the 1st machine's map), it only shows 6 devices instead of 10 - it's just not importing the info properly, even though the commands are succeding..

Sanjay, can you expand on how to do it manually? Again, all the device files exist, and no lvols have been added (just expanded..)

Thx,

Mark

[Sanjay_6]

Hi Mark,

You have to make sure that the two nodes in question see the disk / PV's as the same device files. Verify this by using ioscan and make sure the disks are seen on both the system.

Now one the first node, after you have added pv's and extended the lv's, do a vgexport,

vgexport -p -s -m /tmpvg_name.map /dev/vg_name
rcp /tmp/vg_name.map node2:/tmp

on node2

vgexport /dev/vg_name
mkdir /dev/vg_name
mknod /dev/vg_name/group c 64 0x0n0000
vgimport -s -m /tmp/vg_name.map /dev/vg_name

If you are using PVG strict mirroring,

cd /tmp
rcp node1:/etc/lvmpvg .

edit /etc/lvmpvg and add the vg info for /dev/vg_name from /tmp/lvmpvg

now Try to activate the vg and mount the lv manually,

vgchange -a e /dev/vg_name
vgdisplay -v /dev/vg_name
mount /dev/vg_name/lv_name_1 /tmp_mnt
bdf
umount /tmp_mnt

Do this mount and unmount for all the lv's in the VG. Also do a "strings /etc/lvmtab" and make sure the pv's belonging to the vg's are indeed available over there.

Hope this helps.

Regds

[Sanjay_6]

Hi Mark,

Here is a link from itrc discussing the thing you are trying to do,

http://us-support2.external.hp.com/cki/bin/doc.pl/sid=6570907a0bdde9a04e/screen=ckiDisplayDocument?docId=200000058937871

Hope this helps.

Regds

[Sanjay_6]

Hi Mark,

Take a look at this thread too,

http://us-support2.external.hp.com/cki/bin/doc.pl/sid=6570907a0bdde9a04e/screen=ckiDisplayDocument?docId=200000058849862

Hope this helps.

regds

[Mark Henry_1]

Luc,

Yegads! It only shows 5 Cur PV (instead of 10), and 3 Act PV !

Scary.. I'll check my hardware..

M

[Luc Bussieres]

Mark,

For the 5 current PV I suspect that the information is correct if you are using alternate path. You could confirm that they are all correctly define in your VG by looking at the output of:
strings /etc/lvmtab
Just under the name of the VG you will see the list of all your PV that are configured.

But for the 3 active PV that means that currently 2 of your disks are either bad or have lost their LVM headers.


Luc

[Mark Henry_1]

Yes,

You're right 5 it should be...

So, as far as fsck'ing goes, I can't get the package to start on the original server now - says the lvol is corrupt.

I did a:

/sbin/fs/vxfs/fsck -F vxfs -o full /dev/vg_vob/rlvol

and it still show corrupt - how to fix it from here???

Thanks,

Mark

[Luc Bussieres]

Mark,

If you are missing 2 disks in your VG probably that some of your lvols are gone if the information was on these 2 physical volume.

You should first run a diskinfo one all the disk part of that volume group to find if you have a definitive Hardware problem (if the output of diskinfo does not correctly describe the disk or if the size is zero).

If all your disks answer correctly to diskinfo, the next step is to verify if your disks are readable with the following dd command:
dd if=/dev/rdsk/c?t?d? of=/dev/null bs=1024k
run it on each disks that are not currently active in your VG since this is a very long task and I don' t think you want to run this 5 times. (also don't run it on the alternate paths, just on the primary since it's the same physical disk).

If there is a problem with the disk you should be able to see some I/O error messages in the /var/adm/syslog/syslog.log file.

If all these test works without telling you that you have an hardware error then it's probably (90%) at this time a software error that could be repaired with a vgcfgrestore command. But before doing so try to find the cause of the corruption to the lvm headers. If you don't know how to do this, it may be better for you to place a software call at the response center to have somebody help you looking at the current header of the disk and helping trying to find the root cause of this corruption to make sure it won' t happen again. (You could have use these disks in another VG so you will corrupt the second VG by restoring the header with vgcfgrestore)

Luc

[Mark Henry_1]

Folks,

I appreciate all the assistance..

I'm at a loss to explain why the 2nd machine now has imported the map file (created from the 1st attempt) properly..
The vg contains the correct # of PV's, the filesystems mounted ok, and I can see my data.

I'm somewhat suspicious of that machine and it's flaky behaviour, however problem solved for the moment.

Thanks again,

Happy (US) prez day,

Mark