LVM information dissappearing from 11i v1.6 host

[Keith_115]

LVM information is lost after reboot. Two hosts are directly connected to a DS2300 SureStore disk array. Using our own software we arbitrate which node has access to the drives. All VGs, PVs, LVs, and disks are seen equally from both nodes. When one node is rebooted, the LVM info is lost. The other is fine. vgdisplay -v returns the same info on both nodes. Any ideas why one copy of LVM info keeps getting lost? Please and thanks.

[Cesare Salvioni]

What do u mean LVM info is lost?
When you say that vgdisplay -v returns same information you mean on both systems at the same time?

I'm asking you this because sharing a VG between two hosts should be done really carefully. For instance the creation of the VG must be done following fixed steps
Most important: YOU CANT see the same time th VG from both hosts, it means you haven to change the booting scripts in order to avoid activating of VG on both systems.
On one host do vgchange -a y, on the other do nothing or vgchange -a s (the value for sharing , i'm not sure it is s, see man vgchange)

Anyway you will see all VG info only on a host, to chaneg the host you must do vgchange -a n on the first one and vgchange -a y on the other

Alse creating the VG you must be sure that the ID are taken from the tables on disks so create the VG on one host, vgexport -p -s -m /tmp/lvmap /dev/vgXX, copy the file /tmp/lvmap on host 2 and run vgimport -s -m /tmp/lvmap /dev/vgXX

Hope this helps

[Joe Short]

Best guess with the little info you provided is that both servers are activating the VGs in default mode (vgchange -a y /dev/vgXX) You could change it to activate in shared mode (vgchange -a s /dev/vgXXX)
But, you do risk what is commonly called dual brain syndrome. But you will need to use raw volumes not file systems or the data stored within will be corrupted.
If that doesn't work, then I'd suspect the software you are using for arbitration.

[Keith_115]

Can anyone tell me what is happening at the file or permission level when I make the volume group available as part of a HA cluster using "vgchange -c -s". Is write access being passed back and forth between the nodes? If so what triggers the transfer?Please remember that we're not running serviceguard or any HP software for cluster control. The arbitration between hosts is done by proprietary software. I don't think this is a big deal because the man pages do not explicitly state that the HP software is required.

[Cesare Salvioni]

For what i know there is no way for the two system to pass write permission back and forth: i mean ONLY 1 host may have write permission, the other MUST AVOID Opening it with write permission.
Even SG does not open both in writing mode:
the primary node for the package (sofware configured to swicth between 2 or more nodes through SG) does a vgchange -a y, the other nodes do nothing on that vg. When SG feels that the primary node is done, start the package on a secondary node doing, execing first of all a vgchange -a y.
I made some tries on scratch vg and what i found is that the two hosts doesn't understand the other is using the vg, what is really happening is that both of them change the file system tables destroying the file system itself.
If you want to have both of them using the vg in writing mode it's really a though work because you have to sync the writings operation on both nodes being sure that before one starts writing and modifying the tables, the other has synced all modifications made to superblock, inode table or whatever: i would avoid that !!!

hope this helps

[Dietmar Konermann]

Keith,

marking the VG cluster-aware (-c y) and activating it exclusively (-a e) is only possible if Serviceguard cluster services are running on the system. Part of these services are cmcld and cmlvmd which ensure that the VG is only active on one node of the cluster then.

As others already mentioned, you *MUST* ensure that the VG is *NOT* activated r/w on more than one system.

However, you are talking about 11.22... I remember a similar problem where some EFI (backup) structures were left at the end of one of those affected disks. During bootup EFI found the backup copy and restored the EFI header at the begining of the disk... and consequently wiped out the LVM header. :-(

Could it be possible that the disks in question were boot disks for some OS in the past? In this case you could try to completely erase them before re-creating the VG.

Just a guess...
Dietmar.