VG activation with PVs missing

[Rudy Williams]

I understand the concept of a VG quorum just fine. However, there is something that I have never thought of until a discussion today in regards to LVM.

How exactly does a volume group activate that has missing contituent PVs, yet still has quorum? (Assume no lvols are mirrored.) If the VG is activated, and the filesystem mounted, there stands an excellent chance that data will not be available!

Yikes!

[James R. Ferguson]

Hi:

Yes, that's true. There is an option to handle this situation, although it is not advisable:

# vgchange -a y -q n /dev/vgXX

*However* as the man pages for 'vgchnage' note, "The -q n option can be used to activate the volume group when the disk quorum is not
maintained because too many disks were lost.
Since it ensures the integrity of the LVM
configuration information, it is normally not
advisable to override the quorum."

Regards!

...JRF...

[A. Clay Stephenson]

Yes, that is absolutely true. In fact, when an unmirrored LVOL spans more than one physical disk, you have actually increased the probability of failure and data loss because the failure of any physical disk that comprises the LVOL will result in data loss.
Of course, the more disks that you have, the higher the probabilty that a disk will fail.

Fot that reason, always mirror (or RAID) critical data and always have good backups so that you won't need them.

Regards, Clay

[Sridhar Bhaskarla]

Hi,

While you can override the quorum using -q n option, I would do it only under the circumstances of missing PVs that are either alternate links or mirror disks. I would be atmost careful if the situation does not fall under any of the above.

-Sri

[S.K. Chan]

To add ..
Whether or not a VG can be activated will depends on the quorum rules, we all know that. When it's time to mount the FS, whether it can be mounted or not will depend on the location of the superblock on the FS. If the superblock does not reside on the "failed" PV, the FS will mount until someone try to read/write to the bad blocks before the IO errors become apparent. Of course even before that happened you would know you have a problem when you see error like "missing PV", "cannot query PV" , etc, etc from any execution of LVM related command like vgdisplay.

[Rudy Williams]

Let's assume that I activate the VG and I still have quorum, though on PV is missing. I attempt to access a file that is on the dead PV.

What will the user see when their command fails? I have never actually tried this. (I am sure that the syslog.log will have all sorts of lovely entries.)

Oh, one more question. Will vgchange write out any errors to the console or the shell's stdout/stderr when it determines that not all of the PVs are available?

[Sridhar Bhaskarla]

Hi,

If a PV is missing, it is a rare chance that you can mount the corresponding file systems. It would ask you to fsck the filesystem and it will finally fail.

-Sri

[S.K. Chan · Accepted solution]

What will the user see when their command fails? I have never actually tried this. (I am sure that the syslog.log will have all sorts of lovely entries.)
=> It'll simply hang or freeze. Simple example is say you "cd" to the directory and then do "ls", it'll just hang and you'll not get the prompt back. Any yes you'll see tons of IO errors in syslog.log and "dmesg" output.

Will vgchange write out any errors to the console or the shell's stdout/stderr when it determines that not all of the PVs are available?
=> Yes it will, if I'm not mistaken it'll say something like ..
"couldn't attach to PV /dev/dsk/..."

[Patrick Wallek]

Here is what I have seen-- The VG was already active, but a whole tray (the old HP6000 trays) of disks failed (power supply problem). When a command was issued against the file system that was located on those disks the command would hang as it was waiting on I/O. You could not CTRL-C it, nor could you 'kill -9' it.

If the VG activates successfully and if the LV mounts without an error, I would suspect that you would see the same type of behavior. Lot's of hung processes waiting on I/O that you can't do anything with / to.

[albert_hua]

Yes,it is really a latently serious problem.
I have encoutered similar problems. And users can not detect the question until they access the data which is unavailable at all. I think it is better to use LVM commands such as vgdisplay or lvdisplay after you import one VG.
Then you will see error like "missing PV", "cannot query PV" , etc if there is any missing PV. Correct the errors before the users access it.

Al.