vgchange causes device busy

[joe_91]

I have a volume group (/dev/vg03) which is in service guard and i had to mount it exclusively to trouble shoot an application issue..here is what i did

1. vgchange -a e /dev/vg03
2. mount the filesystem
3. finished the work
4. unmount the filesystem.
5. vgchange -a n /dev/vg03
(could'nt deactivate volume group vg03:
device busy).

i need to deactivate this vg to bring up the SG. anythoughts?

Thanks

Joe

[A. Clay Stephenson]

I'm guessing that the cluster daemon was running when you did the vgchange -a e and is not running now. You will need to restart the cluster and then you should be able to deactivate the VG.

[Steven E. Protter]

Shalom Joe,

cd out of the filesystem folder when doing the command.

check /var/adm/syslog/syslog.log

Do a vgdisplay -v on the volume group and see what logical volumes may be active, possible other filesystems and such and take corrective action.

Halt any packages that might be trying to use the filesystem.

Really the umount should fail, but if there is an open process out there (fuser -cu add a k to kill) to find out if there are processes that think they have filehandles open.

SEP

[Joelmel Roche]

Hi Joe,

Try search for all device files which have a major number of 64 and a minor. I am wondering if they have "duplicate" device files for lvols,Where the duplicates have bits set in the minor number reserved bits (format being 0xVVRRLL - VV = volgrp number, RR reserved and LL lvol number). Or find out how they are using all the device files for this LV? e.g. are they just mounting via the block and not touching the raw? Or do they perform some other operations with them?

[Stephen Doud]

Are all files closed on the f/s? May have to use 'lsof' to identify any files that are kept open by NFS or similar socket-based processes. If such is the case, drill the application that is holding the file open indefinitely (such as an aborted automountd).
Update fuser patches.
Update the latest NFS, STREAMS and ARPA Transport patches.

If the umount didn't work properly:
rm /etc/mnttab and do 'mount' to rebuild it.
Update the latest umount patches.
Update the latest vxfs patches.

If the vgchange really is the problem...
Verify the status of the disks using ioscan. Do the special files exist? State = NO_HW?


Do you also see:
"vgchange: Failed to notify clvm daemon about volume group deactivation - No
such file or directory"

Potential causes:
- The LVM structures on disk may be corrupted = use vgcfgrestore to reload.

- Failed to use 'vgchgid' on Business Copy set or SRDF backup set. (duplicate VGID) = use vgchgid

This is not an exhaustive list... just some ideas.

[Gary L. Paveza, Jr.]

There is a known issue with vgchange.

From the ITRC:

The root cause for this problem is an old, undocumented behavior in LVM in which deactivation can not succeed while a resync is in progress.


It's documented as a ServiceGuard issue, however, I experienced it with BCVs so it's not exclusive to SG. The fix is pretty easy. Just issue a vgsync /dev/vg<> before the vgchange.

I searched for vgchange + failure on ITRC.