fsck failure on just 1 node of SG cluster

[Mark Henry_1]

Hi All,

I have a few 11.0 machines in a serviceguard cluster, which share certain volume groups based on the package running.
Well, for some reason one of the nodes flat out refuses to mount the lvols in one of the VG's when the package is activated on that node. It seems to be a fsck failure.
These lvols checkout fine (and work) on the other nodes.
Is there perhaps some bit somewhere on the failing node that needs to be cleared which is failing the lvols without even checking them properly?
The output is..
Jun 15 17:11:39 - Node "bananas": Checking filesystems:
/dev/vg_data1/lvol1
/dev/vg_data1/lvol3
/dev/vg_backup/lvol1
ERROR: Function check_and_mount
ERROR: Failed to fsck one of the logical volumes.
Jun 15 17:11:39 - Node "gollum": Mounting /dev/vg_data1/lvol1 at /data1store
Jun 15 17:11:40 gollum vmunix: msgcnt 1 vxfs: mesg 021: vx_fs_init - /dev/vg_data1/lvol1 file system validation failure
vxfs mount: /dev/vg_data1/lvol1 is corrupted. needs checking
Jun 15 17:11:40 gollum vmunix: msgcnt 1 vxfs: mesg 021: vx_fs_init - /dev/vg_data1/lvol1 file system validation failure
ERROR: Function check_and_mount
ERROR: Failed to mount /dev/vg_data1/lvol1 to /data1store

Thx,

Mark

[Mark Henry_1]

note, the node is the same - i changed the hostname from gollum to bananas when posting just for privacy reasons but forgot to change the rest.. :)

[Tim D Fulford]

Mark

I have not come across anything that needs to be cleared for fs failures. But

when you do "cmviewcl -v" is the package switching enabled? If not "cmmodpkg -e -n gollum ".

You could also try activating the package manually outside of ServiceGuard.
# cmhaltpkg
# vgchange -a e vg_data1
# mount /dev/vg_data1/lvol1 /datastore
# ...
See if this throws any light on the problem.

The only other thing I would check is that if /etc/lvmrc file is correct (I think it is VG_AUTOACTIVATE=0, I'm not at work so can't check)

Good luck

Tim

[Steve Lewis]

Mark,

Check the following:

That on this node it tries to mount the correct type of filesystem (e.g. vxfs) in your package startup script.

That the package startup script on this node is up-to-date, mounting the correct lvol.

That this node does not have this filesystem in /etc/fstab and try to mount it at boot-up.

That this logical volume does exist on this node (vgdisplay -v or lvdisplay). If it was a late addition you may need to re-vgimport the volume group.

Steve