Understanding Linux/Linux LVM

[brian_31]

I was rebooting a RHEL5 server and it put me in maintenance mode. It complained about a /dev/vg_name/Volname device not available. when i did mount -o remount, rw / and edited the fstab to comment that vg line it rebooted OK. I found from fdisk -l that it was a linux LVM partition on the /dev/sda (i think /dev/sda7). I also noticed that i had earlier changed the lvm.vonf fileter options for some multipath configurations. when i reverted to the old lvm.conf and rebooted again i did not have problems. My questions as follows.

1. what is the idea behild the mount -o remount /?
2. what if there is no filtering (defualt setting), I mean what do we lose other than some warning messages for duplicate paths (similar to HP-UX alternate link messages)?
3. since my linux LVM partition on /dev/sda7 was mounted on /opt/app why it would not boot the system fully (and e2fsck -b 8193 also fails since there is no device under /dev/

Please help me understand this better. I am fairly new to Linux and have previous HP-UX experience.

Thanks

Brian.

[Alzhy · Accepted solution]

1. what is the idea behild the mount -o remount /?

Allows you to mount r/w your ever important / partition so you can make edits to your system files.

2. what if there is no filtering (defualt setting), I mean what do we lose other than some warning messages for duplicate paths (similar to HP-UX alternate link messages)?

By default, /etc/lvm/lvm.conf's "filter=" line WILL scan all disks and partitions on your system for LVM objects. For a system with large SAN connections or LVM on SAN devices -- it will mean longer boot times and those annoying Duplicate PVs messages.


3. since my linux LVM partition on /dev/sda7 was mounted on /opt/app why it would not boot the system fully (and e2fsck -b 8193 also fails since there is no device under /dev/

Because whatever changes you made in your lvm.conf - likely masked your LVM bits on the partition from beingrecognised/imported.


Hope this helps and clarificatory anough.

[Alzhy]

Oh btw, my standard lvm.conf filter line for all Proliants and BigBlue machines are:

filter = [ "a/mapper/.*p1$/", "a|/dev/cciss/*|", "a|/dev/sda[0-9]|", "a|/dev/sdb[0-9]|", "r/.*/" ]


Caveats:

Our SAN multipathed disks are always paritioned wholly and used under LVM so LVM should only scan the "p1's"

If you use whole disks - mapper or not in LVM sans paritioning -- adjust your filter accordingly.


Hope this helps.

[Matti_Kurkela]

1.) Also in HP-UX, if you boot the system to single user mode ("hpux -is"), the root filesystem will be read-only. This would allow you to, for example, run fsck on the root filesystem: since the read-only mode guarantees the kernel won't make changes to the filesystem, the fsck can make changes without interference. If you run fsck on the root filesystem in this way and the fsck process actually makes any repairs, you should reboot the system immediately afterwards, without trying to transition to multi-user mode first.

Unlike Linux, HP-UX has a convenient "mountall" command which will first execute "mount -o remount,rw /", and then attempt to mount all the remaining filesystems listed in /etc/fstab.

2.) Without filtering, if you have multipathed disks, you won't have absolute control on which devices the LVM system uses. Instead of using the multipath device, LVM might use the regular /dev/sd* device - which means it will be using a single path instead of the sum of all the available paths. If that single path would fail, you would experience filesystem errors.

If you don't have multipathed disks, the default (accept all disk devices) is usually good.

3.) Since the boot was stopped in single-user mode, your volume group was not activated yet. "vgchange -a y" would have fixed that.

In Linux, the LVM /dev/ directory does not exist if the VG is not activated. This is because all the devices in /dev/ directory are automatically managed by the udev subsystem. The VG device directory will automatically appear once you activate the VG.

MK

[brian_31]

Thanks MK. Execellent explanation...

Further Question..

In Linux, the LVM /dev/ directory does not exist if the VG is not activated. This is because all the devices in /dev/ directory are automatically managed by the udev subsystem. The VG device directory will automatically appear once you activate the VG.

But how will i know what VG's are there on the system. The command vgs does not list all the vg's (only activated VG's). so even after the server comes up if that particular LVM is not activated (say VG is not activated) even pvs command is not showing that particular volume (in my case /devsda7 which was LVM partition mounted on /opt/app)..so how would i know what VG's and PV's i am missing??

Thanks

Brian

[Matti_Kurkela]

In HP-UX, "vgscan" is a command that should be used only after careful consideration. Not so in Linux.

In Linux, vgscan is part of the standard boot procedure. It reads the lvm.conf files, then scans the disk devices according to the filter rules, and updates the LVM device cache: /etc/lvm/cache/.cache.

If you change the LVM filter rules, you should at least re-run vgscan afterwards. If you want to be really sure, you can delete the device cache file before running vgscan: the vgscan command will automatically re-create the cache file if the root filesystem is writeable.

The "pvscan" command could be used to detect all LVM PVs on the system, i.e. all disks that have been prepared for LVM use with pvcreate, no matter whether they're currently part of a VG or not. The pvscan command even identifies the VG to which the PV belongs to, if applicable.

The "vgs" command should certainly show all VGs detected by the latest vgscan, whether activated or not. I can only think of two reasons that would cause "vgs" to miss a VG: when the latest vgscan was run, all the PVs that contain the VG either were not available, or vgscan was told to ignore them using the LVM filter rules.

Rule of thumb: on Linux 2.6.* LVM, pvscan/vgscan/lvscan commands will scan the actual disks; pvs/vgs/lvs commands will use the cached information provided by previous scan commands. Vgscan scans everything; the other scan commands might be useful if you want a more limited scan.

By the way: on Linux, un-learn everything that you might have learned about exporting and importing volume groups on HP-UX.

Linux LVM *does* have commands called "vgexport" and "vgimport", but they won't work at all the same as in HP-UX. (There is no /etc/lvmtab; there are no map files, and you won't need them.)

MK

[Alzhy]

I often tell my admins:

"Linux LVM is similar to HP-UX LVM but not the same" ;^))

I tell them the commands are similar but not the same and simply "observe" and try and do and explore:

pvs
pvscan
vgs
vgscan
lvs
lvscan
vgmerge
vgremove
vgrename
vgerduce
pvextend
pvremove
pvcreate
pvmove
lvcreate
lvextend
lvrename
lvreduce

... notice the "genericness" of the commands?

Now if Only ZFS can get into mainstream Linux -- we all simply need to be familiar with 2 commands for volume and filesystem manageent: zpool ans zfs. But that's a different thread - ;))

[Josh D Trutwin]

"Unlike Linux, HP-UX has a convenient "mountall" command which will first execute "mount -o remount,rw /", and then attempt to mount all the remaining filesystems listed in /etc/fstab."

FYI you can run "mount -a" to mount all unmounted file systems, though it won't automatically remount / in rw mode.

Josh

[brian_31]

Thanks everyone..closing now..

Regards

Brian.