cmresmond and disk devices

[Jarle Bjorgeengen]

Hi fellows,

We implemented a SGLX cluster with RHEL4 on 4xDL585 seeing lun's from 2 EVA5000 .

You all know what happens when adding a new LUN from the EVA with lowest numbered port/controller WWN , all LUN's on the second EVA have their device files shifted up by one when machines are rebooted. (/dev/sdd becomes /dev/sde and so on.)

Now LVM2 handles this very well because it will scan all disks for vg_id and activate the volumes based on that, and not the devicenames of the physical volums.

Serviceguard's cmresmond however is only capable of using the /dev/sd* devices in the configfile. We created custom udev rules to make device files that were persistant. (using target-lun naming) and use them in the cmresmond config but we found out cmresmond did not work with the custom made udev-devices because. It also insisted on looking at /proc/partitions , and if the given devicefile-name is not found in /proc/partitions (and the udev-devices is not seend there) the device is always reported as down by cmresmond.

We ended up having to preallocate loads of LUN's on the EVA's so we didn't have to take down the whole cluster every time we need to add disk-space to existing packages, or create new packages.

Also I logged an enhancement request on this to HP-serviceguard team , but I never heard if something was done .

In this case the cmresmond is a showstopper to be able to do proper online administration of the cluster that one should expect from serviceguard.

Does anyone from the SGLX team know if this is beeing considered fixing ?

[Steven E. Protter]

Shalom Jarle,

The place to deal with this issue is not here but in person with your HP rep.

This forum can be useful in this matter only if it inspires people to contact their HP reps and demand the feature.

I have found that when enough customers demand a feature, they get it.

SEP

[Jarle Bjorgeengen]

Hi SEP,

I just noticed that the US serviguard team hangs around on the forums here, and wanted to shortcut the path :-)

Like I said there was an ER logged to HP about it, and yes I'll follow up towards them , but I thought it might be intersting for other people to know about such limitations, to care for planning.

- Jarle B.

[Mike Chisholm]

Jarle,

Hi. The enhancement request is still open, however I am not with the lab so I cannot comment on it's status other than that. Have you looked at the whitepaper "Using udev to Simplify HP Serviceguard for Linux Configuration, October 2006" available at http://docs.hp.com/en/8859/udev_sglinuxconfig.pdf

I admit I have not tried this myself so I do not know if it is directly applicable to your situation or not. If you need something more formal on your request you do need to follow up with a call to support or your sales rep most likely.

[Jarle Bjorgeengen]

Hi Mike,

this is exactly what we tried, and at that time (august'ish 2006 i think) we were not able to get cmresmond to accept udev devices. We think the reason is that cmresmond only accepts devices listed in /proc/partitions (found out by looking at strace output of cmresmond), but udev-generated devices is not listed in /proc/partitions .

[Serviceguard for Linux]

First, the comment I usually give when someone is monitoring a dual path LUN.

It is really not necessary. Specifically because you need two failures to run into that case.

That said, I know some folks want to do this anyways. I'm checking on the state of the changes that have been done for cmresmond for the next release and whether it covers this problem.

Reading carefully, did you (or can you) try creating a partition on the udev device you are trying to monitor to see if it shows in /proc/partitions? (Note: I'm saying this without checking anything. I'm just hoping it is easy for you to try.) I'm not sure if you crreate a partition on a udev device whether it winds up there.

[Jarle Bjorgeengen]

Hi "ServiceGuard for Linux" :-),

good point with the multipath argument, although it is still a good thing to check device availability.

udev uses hotplug and sysfs, and it lives entirely in user space. /proc/partitions is a legacy thing from the 2.4 kernel, and I think it is more expected of applications running on 2.6 kernels to use the sysfs way to find out what is going on in the kernel.

That said, it is a little unprecise to say that the disks are not in /proc/partitions. The same disk as the udev-device refers to is actually in /proc/partions, it is just that it is called something else than what is reflected by the udev-device-filename, namely the sd* notation. Since udev lives in userspace there is no way that udev can tell the kernel how the world looks like :-)