Operating System - Linux
1748106 Members
4959 Online
108758 Solutions
New Discussion юеВ

Linux Raw Devices under RHEL 5.4 and Multipathing

 
SOLVED
Go to solution
Alzhy
Honored Contributor

Linux Raw Devices under RHEL 5.4 and Multipathing

RHEL 5.4 suppsedly restored support for raw devices (probably for Oracle RAW/ASM and other database storage needs).

However /dev/dm-NN devices seem to be not fixed and can change accross reboots -- how does one map them to raw devices.

With ASMLib - this is not a problem and /dev/dm-NNN can change all it wants and OacleASM will always find the disks it had tagged. But with RAW -- how does one map the multipath devices?

My experience with dm devices is when a disk is provisioned and a mutlipath device is allocated - on reboot the orginal /dev/dm-NNN changes.

TIA.
Hakuna Matata.
7 REPLIES 7
Matti_Kurkela
Honored Contributor
Solution

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

Basically, map the /dev/raw/raw* devices to appropriate /dev/mapper/mpath* devices.

The /dev/mapper/mpath* names are bound to disk WWIDs reported by the storage system, using the information in /var/lib/multipath/bindings file.

Therefore, once a /dev/mapper/mpath* name has been associated with a particular multipathed disk, the name stays with that disk until the multipath binding is explicitly cleared (with "multipath -f " or "multipath -F).

The /dev/dm-NN names are essentially for device-mapper subsystem's internal use only and should not be used by the applications.

RedHat's recommendations:

http://kbase.redhat.com/faq/docs/DOC-10164

Please see method 5, "Creating persistent raw devices for multipathed LUNs or LVM device" near the end of the document linked above.

MK
MK
Kapil Jha
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

try moving /var/lib/multipath/bindings file to /etc/multipath/binding
and make a soft link to /var/lib/multipath/bindings

this would help
and after system comes up
#multipath -v2 -d
if this does not return anything you are good.

BR,
Kapil+
I am in this small bowl, I wane see the real world......
Alzhy
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

Matti,

Selamat Tembang (Malay for Thanks!)

The KBDOC it seems has been updated too for 5.4.

Muchas Gracias!
Hakuna Matata.
Alzhy
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

Matti,

"Unfortunately it is not possible to write udev rules for creating raw devices on multipath devices (/dev/dm-*) without manipulating existing udev rules. Modifying existing rules for this purpose could cause unforeseen problems and is not supported by Red Hat Global Support Services. If absolutely necessary, an alternate method for creating raw devices on top of multipath devices could be to create the raw devices in /etc/rc.d/rc.local, so long as the raw device is not required before rc.local is executed."

So what is "SUPORTED"? Surely Linux DB servers that hook up to multi-pathed SAN arrays are prevalent these days. Does that mean that RAW devices under Oracle, Sybase, etc. are "pase" already and ASMLib is the norm for Linux environments?

/dev/dm-NN devices work beatufifully under ASMlib and even LVM -- both are agnostic to device name changes. But I've a DBA who insists on going with RAW devices under ASM. And personally - I prefer Oracle to simply use ASM-less RAW storage even.


Hakuna Matata.
Alzhy
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

Also, should I have /dev/mpath/mpathNNN for every /dev/dm-NNN?

All I see under /dev/mpath are 34 character entries (hex) linked to /dev/dm-NNN devices like:


[root@redhat mpath]# ls -l 3600508b4000ce3920003c00005c80000
lrwxrwxrwx 1 root root 8 Oct 6 09:37 3600508b4000ce3920003c00005c80000 -> ../dm-61

I have the Storageworks multipath.conf from the HP package.


Hakuna Matata.
Matti_Kurkela
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

RedHat seems to say "Do not mess with the udev rules: modifying the existing rules is not supported by RH." They seem to be happy with adding the necessary commands to /etc/rc.d/rc.local.

But why ask me? As you're using RedHat 5.4, you should have a valid RHN account and could send your question to RedHat support instead. You would get your answer "straight from the horse's mouth", so to speak.

> /dev/dm-NN devices work beautifully under ASMlib and even LVM -- both are agnostic to device name changes. But I've a DBA who insists on going with RAW devices under ASM.

Yes, the device nodes /dev/dm-NN have just the same major & minor numbers as the corresponding devices with persistent names in /dev/mapper. The kernel does not care which device node you use: feel free to use the /dev/dm-NN devices with ASM if you like.

My point was that the /dev/dm-NN devices definitely seem to be not fixed - because they are not *designed* to be fixed at all.

If you use ASM, you can just set up your system to create RAW devices for each disk and let ASM sort them out if you wish.

Your 34-character hex strings under /dev/mpath/* are a direct representation of the WWIDs of the disks - apparently the Storageworks multipath.conf sets FriendlyNames=Off. In RedHat's default configuration it is On, which causes the /var/lib/multipath/bindings file to be used and the names to appear in the shorter /dev/mapper/mpath* form.

The long hex string is somewhat inconvenient, but it uniquely identifies each multipathed disk with no need for a mapping file. The WWIDs are allocated to disks and storage systems like MAC addresses to network cards: you can be pretty sure you cannot find two disks with matching WWIDs unless the disk manufacturer has made a mistake with their WWID allocation.

MK
MK
Alzhy
Honored Contributor

Re: Linux Raw Devices under RHEL 5.4 and Multipathing

MK,

So are you saying I am free to use /dev/mapper/XXXXXXXXXXXXXXXXXXXXX in my ASMlib configs? Also to map it to raw too?

The multipth.conf blocks for the EVA and XP arrays I have hoked up to this Redhat machine does not explicitly declare FriendlyNames in each block. The Default section declares it as Yes though.

For EVA mapper entries - there is an extra "3" before the EVA VDISK WWID. For XP -- I still ave to deciper where in the 34 character string CU:LDEV is depicted.

We have both StorageWorks and Redhat Support but I wanted to consult this esteemed forum in the hope this post aso helps sould out there who are into the same adventure with BIG Linux as we are.

Thanks very much good sir.
Hakuna Matata.