Raw devices for Oracle 11g on SLES 11

[Jeff_Traigle]

I've been trying to figure out how to get SLES 11 talking to our EMC storage arrays using multipath the past couple of days so we can start testing Oracle on that platform. I finally have a documented setup that seems to work. Loads of fun trying to piece it together from two EMC documents and one Novell document... yeah, right.

Now I'm at the point where I need to create raw devices for the DBAs to create the cluster and databases. In HP-UX, we simply created new device files with the same major and minor numbers in /dev/rdisk with some meaningful names like ora_data_disk01_array1 or ora_vote_disk01_array2, changing the ownership and permissions to oracle:dba and 660, respectively. It doesn't look like things are quite that simple in Linux. This morning, I did find Novell's method of creating raw device files and keeping them persistent on reboot (/etc/raw config file used by the raw service script). Unfortunately, Linux only seems to allow you to create names raw1, raw2, raw3, etc. Are there any non-cludgy ways to use meaningful names and to change the permissions? I found a document online stating to add chown and chmod commands in /etc/rc.local for the ownership aspect and creating files with meaningful names could be incorporated into that (although I don't know that SLES uses rc.local specifically). Just wondering if anyone else on here has faced this and come up with a cleaner solution.

[Steven E. Protter]

Shalom,

Might want to go to with the default names and:

use softlinks to point to these raw devices with meaningful names.

SEP

[Matti_Kurkela · Accepted solution]

Actually, regular Linux disk devices are capable of acting like raw devices when opened with the O_DIRECT flag. Oracle 10g Release 2 and newer do this automatically when configured to use non-file-based storage.

http://www.puschitz.com/TuningLinuxForOracle.shtml#UsingRawDevicesForOracleDatabases

In some cases, using the /dev/raw devices may still have some administrative benefit: since their creation is controlled by the sysadmin, it may be easier to assign the necessary ownership & permissions to them in a persistent fashion than on the corresponding regular disk devices. But looks like this is not the case with SLES 11.

I'm not very familiar with SLES 11, but RHEL 5.3 and later (and obviously RHEL 6 too) allow the configuration of customized device ownership and permissions in /etc/multipath.conf in a per-LUN basis. Surely this feature has been included in SLES 11 too?

Since you mentioned you're using multipath, if SLES 11's multipath.conf allows you to specify ownership and permissions, you could create a configuration block for each of your database disks in /etc/multipath.conf, specifying the LUN by its WWID (which the multipath subsystem uses to identify it anyway), and assigning it both a human-readable name and the necessary ownership & permissions in a single location.

The appropriate multipath.conf section would look like this:

multipaths {
multipath {
wwid
alias my-database-index-1
mode 0660
uid 800 # or whatever is the UID of the Oracle user
gid 800 # or whatever is the GID of the Oracle user
}
multipath {
wwid
alias my-database-data-1
mode 0660
uid 800
gid 800
}
# ... etc.
}

This configuration should result in devices
/dev/mapper/my-database-index-1 and /dev/mapper/my-database-data-1, which should automatically have the proper ownership and permissions.

After modifying /etc/multipath.conf, you may have to use "multipath -F; multipath -v2" to destroy and re-create the multipath device mappings. A reboot should also do the job, if you wish.

MK

[Jeff_Traigle]

Thanks, Matti.

I hadn't noticed the mode, uid, and gid entries in the sample multipath.conf file under the defaults section. Those parameters work in the multipath sections for each LUN as you described. It's great we don't need to mess with the raw service also for RAC to work.