Connecting Storage using Oracle on VMware

[briffle52]

We are looking at running Oracle on VMWare (5.1)

I have read the best practices for Nimble and VMware, and the best practices for Nimble and Oracle.. not sure how to best combine them.

Right now, our biggest question is storage:

Do we have the guests connect directly via ISCSI to the nimble?

Do we have the guests connect via vmware RDM?

do we have the guests use the VMWare VDMK's..

We have tested with going direct ISCSI, but currently only have a single machine running the "free" esxi for testing.. I have new hardware coming in soon, so I am going to be testing more.. but some of the things we are not yet able to test include the VMWare plugin for nimble (it seems to need vSphere) and migration of live vm's.

We're not sure what direct ISCSI might do to with migrations, and were not sure how the vmware plugin and RDM will work together.

Any tips or pitfalls to look out for?  We are hoping to automate the process of mounting 'clone' databases also in vmware, taken from snapshots of the server.

[jleonardini58]

All of the ways you ask about will work - but it isn't clear if this is Oracle on Linux, or on Windows.

There is no one size fits all answer.  If you are planning on using SRM, then you want to avoid the guess initiator route. With that one exception, I personally prefer guest initiator from guests, especially if the guests are Windows clients, because the MPIO suite from Microsoft is pretty damn sweet.  The benefit there has to do with additional transportability and recovery options that the guest initiator options open up.

RDM has some of those advantages, along with the *sort of* advantage that you can "See" the volumes from vCenter.  While the newest implimentation of RDM has had a lot of the "Suck" removed compared to precious examples, it has a bad enough reputation and persnickety hooks into VMFS that I personally wouldn't go that way.  Since you are doing Oracle, and I am going to assume on Linux, this is less of an issue, but RDMs used with Microsoft guests can sometimes have trouble being restored from snapshot.  I know quite a few people over at VMWare - and I don't know anyone who recommends RDM.

VMDKs - probably the easiest.  There are some minor performance implications as your Oracle instance is likely using a different block size from the VMWare native 4k, but that is a minor point honestly.  That would be easiest to implement, would be compatible with SRM, could be completely run by the vCenter plugin.  You give up some of the recoverability options that guest initiator provides - mostly the ability to clone the data volumes and logs from a snapshot then mount them to a recovery mount point , thereby enabling "restore by query".   That can be a powerful tool in some environments, completely irrelevant in others - I will let you be the judge.

For your other questions:

guest iSCSI does not impact migrations (assuming you mean vMotion here) as long as all hosts have the same networking setup - the guest initiator moves with the guest and keeps connection with the target.

You DO need vVenter server to use the plug in - I would not try to run a production instance of ESX without a vCenter server, particularly with your goal of automating clone mounts.

By the way, given the goal stated, I would lean strongly towards guest initiator attached volumes.

[briffle52]

Thanks for the info.  We are running Linux (centos 6.5 right now)

We will also have vcenter, and multiple hosts, were just waiting for hardware to arrive.

We have not been planning on SRM at this point in time, but I will definately keep those in mind!

[tdau98]

Brian,

Before configuring Oracle on VMware, you need to take into considerations for Oracle RAC, single instance, Oracle ASM, EXT file system, backup, recovery, Nimble snapshots, DR, performance, and migration (which you mentioned).  With that said, if you're looking for performance, then I suggest you use in guest attached and not RDM or VMDKs.  I am not a VMware expert but if I am not mistaken, vMotion still works with in guest attached volumes.  In guest attached volumes provides flexibility for your Oracle databases such as:

- Nimble snapshot (per database instead of the whole VM datastore)

- Oracle ASM

- Nimble replication (per database instead of the whole VM datastore)

- Performance (no VMware layer in the middle)

You mentioned about automating of mounting clones.  I am assuming that you only want to clone the Oracle database on a regular basis and not the operating system every time, correct?  If this is true, then you only need to snapshot the Oracle volumes and mount them on another VM for test or dev.  You can do so with in guest attached volumes.

I am not sure if this answers your questions but if you'd like to have a discussion, I can talk with you along with one of our VMware experts.

thanks,

Tom D.

[#5]

Hi Tom,

Wanted to revisit this thread for a moment if we could.

We are setting up a new environment using an physical Oracle database on Cisco UCS with Nimble storage (Linux OS) and looking to create a VM DB server to leverage for offloading RMAN backups and possibly as a test environment for mounting point in time clones of our DB's. 

The VMWare group is pushing for using RDM to mount the volumes but I would prefer to just use iSCSI so we can leverage the existing scripting and methods we use with the "hard" db server and eliminate any additional network overhead in the storage traffic.

What is the "current" best practice for this plus advanatages/disadvantages to using RDM vs direct iSCSI attached volumes in a dynamic setup like this?

Thanks,

Steve

[eforgette119 · Accepted solution]

Hi Steve,

If you're planning on leveraging automation (you mentioned existing scripts), iSCSI is going to be the path of least resistance.  Using RDMs adds another layer that your scripts will have to deal with (add/remove devices from ESX first).

There isn't a best practice because the answer depends on the environment.  The performance will be the same as long as the VM has the same network resources as the ESX server (this isn't always the case).

The pros for guest initiated iSCSI are really;

1.) Much simpler scripting

2.) No need for vCenter credentials (needed to attach/detach RMDs from ESX and the VM)

3.) Exact same procedures on physical and virtual servers

The pros for RMD:

I can't really think of any.  Your VMware group may be pushing for RDMs so that all the storage traffic uses a dedicated network (perhaps there is more bandwidth there).  If this is the case, you could ask to get the VM interfaces on this network perhaps.

I hope this helps,

-Eric