Multiple LUNs vs single LUN in HPUX 11iv3

[Liviu I.]

Hello,

Considering an EVA 8100 (or any EVA with active-active capability) and an HP-UX 11iv3 server what would be the best design principle in order to maximize storage access performance (IOPS and BW), to use 1 vdisk of 500GB (LVM: 1 PV in 1 VG with with single or multiple LVs) or 5 vdisks of 100GB (LVM: 5 PVs in 1 VG with single or multiple LVs) ?

I think that since the server has 2 dual-port HBAs (dual-fabric SAN), the OS supports native multipath, the storage supports active-active, the best idea is to be able to use the parallelism at any level given multiple IO operations, thus I would adopt the latter ...

I would appreciate any opinion/suggestion eventually doc.

Thank you
L.

[Stephan._1]

Hi,
I would use multiple paths and disks to get the benefits of lvm striping and using both controllers/caches.

Remember that the EVA use multiples of 8 for expansion and the good old binary system.

For example 4 Paths 8 Disks 64GB sounds nice.

Some Hands on for the EVA:
http://h20195.www2.hp.com/v2/GetPDF.aspx/4AA0-2787ENW.pdf

[Zinky]

Liviu,

If said disk is to be used under ASM in a Oracle - I will sugegst for say 5x100GB Vdisks and path prefrenced alternating between A and B controllers. That way you fully harness the bandwidth to your EVA and the dual controllers.

But you can also do a comparative performance since it is quite EASY to colapse and ercarve VDISKS (LUNS) on an EVA.

I suggest you use ORACLEs ORION Tool which is very easy to use and simulates comparative IO and even striping that happens in ASM. The tool is at:

http://www.oracle.com/technology/software/tech/orion/index.html


Let us know your results.

[Liviu I.]

Thank you for your ideas.
Well, unfortunately Orion testing is not an option so far, since I am talking about a production environment (and the latest tests I ran, with IOmeter, turned the storage unresponsive). ASM is also not used now. But for sure I appreciate your idea.
After reading the HP whitepaper svo suggested, I draw a conclusion , and I would appreciate any comments on this : page 22 (controller balancing influences performance): I would split the disk-group in 5 vdisks (LUNs) in order to have flexibility in balancing the workload. But the following phrase can ignore all your plans :

"HP Continuous Access EVA requires that all LUNs within a DR group be owned by the same controller. Load balancing is performed at the DR group level and not the individual LUN. Additional configuration information can be found in the HP StorageWorks Continuous Access EVA implementation guide."

Thus the flexibility is reduced when using Continuous Access.
As an example : if one has 2 applications each needing 500 GB space; and it has a DR group for each , then it makes no sense to split each 500GB in 5 LUNS since all of them will be owned by the same controller.
Another note, stating that there is no performance improvement when splitting on the same controller:

Striping LUNs within a disk group on the same controller provides no additional performance value. The EVA automatically stripes each LUN across all disks in a disk group.

Since the discussion turned to somehow a storage design issue (well they are very much related , and I'd be glad to have here members with experience in both domains), I would like to ask what do you think about my question from the LVM point of view ? I refer here at your experience in balancing filesystem access for performance improvement. Logical Volumes stripped or not ?

Thank you.
L.

[Stephan._1]

Hi,
in my opinion you will only get a final answer by testing and comparing the results.

Following the whitepaper splitting and stripping doesn't make sense, so the only reason to use lvm will be the flexibility if you need to change it (increase, .. ).

In our enviroment we use EVA's as described above, even if we don't get, according the whitepaper, benefit of it.

If you look at http://communities.vmware.com/thread/73745 - these guys tested a lot of EVA's with iometer without any problems, so perhaps you give it a second try.

[Liviu I.]

Well LVM is a must nowadays , especially in a mission-critical environment 24x7. And since we use LVM an I/O sequence (from the OS perspective) when you have a single LUN (translated in a PV) I see it as a sequential sequence; when using multiple LUNs (multiple PVs in a VG) I see it as a parallel sequence(expecially if the LV is stripped).
Am I right , or is it just my imagination ?

Thank you.
L.

[Wim Rombauts]

My experience is limited to HP-UX 11i v2, I have no v3 running here yet.
I also use LVM, multipathing, mirroring, ... and noticed that HP-UX as an IO queue per LUN.

So, if you have 1 big lun of 500GB, all IO is done sequentially. If yo have 5 luns of 100GB, 5 IO's can be done in parallel. So, in theory, you can flush your IO to the EVA's NVRAM 5 times faster with 5 LUN's as with 1 LUN, provided that your IO-card is not a bottleneck and the EVA's disk can keep up with the IO throughput.
Reading is the same 5 reads can be done in parallel, but at the bottom line, the EVA has to get the data from the same set of disks, so if the EVA itself is heavily loaded, it probably won't make any difference.

[Liviu I.]

Thank you for your reply, thus you confirm what I thought. I can conclude somehow that it's better to use parallelism as much as possible: in the worst case, the bottleneck will be somewhere else on the SAN, EVA, etc. But from the point of view of a HP-UX admin design, it should be the best architecture one could achieve.
Liviu.

[Zinky · Accepted solution]

L - the following may provide ye with good reading.

http://www.oracle.com/technology/deploy/performance/pdf/EVA_ORACLE_paper.pdf

[Liviu I.]

Now this is a great paper, although a little old. It would be great to find one with a new version of HP-UX, Oracle and EVA. But this comes as a confirmation of what I almost knew (or guessed), and eventually it can be used in a real world scenario design.

Anyway I still have to understand the stripe size LVM mechanisms and the fine tuning according to different data types.

L.

[Zinky]

My HP-UX (11.11/1.31) on EVA8XXX / Oracle Recipe is we provide a 100 to 200 GB Vdisk sizes with a volume or a filesystem sitting on top of each Vdisk. SCSI QDepth I have set to 32. I leave it to the DBAs to stripe their datafiles accross these filesystems.

Under ASM, it is the same -- 100 or 200 GB VDisks. We expect ASM to just stripe/Balance accross these volumes.

Several years ago though -- I ahd a mini experiment with an adventurous DBA. I had him compare the performance of a DB that was on a single 1TB LUN versus 5 200GB LUNs. With the 1TB LUN experiment -- I set my Q depth to 128. There were hardly any difference in performance. Back End RAID setup was VRAID10.

[Liviu I.]

I did the same having 100 GB vdisks, but the drawback I find is the harder management: having to present many "small" vdisks instead of 1 "big"; having to scan/import many PV's. But this is maybe my fault, since I should have had scripts for this.

My conclusion to this thread would be that it is better to have parallel access at any layer in an I/O stream of transactions, even if it is not a performance improvement in some situations.It turns out to be not a notable performance penalty while staying on the safe side also.

Thank you all who replied.
L.

[Liviu I.]

thank you

[R Bray]

I realize this is an old thread, but I haven't seen this sort of information posted after searching for it. We have had favorable performance benefits with splitting the filesystem of an Oracle database onto multiple LUNs on the same physical disk group on an EMC SAN. The originial queue length of 2-3 per path (c14 and c16 are separate FC cards) led us to believe that Oracle and the filesystem were paralleling read requests sufficiently well, but it seemed that somewhere in block transfer world, I/Os were being serialized (the SAN never reported a queue depth of >1 on the LUN, and the backing disks were not saturated). This is HP-UX 11iv2, Oracle 10g, on LVM/VxFS with EMC PowerPath 5.1.0. Some sar -d output before and after the split:

device %busy avque r+w/s blks/s avwait avserv
c14t0d0 98.01 2.48 481 8234 2.14 4.43
c16t0d0 97.51 2.80 499 8554 2.23 4.25


device %busy avque r+w/s blks/s avwait avserv
c14t1d0 80.00 0.50 768 75824 0.00 1.61
c16t1d0 75.00 0.50 713 70936 0.00 1.54
c14t1d1 25.00 0.50 208 7216 0.00 2.03
c16t1d1 30.50 0.50 226 8960 0.00 2.15
c14t1d2 40.50 0.50 461 15776 0.00 1.77
c16t1d2 47.00 0.50 528 18032 0.00 1.87
c14t1d3 30.50 0.50 386 13288 0.01 1.81
c16t1d3 35.00 0.50 430 14360 0.00 1.89
c14t1d4 0.50 0.50 2 40 0.00 2.93
c16t1d4 1.50 0.50 3 48 0.00 6.45
c14t1d5 7.50 0.50 101 2557 0.00 1.66
c16t1d5 9.00 0.51 100 2115 0.00 1.78

Application performance is improved, and tools like sar and GlancePlus no longer indicate a disk bottleneck. This was achieved with no hardware changes.