How to achive optimal disk performance

[Armin Schneider]

Hallo,

I have to tweak the last bit of performance out of the following system:

HP-UX 11.11 (RP4440)
Online JFS 3.5
Oracle Database (9.2.0.6) > 1,1 TB
Disk System: EMC Clariion CX700 (Clariion LUNS are already optimized)
Mirroring using Mirror-UX (operating system raid 1...)

situation as is:
several Volume groups
like the following one:

--- Volume groups ---
VG Name /dev/vgsapdata4
VG Write Access read/write
VG Status available, exclusive
Max LV 255
Cur LV 1
Open LV 1
Max PV 16
Cur PV 10
Act PV 10
Max PE per PV 19200
VGDA 20
PE Size (Mbytes) 32
Total PE 7670
Alloc PE 7500
Free PE 170
Total PVG 2
Total Spare PVs 0
Total Spare PVs in use 0

Volume groups using PVG's
logical volumes created with following command:

lvcreate -s g -D y -m 1 -r n -L 120000 -n lvol1 /dev/vgsapdata4

Each of this filesystems (one logical volume each vg) is striped over more than 75 physical disks

It is possible to get a downtime for two days to reorganize the whole layout ...

Now my questions:


- what do you recommend to achieve optimal performance ?
=>
- which is the optimal filesystem / logical volume size for performance ? (is smaller better (for example 50 GB) or is bigger (400-500 GB) better .. ?)
- which physical extent size is best for performance reasons ?
- I heard, that HP-UX has the limitation of 8 parallel I/O per LUN - correct ?


thanks

armin

[harry d brown jr]

- what do you recommend to achieve optimal performance ?
==> Move to the second generation of EMC's DMX, using the smaller GB drives that have an RPM rating of 15k. Then go to 4GB FC if possible.


- which is the optimal filesystem / logical volume size for performance ?
==> It depends. What is the IO activity. How is the data organized? How is consistency achieved? What kind of data? How are backups accomplished?

- which physical extent size is best for performance reasons ?
==> Depends on IO activity. Typically one doesn't change the 4MB default, but there are times one should.

- I heard, that HP-UX has the limitation of 8 parallel I/O per LUN - correct ?
==> Good question. I don't know, but I'd love to know.

live free or die
harry d brown jr

[Steven E. Protter]

Oracle recommends raid 1 or raid 10 for optimal performance.

Any kind of striping or raid 5 configuration is going to have much slower performance on writes, because of how many disks have to be updated for each write.

Also, its generally better to handle mirroring and such on the hardware side than with mirror/ux. This is coming from a habitual mirror/ux user.

What lvol size is best for performance depends on the data, which you have not given us any insight into. The notes I've just made are more likely to have an impact on peformance than logical volume size.

SEP

[Alzhy]

Armin,

I do not understand why you still "mirror" your volumes. Are'nt your Clarion LUNs already protected as do you paths to those LUNs?

Also, what value of "-r n" do you use -- meaning how wide or what's the number of columns are you building your stripes.? I usually go no more than 8-way nor go below 4-way when building my stripes.

My suggestion is to adopt an 8 way stripe as follows:

1. Create a VG that will contain 8 LUNS (or PVs).

2. Build your LVOLs as follows using a stripe size of 64 (which IMHO is neutral purpose -- addresses both OLTP and DSS needs, of course ymmv):

lvcreate -L 120000 -i 8 -I 64 -n sapvol3 /dev/sapvg3


For both my VxVM and LVM configs, I use either 4 or 8 way stripes - whether I am dealing with protected or un-protected LUNs (ie. JBODS).


Again, unless I am wrong -- your Clarion LUNS are already protected so there is no need to mirror them.

[Armin Schneider]

ok to shed more light on this:

- we use two cx700 - we mirror across boxes
- upgrading to dmx is no option ...
- it is a SAP System (based on oracle)
- raid5 is as fast as raid1 or raid10 - this rule of thumb is reffering back some years when storage systems had no fast processors and just a little write cache.
(we tried this out more than once ..)
=> SAP does about 90% read's and just 10 % writes ...
- option lvcreate -r n => no bad block relocation as the EMC does this itself.

- striping and mirroring is not possible once at a time => therefore we use option -D y (round robin of extents across the disks in PVG)


armin

[Alzhy]

Armin,

I was incorrect in asking "-r n". I meant to ask "-i n" - how wide do you stripe your Clarion LUNs "ON THE HOST".

It does not matter if those LUNs are already "striped" or RAIDed on the innards of the Clarion or any other array (except the EVA). You will always get better performance by further striping these LUNs on the host as I've outlined in the syntax. I really thing this is where your problem is.


I put an exception on the EVA as among the arrays out there, I believe the EVA does all the optimization for you in the back end.

[RolandH]

Of course, you will loss a lot of performance,
if you not use the performance opportunities that your OS and FS will offer you.

Have you ever read the manpages of tunefstab
or vxtunefs?

You should do this asap!

An example for Nelsons suggestions of a 8 way stripe can look like this in the /etc/vx/tunefstab

/dev/sapvg3/sapvol3 read_pref_io=256k
/dev/sapvg3/sapvol3 write_pref_io=256k
/dev/sapvg3/sapvol3 read_nstream=8
/dev/sapvg3/sapvol3 write_nstream=8
/dev/sapvg3/sapvol3 discovered_direct_iosz=8M
/dev/sapvg3/sapvol3 pref_strength=30
/dev/sapvg3/sapvol3 read_ahead=2
/dev/sapvg3/sapvol3 max_diskq=134217728

That is just an example. Read the manpages first an analyze your data on the disk. Then, I am sure you will be able to increase your performance.

Roland

[Armin Schneider]

thanks for your reply's
I'll go "studying" fs manpages and come back some days later with the result (including assigning points) or with new questions :-)


armin