topic Re: IO Perfs on Montecito server (RX6600) in Operating System - HP-UX

IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Mon, 20 Nov 2006 12:00:01 GMT

Hi all !

I am running Iozone on my newly installed servers and get very low performances when record size is 256K or more. This is the case when accessing either internal SAS disks or external EMC array.

My conf is HP-UX 11.23 MCOE 09/06 with up2date patches.

Iozone has been compiled on these servers.

Did some of you experience the same behaviour ?
Thanks for your help
Jean

Re: IO Perfs on Montecito server (RX6600)

Alzhy — Mon, 20 Nov 2006 12:48:54 GMT

Interesting.

Can you describe how you're testing? The nature of the filesystems and their underlying volumes?

Re: IO Perfs on Montecito server (RX6600)

rick jones — Mon, 20 Nov 2006 20:54:36 GMT

ISTR that VxFS will use "direct I/O" when the I/O size is 256KB or larger. This means that the DMA is performed directly to/from your application buffers, which means the application becomes blocked during the I/O and opportunities for read-ahead and/or write-behind become, well, very limited.

Alas, I cannot recall what the magic command is to change that value - something involving vxfstune or the like?

Re: IO Perfs on Montecito server (RX6600)

Alzhy — Mon, 20 Nov 2006 21:01:45 GMT

Rick,

Are you surmising "Discovered DirectIO" ? There is indeed a tunable for that... I will post once I recall definitely what it is.

However, iozone features tests for RAW device IO.. Our friend should test that as well.

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Tue, 21 Nov 2006 06:00:21 GMT

Thanks for your valuable answers ! I didn't know the information you stated, Rick.

Please notify when you remember the right tunable. I'll also try to find this information.

Thanks a lot.
Jean

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Tue, 21 Nov 2006 06:02:35 GMT

BTW, I am not nearby these servers right now and cannot perform the raw devices tests provided by Iozone.
My Oracle databases will be installed on VxFS filesystems.

Re: IO Perfs on Montecito server (RX6600)

rick jones — Tue, 21 Nov 2006 12:45:17 GMT

$ man -k tune
gettune(2) - get the value of a kernel tunable parameter
kmtune(1M) - query, set, or reset system parameters
settune(2) - set the value of a kernel tunable parameter
tunefs(1M) - tune up an existing HFS file system
tunefstab (vxfs)(4) - VxFS tuning parameters table
tuneinfo(2) - retrieve detailed information about kernel tunable parameters
vxtunefs(1M) - tune a VxFS File System

vxtunefs(1M) vxtunefs(1M)

NAME
vxtunefs - tune a VxFS File System

SYNOPSIS
/sbin/vxtunefs [-ps] [-f tunefstab] [-o parameter=value]
[{mount_point|block_special}]...

...

The VxFS tuneable parameters are:

...

discovered_direct_iosz
Any file I/O requests larger than the discovered_direct_iosz are
handled as discovered direct I/O. A discovered direct I/O is
unbuffered like direct I/O, but it does not require a synchronous
commit of the inode when the file is extended or blocks are
allocated. For larger I/O requests, the CPU time for copying the
data into the buffer cache and the cost of using memory to buffer
the I/O becomes more expensive than the cost of doing the disk
I/O. For these I/O requests, using discovered direct I/O is more

Hewlett-Packard Company - 2 - HP-UX Release 11i: November 2000

vxtunefs(1M) vxtunefs(1M)

efficient than regular I/O. The default value of this parameter
is 256K.

I have also heard, but do not know where to go to confirm, that things like MSA arrays will bypass their own caches for "really large I/O" - also > 256 KB. It might be a good idea to check the specs of the storage you are using to see if it does that too.

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Wed, 22 Nov 2006 06:48:38 GMT

You got it !

I changed this parameter (discovered_direct_iosz) on my test servers and the Iozone results dramatically improved.

I'll try to know what value was applied on filesystems before the September 2006 release of HP-UX 11.23.

Thanks a lot for your help.
Jean

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Wed, 22 Nov 2006 07:47:41 GMT

With March 2006 release of HP-UX 11.23, I had these parameters for VxFS filesystems :

Filesystem i/o parameters for /test

read_pref_io = 65536
read_nstream = 1
read_unit_io = 65536
write_pref_io = 65536
write_nstream = 1
write_unit_io = 65536
pref_strength = 10
buf_breakup_size = 131072
discovered_direct_iosz = 262144
max_direct_iosz = 1048576
default_indir_size = 8192
qio_cache_enable = 0
write_throttle = 0
max_diskq = 1048576
initial_extent_size = 8
max_seqio_extent_size = 2048
max_buf_data_size = 8192
hsm_write_prealloc = 0
read_ahead = 1

Today, with 09/2006 release of HP-UX 11.23, I have the same (values for these parameters are identical in 03/2006 and 09/2006), plus these new ones :

inode_aging_size = 0
inode_aging_count = 0
fcl_maxalloc = 65075200
fcl_keeptime = 0
fcl_winterval = 3600
oltp_load = 0

The I/O behaviour for filesystems created with these 2 versions of HP-UX seems to be very different.

Re: IO Perfs on Montecito server (RX6600)

Alzhy — Wed, 22 Nov 2006 08:19:48 GMT

Jean,

There should as well be a document on tunuing OJFS (aka VxFS) somewhere on HP's websites. In that document, these parameters are fully described so you can tune accordingly.

Have you tried at all comparing RAW performance versus your adjusted filesystems (for discovered DirectIO)? It would be very interesting indeed to find out the differences.

Thanks.

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Wed, 22 Nov 2006 09:52:13 GMT

Nelson

I have not been able to perform Raw IO tests yet (maybe at the end of this week).

I could not find any information concerning the new FS parameters. If you did, please provide the corresponding URL.

Thanks
Jean

Re: IO Perfs on Montecito server (RX6600)

Juan_20 — Tue, 27 Feb 2007 15:42:19 GMT

Hi

Do you solve this problem Jean ??
Same problem here.

Re: IO Perfs on Montecito server (RX6600)

Jean Mesquida_1 — Wed, 28 Feb 2007 03:02:58 GMT

Hola Juan !

This problem seems to be met whenever a new version of Veritas VxFS is installed (on HP-UX or Sun Solaris). The "solution" is to increase the discovered_direct_iosz to, say, 10 Mo : then the behaviour looks more like it was one year ago... Use vxtunefs to do this.

HTH
Jean