OnlineJFS/VxFS performance problems?

[Eric Ladner]

I have an L2000 running OnlineJFS (with nodatainlog delaylog mincache=direct convosync=direct set in /etc/fstab). The disk subsystem is SC10's with mirrored pairs mirrored with a RAID 4si card.

The problem we are seeing is that when a large operation is taking place, in some cases, takes a terribly long time, but others are as fast as expected.

For example, to copy a 300MB tar archive from one disk to another takes about 20 seconds. To untar that same file from one disk to another takes over an hour. Something isn't quite right here.

Any thoughts?

[A. Clay Stephenson · Accepted solution]

Hi Eric:

This makes perfect sense. These options which intentionally bypass the UNIX buffer should ONLY be used in applications which buffer the data themselves (e.g. Oracle). You are crippling tar because it can not take advantage of the buffer cache.

Regards, Clay

[Eric Ladner]

Still, even looking at the raw throughput at the disk level, there's no way it should take an hour to write 300 meg of data to a 10,000 rpm disk.

True?

[A. Clay Stephenson]

Hi again:

I should also add that OmniBack has an option OB2VXDIRECT that does allow high performance backups of these filesystems.

[A. Clay Stephenson]

If both the source and destination were mounted like this, the performance would be awful. I suppose the easy test is to remove the mount options convosync=direct and mincache=direct and see if that clears up your problem but I can tell you that I have seen very poor performance with these options althogh yours is extreme.

[Eric Ladner]

Given that this is indeed a volume used for Oracle, is there a slight compromise in mount options that will stil give oracle most of the benifits of direct i/o but not kill large file operations to the extent I'm seeing now?

[T. M. Louah]

To add to above:
The convosync=direct, mincache=direct bypass the UNIX buffers -
the performance is almost exactly the same as raw disk. If you use this option, you should
reduce the UNIX buffer cache and use the memory to increase the SGA buffers.

refer to this discussion thread :
http://forums.itrc.hp.com/cm/QuestionAnswer/0,,0xd429cbaac6dcd5118ff40090279cd0f9,00.html

check this document about VxFS Performance-Cache Advisories:
http://docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B3929-90011/B3929-90011_top.html&con=/hpux/onlinedocs/B3929-90011/00/00/22-con.html&toc=/hpux/onlinedocs/B3929-90011/00/00/22-toc.html&searchterms=performance%20disk&queryid=20020216-060116

G`d luck
T+-

[A. Clay Stephenson]

Hi Eric:

In a word, no. The options that bypass the cache are the ones that optimize Oracle. Your best option, if you want to use this filesystem for both Oracle and other more general purposes is to remove both =direct mount options. If you have plenty of memory this is a good option.