Chunksize query

[Andrew Houghton]

I'm just reconfiguring our SAN (EMA12000) and reconstructing RAID sets. The manuals mention setting chunksize rounded to a prime number - yet defaults are either 128 or 256. As these are not prime, does anyone recommend a suitable chunksize if we're writing approx 30,000 x 2k records every 30 seconds - mostly sequential stuff. Previously, it had been set at 128 although I'm unsure as to why
(OpenVMS 7.3-1)
cheers
Andrew

[RBaker]

Here's the official line:

Chunk Size
Specify the chunk size of the data to be stored to control the stripesize used in RAIDsets
and stripesets:
n CHUNKSIZE=DEFAULT lets the controller set the chunk size based on the number of
disk drives (d) in a stripeset or RAIDset. If d £ 9 then chunk size = 256. If d > 9 then
chunk size = 128.
n CHUNKSIZE=n lets you specify a chunk size in blocks. The relationship between
chunk size and request size determines whether striping increases the request rate or
the data-transfer rate.
Increasing the Request Rate
A large chunk size (relative to the average request size) increases the request rate by
allowing multiple disk drives to respond to multiple requests. If one disk drive contains all
of the data for one request, then the other disk drives in the storageset are available to
handle other requests. Thus, in principle, separate I/O requests can be handled in parallel,
thereby increasing the request rate.
Large chunk sizes also tend to increase the performance of random reads and writes. It is
recommended that you use a chunk size of 10 to 20 times the average request size,
rounded to the closest prime number. In general, 113 works well for OpenVMSâ ¢ systems
with a transfer size of 8 sectors.
To calculate the chunk size that should be used for your subsystem, you first must analyze
the types of requests that are being made to the subsystem:
n Lots of parallel I/O that use a small area of disk should use a chunk size of 10 times the
average transfer request rate.
n Random I/Os that are scattered over all the areas of the disks should use a chunk size
of 20 times the average transfer request rate.
n If you donâ t know, then you should use a chunk size of 15 times the average transfer
request rate.
n If you have mostly sequential reads or writes (like those needed to work with large
graphic files), then make the chunk size a small number (i.e. 17 sectors).
Increasing the Data Transfer Rate
A small chunk size relative to the average request size increases the data transfer rate by
allowing multiple disk drives to participate in one I/O request.
Increasing Sequential Write Performance
Sequential write and read requests on stripesets (or striped mirrorsets), should use a small
chunk size relative to the I/O size to increase the performance. A chunk size of 17
generally wor

[Andrew Houghton]

Richard,
Appreciate the answer and advice. However, I'm still confused, although no where near as confused as HP seem to be. Most of your answer is as you say 'the official line', (as I've read in the manuals). However - I'd not come across the statement "In general, 113 works well for OpenVMS systems"
Whats confusing is :-
8.5 CLI guide - gives the same text as you mention.
The 8.6 guide uses the same text, but qualifies it with "Compaq does not recommend a chunksize less than 128 blocks (64K)". Now, consider that statement and then explain:-

The 8.5 configuration guide (the only version I've got or found) - This is where it suggests using primes, and indeed gives a table with sizes ranging from 41 to 317. Pages 2-25 and 2-27 state "If you have mostly sequential reads or writes ..... then make the chunk size a small number (i.e. 17 sectors)." and "Sequential write and read on stripesets (or striped mirrorsets), should use a small chunk size.......A chunk size of 17 generally works well".

Why 17 or any figure below 128 if less than 128 is not recommended?
What is 17 sectors?

And why the next statement :-

"IMPORTANT: The default chunksize should be used with caution.......an improper value may cause serious performance problems"

Do they suggest primes or not?
Do they suggest less than 128 or not?
Do they suggest defaults or not?


Yours,
Confused at HP.com

(N.B. I've already had to take the plunge, and have set them as previous - 128)

Andrew

[RBaker]

These are the generalizations I use â in Englishâ when optimizing: (Please check before use), some may not be scientific but work.

1) The controller cache works in blocks of 512 Bytes (1/2k).

2) 2-4k may be considered as high request, 32-64k may be considered as high data rate.

3) In a high request rate environment, Chunksize 10 â 20 times the I/O size, usually rounded to the nearest power of 2, e.g. 4k I/O size is 64 â 128, balancing the I/Oâ s evenly across all the spindles. In your case 2k yields 32 â 64.

4) In a high data rate environment then a smaller Chunksize, matching the I/O size, e.g. 128k I/O across 6 spindles yields approx 32.

5) A Chunksize that is too small for the I/O multiplies the number of I/Oâ s necessary *2, *4, *8 etc., one that is too large just minimizes the potential performance to be gained by tuning. So is much better to go with a slightly larger one than that calculated or the default. Which in your case is the next power of 2, i.e.64-128.

6) Not a brilliant explanation but the prime number thingy is to avoid possible â natural boundariesâ a bit like the cracks joining up when bricklaying, or if the I/O is variable and is rounded up will minimize possible I/O multiplication.

7) VMS has some â driver tuningâ built in, so in the main, I/O size tends to be larger, hence a lower chunksize than the norm would be acceptable.

In all cases use the next power of two when calculating and add the prime factor when I/O size is variable. In addition remember a larger chunksize will have nominal effect, a smaller will multiply I/Oâ s dividing your performance as appropriate. After significant performance testing on my part, a chunksize of 128 has been appropriate, little gain has been made less than 64 or greater than 256. In addition as the SCSI packet has a maximum payload of 64k which equates to 128 1/2k blocks so => this figure will

[mark_511]

bling... 128 is the correct choice!