topic Re: calculating transfer size on vms disk in Operating System - OpenVMS

calculating transfer size on vms disk

mark_511 — Fri, 16 Jul 2004 05:13:56 GMT

how can I calculate the transfer size on a vms disk. Required for setting chunksize on RAID set. I need to find what the average transfer size is to the current drive.

Re: calculating transfer size on vms disk

Ian Miller. — Fri, 16 Jul 2004 05:19:32 GMT

on current version of VMS see
SHOW MEM/CACHE/FULL

Re: calculating transfer size on vms disk

mark_511 — Fri, 16 Jul 2004 05:21:56 GMT

Yes, tried this first but lists x Block IO. What does "Block IO" refer to in this context

Re: calculating transfer size on vms disk

labadie_1 — Mon, 16 Sep 2024 09:33:07 GMT

Until recently, most of the I/O made were of less than 16K.

$ monitor disk/item=all

gives the queue length and the I/O operation rate. If you find how many I/O your model of disk gives when it has a queue length of 1 (for example), then it will be quite easy.
Check the specs of your disk.

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 05:33:45 GMT

Performace advisor can give you the IO size per program.

Wim

Re: calculating transfer size on vms disk

mark_511 — Fri, 16 Jul 2004 05:39:34 GMT

no, sho cache gives me what I need but I require clarification of what the "Block IO" refers to. Is it Bytes, Blocks (if so what size), K?

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 05:45:32 GMT

Current setting of the system parameter
VCC_MAX_IO_SIZE in blocks.
The extended file cache (XFC) will only cache files up to this size.

So it is blocks and for all IO's bigger than 127 blocks there are no statistics.

Wim

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 05:48:37 GMT

And show mem is not per disk.

PA can give you the IO size per disk.

Wim

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 06:39:36 GMT

And the block is 512 bytes.

What can you do with sh mem/cac/fu ?

Re: calculating transfer size on vms disk

Ian Miller. — Fri, 16 Jul 2004 06:45:11 GMT

"What does "Block IO" refer to in this context"

Consider this Example.

Transfer Size: Reads Read Hits Writes
1 Block IO: 1339139 1224267 624906
2 Block IO: 87599172 77137023 616565
3 Block IO: 6046985 5196849 181878

This saying there has been 1339139 read requests for reads of 1 512 byte block.
There has been 87599172 Reads for 2 blocks
There has been 6046985 reads for 3 blocks.

From this sort of data you can determine the most frequently requested size of read or write on your system. If you take this data and paste it into a spreadsheet you can plot histograms, calculate averages etc

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 06:46:36 GMT

But not per disk.

Wim

Re: calculating transfer size on vms disk

Wim Van den Wyngaert — Fri, 16 Jul 2004 07:34:26 GMT

And are the IO's bybassing the cache in it ?

Re: calculating transfer size on vms disk

Keith Parris — Mon, 19 Jul 2004 11:28:17 GMT

Another way to calculate average transfer size is to look at it down at the controller level. If you have HSx controllers, VTDPY can give you, on a per-disk basis, the average data transfer rate and the I/O request rate. Dividing the average request rate into the average data transfer rate in kilobytes gives you the average I/O tramsfer size in kilobytes.

Re: calculating transfer size on vms disk

Hein van den Heuvel — Mon, 19 Jul 2004 12:33:55 GMT

Ok folks, enough chatting about block sizes and caches. Let's get back to be basic question which is NOT calculating the transfer size as the subject suggestm, but it is figuting out an optimal chuk size for a raid set.

None of the prior replies are going to help with that.

IMHO are stuck by picking a highish number like 255 or 256. High enough to avoid excessive splitting of IO, low enough to pread hotspots.

It is exceedingly unliky that you can actually pick a hardware stripe size to optimally match your average IO size.

The AVERAGE is completely irrelevant in this calculation IMnsHO.
Only the MODAL value counts (or whatever the name of the statistical term that means 'most often used' :-). This is because a lot of VMS IOs are 1 blocks. For those the chunk size does not matter at all, but they do influence the average!

This is one of the few times where you may want to take the DISK ALLOCATION CLUSTER SIZE into consideration. Why? because you know there is a good chance that IOs will start as a cluster boundary. After all, every single file starts at a cluster boundary.

So by selecting the chunk size to be a straight multiple of the cluster size you will make sure that the first IO from the beginning of a file will not be spilt over chunks.

The following IOs are dependend on the IO sizes. For sequential files IO that is OFTEN, but not always 32 blocks. (SHOW/SET RMS!)So to keep those from fragmenting you could consider
CLUSTERSIZE = N * 32
CHUNKSIZE = N * CLUSTERSIZE

An other popular IO access pattens is Indexed file IO. There the size is BUCKETSIZE, but the first datablock (by default) tend to start at block 3 or 4, and does NOT start on a multiple of cluster no bucket size. Note, blocks intentt will be alligned, and you can play fdl games to start out aligned.

I'd pick 256 if I had some hope to force allignment on 2**N boundaries. I'd pick 255 if i was afraid to 'fall out of sync' and cause a pattern of splits. With the chunk size being prime you'll distribute the pain evenly :-).

hope this helps,
Hein.

Re: calculating transfer size on vms disk

Hein van den Heuvel — Tue, 20 Jul 2004 15:27:16 GMT

[ 0 points for this reply ]

I'm glad you like my reply (judging by the 10p) despite the horrible spelling. Sorry 'bout that. I just started typing and never looked back.

One cute (imho) line I was going to put in there was "when is the last time you did an 'average' IO?". Just to put further focus on the 'modal' value selections, not median, not average.
4 IOs 1 block
3 IOs 4 blocks
2 IOs 8 blocks
8 IOs 16 blocks.

Total 17 IOs, 160 block, avg=9.4, med=8.5, mod=16. Pick 16 as the number for the sizing, not 9.5.

cheers,
Hein