Block / Cluster calculations

[Aaron Lewis_1]

I am trying to optimize a volume to store about 300,000 small RMS files. currently they are on a 9GB drive (DKC100) with the default cluster size of 18. I found an 18GB drive (DKE100)and did an init/limit with the default cluster size of 8.

I have 5 test files, size: 1,2,2,3,5

Pure: 13 blocks (1+2+2+3+5)
DKC100: 90 blocks -- 5 * 18 (1 cluster)
DKE100: 100 blocks -- 5 * 20 (2.5 clusters?)

Why do these files take up more space on a disk with a smaller cluster size? Also, I thought a file had to occupy a full cluster.

[Hein van den Heuvel]

You do not indicate a VMS version.
If space is that critical, maybe you should upgrade to a VMS version that allows more than 1,000,000 clusters on a disk?
Oh wait... you have that already. Never mind.

It's just a matter of math.
8 can only be divided by 2 and 4.
18 can be divided evenly by 2,3,6 and 9.

I'm afraid I did not fully understand the description of the sizes you gave, but one has to assume that the 18 block cluster allows a few more files to fit 'closer'.

I would suggest you try a cluster size of 6 next.


Hint: for hands-on experiment with this consider virtual devices like the LD (logical disk) and notably the MD memory disk driver.

hth,
Hein.

[Aaron Lewis_1]

Hein, I am using 7.3-2. My understanding of RMS files is that each file allocates at least 1 cluster, reguardless of the actual size of the file.

This appears to be true on DKC100, since each of the small files, with real sizes of:
1 block, 2 blocks, 2 blocks, 3 blocks & 5 blocks, each allocate a full cluster of 18 blocks resulting in 90 blocks of storage being used.

However, when I copy these to DKE100, with a cluster size of 8, each file allocates 2.5 clusters, or 20 blocks each using up a total of 100 blocks of storage. Since each file is smaller than the defined cluster size, I would expect it to allocate only 1 cluster each for a total storage of 40 blocks. Also, I did not think that a file could allocate anything less than a full cluster.

[Volker Halle]

Aaron,

the allocated file size (DIR/SIZ=ALLOC)always is a multiple of the cluster size, the used file size (DIR/SIZE=USED - this is the default for DIR/SIZE) can be between 0 and allocated size.

Volker.

[Volker Halle]

Aaron,

is the cluster size really 8 ???

$ WRITE SYS$OUTPUT F$GETDVI("DKE100","CLUSTER")

Volker.

[Hein van den Heuvel · Accepted solution]

> each file allocates at least 1 cluster, reguardless of the actual size of the file.

Right. (allthough 0 clusters is also allowed :-)

>> However, when I copy these to DKE100, with a cluster size of 8,

Ah... copy will round up file allocation as it can not be sure the the EOF is meaningfull. Depends on high-water-marking and such. Application are allowed to put data beyond the 'official' EOF. It would be silly, but it is possible. Copy has to play it safe and copy all. Su each file which needed 1 18 block clusters now will need 3 - 8-block clusters.
That's why I suggested 6 block clusters, as they divide nicely into 18.

>> each file allocates 2.5 clusters

As you suspect, this is wrong. Only whole clusters can be allocated. TO understand how and where, I would recommend using DUMP/HEAD/BLOC=COUN=0 for each of the 5 files and look for the MAPPING POINTERS to get the explanation.

There are more details on the round up / truncate / copy vs backup that I have written about before. I can dig that up if critcally needed.

It comes back to the old 'What problem are you really trying to solve'?
If that problem includes copying from a 18-bluc cluster disk to 8, then we have a problem which you can workaround with clustersize 6.
If the real application will simply create the new files on the new disk, then there is no problem. You'll see 8-blocks / file for every file of 8 or less.


Hein.

[Jeroen Hartgers_3]

with dir/size=all you see used/allocated. VMS 7.3.2 gives you the opetunatie to size down your cluster size during the init of your disk. don't forget to make a backup. init is the same as format(msdos).

If you make your clustersize smaller you need a lager index file and you must calculte de max file for the device by hand.

the hel-p of vms tells you ervery thing about init ( $help init) The easiest way to learn/test this is to use a spare disk and look wath's happening.

[Jeroen Hartgers_3]

with smaller clusters, defragmentation is possible and you need a bigger indexf.sys

[Joseph Huber_1]

I don't think it is VMS version dependent, but anyhow:
on my 7.3-1 system I do not see such behaviour. If I copy from big-cluster-sized to smaller-cluster-sized disk, it allocates only the cluster-size.

BUT: copy using backup takes the source allocation (and rounds up to multiple of cluster-size), exactly what You probably see on Your disk.

[Robert Gezelter]

Aaron,

Often space allocation will have a "sweet spot" (or an egregiously bad spot) for a population of files.

Consider the following file sizes:

Space Used (by cluster size)
File Size 3 6 9 12 15
1 3 6 9 12 15
10 12 12 18 24 15
20 21 24 27 24 30
40 42 42 45 48 45

This is referred to as "breakage", since, as you noted, it is not possible to allocate "half a cluster".

Out of curiosity, what are these files? Sequential, Indexed, etc.?

- Bob Gezelter, http://www.rlgsc.com