Directory corrupted?

[Willem Grooters]

I encountered the following problem:

Within an executable, a file is to be created.
The filename is determined by data that is used within the program (eq. "0300889.XXX")
For years there has been no problem, but suddenly this file can not be created and the programs stops. Running the very same sequence on other data that is close to this number (for instance "03000891.XXX") gives the same error but when further away ("0300780.XXX") there'se nothing wrong, the file is simply created. Creating file with any name in another directory is no problem, nor in this directory, but then only as long as the filename isn't ""0300889.XXX"...

I renamed the directory the files are created in, created a new directory to hold the files and copy them all into it. Now the problem is gone and as I found out, today I can create the file without problem - in this (renamed) directory.
However, I'm still concerned what could have caused it; My guess is a corrupted directory.

It concerns a 2-disk shadowset, that is about 90% in use; none of the disks show errors in SHOW DEVICE. Once a week the disk is defragmented by Diskeeper, and the disk fragmentation index is about 36 - said 'good'. One day it crashed exceeding process quota but that _should_ not lead to a corrupted directory.
There is a heavy load on this disk: files are created, modified and deleted quite often so Diskeeper will hardly do any defragmentation in background.

Any ideas of the cause?

[Hein van den Heuvel]

Cause: not enough contig space. Solution: DFU

How big was the old Directory?
How much CONTIGUOUS free space do you think there might have been at the time of trouble?

Directories need to be contiguous. They are NOT compacted after a simple insert. A simple inserts goes to a target block directed by the name and if there i no room then the whole directory is grown through copuy into a new contiguos allocation. If that room is not there, then the file create failes (well, the enter fails). A file with a slightly different name could easily be targetted to a different block, with some spare room, allowing create to succeed. Eventually that will also run out though.
The massive rename compacted the directory requiring a smaller one, and thus succeeded.

The real solution is to learn to use DFU (on the VMS freeware (cd, web) and 7.3.-1 distribution. DFU gives you quick and cool disk usage stats, finds, and can compress directories.

Met vriendelijke groetjes,
Hein.

[Rick Dyson]

It is too late now, but I would have suggested an
Analyze/Disk/Log. It would have been interesting
to see if the filesystem could detect the problem
within the DIR file.

For years, my heavily used DIRs will occasionally
get corrupted blocks that Analyze will report.
I typically just create a new one side-by-side w/
a temp name, move all the files from the old to
the new, delete the old and rename the new back
and all is well.

To be honest, I have never tried to locate the
source of the problem. I just consider it normal
routine maintenance. :)

NOTE: I only use the HP DFO product. Never used
Diskeeper.

Regards,
rick

[Rob Buxton]

Are you sure this wasn't just a version number problem?
You can only create file versions up to 32767.

[Willem Grooters]

As Hein suggested, the cause is very likely lack of contiguous space. I checked: disk has clustersize of 12, directory-file was 216 blocks in size and contained approximately 1500-1600 files. So if the largest chunk of contiguous space was less than 216 + 12 = 228 blocks, the _file_ could be created, but when the directory needed expansion, ther'se no room. (A better solution would IMHO be another message: "Not enough contiguous space to expand directory, file not craeted", or something like that, it tells you what is the case!)
Alas, since the directory was cleaned-up (purged and unneeded files deleted) so there's no way to get more info on the state. Anyway, the disk needs to be reorganized (both functional and physically) and it has been scheduled.
I'll checkout DFU.

Still: a question on this matter.
Since just a single name causes a problem - and names close to it, while names that differ quite a lot, do NOT, there must be some algorithm within RMS that causes this problem to occur. Can anyone explain this (I do not need a full RMS course, just a brief explanation ;-))

[Hein van den Heuvel]

RMS does NOT write/update Directories.
It only reads them. The XQP writes them.

The XQP organizes directories records(filename + (version/file-id) tuples)
in 512 byte blocks. If a block is full,
it is split. As filenames get removed (delete), blocks are only recovered when
completely empty. So some blocks might
only have bytes left, while other blocks
might only have a few bytes used (well, filename size + 16-bit version + 48-bit file id).
For grins, check out DUMP/DIR/BLOC=(STA:x,COU:y)

You'll love DFU.
Hey, A dutch guy (Jur) wrote it, It's gotta be good no?

Hein.

[John Gillings]

Willem,
> (A better solution would IMHO be another message: "Not enough contiguous space to expand directory, file not craeted", or something like that, it tells you what is the case!)

You didn't post your failure message. The "usual" message for this condition and its HELP/MESSAGE text is:


DIRALLOC, allocation failure on directory file

Facility: SYSTEM, System Services

Explanation: The file system failed to allocate space to increase the size of a directory file. Because directory files must be contiguous, this error might be caused by the disk being full. More likely, there is not enough contiguous space on the disk for the directory, so the free disk space is being fragmented.

User Action: Reorganize the free disk space by copying it with the Backup utility, or restructure your application to use a larger number of smaller directories.


VMS does not "clean up" directories automatically. As Hein suggested, DFU has various function for maintaining and repairing directories.

[Willem Grooters]

John,
I wish I had the VMS message, but the error is handled by the application - so it's hidden for the end-user - and by the runtime environment (Synergy/DE (Dibol)) - so I didn't get the real problem or message. Sorry, I forgot that info. It's well possible that VMS signalled DIRALLOC error that is 'translated' by the runtime-library to a more generic status.
And Hein, yes, quite a number of very usefull VMS-freeware has been written by Ducthmen - LDDriver to name one.

[Pim van Velzen]

>You'll love DFU.
>Hey, A dutch guy (Jur) wrote it

I think you are mixing up two dutch guy's (kaaskoppen), here ...

[Hein van den Heuvel]

>>You'll love DFU.
>>Hey, A dutch guy (Jur) wrote it
>
>I think you are mixing up two dutch guy's (kaaskoppen), here ...

Ooops. It was Ton Dorland. I knew that :-).

heren,

Al die Hollanders lijken zo op elkaar! (niet).
Zo verwarrend:
- Ton Dorland voor DFU (in 1993),
- Jur van der Burg voor lddriver,
- Hein van den Heuvel voor sidr/rms_tune_check

Het opschonen van directories zal wel vaak en door velen her-uitgevonden zijn, maar het eerste serieuze werk dat mij bekend is was ik zei de gek, in 1985 of zo in Valbonne, met Colin Blake. Uiteindelijk werd dat het 'comdir' tool en nu dus dfu.

Groetjes,
Hein.

[Willem Grooters]

Ach ja....

I'll continue in English - (er zijn niet zovell Nederlanders (wat ruimer dan Hollanders' nietwaar, Hein?)

- Het opschonen van directories zal wel vaak en door velen her-uitgevonden zijn,

(tranlation: Cleaning up directories will have been re-invented very often by many ....)

so I wonder WHY there is no function in VMS to do this - in other words: whu is DFU not integrated and do we have to rely on the FreewareCD?

(Put it on the wishlist for 8.1)

[John Gillings]

Willem,

Your wish may be granted sooner than you think...