IDXFILEFULL, index file is full

[Winfried Wetzel]

I am receiving the following message:
-SYSTEM-W-IDXFILEFULL, index file is full

Is there a way to extend INDEXF.SYS?

[Willem Grooters]

AKAIK: No. Full = Full.
You've run to the maximum number of files you have allowed on that disk during INIT. This equals the number of entries you allow in INDEXF.SYS (actually, the other way round but the result is the same)

Then, these are your options:

* purge the whole disk.
* move files (preferably whole directory trees) to another disk.
* make a backup /image of that disk, init the disk with a larger number of files allowed and restore. There is a calculation on what you could allow, my guess is the system mnanager's manual.

To prevent, you could limit the number of copies, if the problem is caused by large amount of copies of files.

Willem

[Lokesh_2]

Give $SHO DEV/FULL to see the maximum no. of files allowed on the disk. And take actions as suggested by Willem.

Best regards,
Lokesh

IDXFILEFULL, index file is full

Faci*ity: SYSTEM, System Services

Explanation: No more files can be written on the volume, because the
maximum size of the index file specified is reached when the
volume is initialized.

User Action: If the volume is a private volume, delete the unwanted files
on the volume. If the volume is a system volume, notify the
system operator or system manager.

[Didier Morandi_1]

Although I fully agree with my dear colleagues, may I suggest that you first give up with unused headers? Those who are marked for delete, for example. You should find plenty.

To do that:

$ ana/disk/rep/nocon your_disk

forget about error messages, then you should get a clean index file. If this is not inuf, then you'll have to consider other options.

D.

[Willem Grooters]

Didier,
I thought that when a file is deleted, it's index slot will be re-used? Or is that after 'some time'. If it says "Marked for deletion", when is it actually deleted so the index slot can be reused?

[Didier Morandi_1]

>I thought that when a file is deleted, it's
>index slot will be re-used? Or is that after
>'some time'. If it says "Marked for deletion",
>when is it actually deleted so the index slot
>can be reused?

There is no "index slot" in ODS-2/5 file management.

A file "existence" is due to two parameters:

the file header, which is an entry in the INDEXF.SYS file

a bit set in the BITMAP.SYS file

When a file is created, the system scans the INDEXF.SYS file for a free block in which to write the header data of the new file (more or less the same data as the ones you see when doing a DIR/FULL on the file), then it sets a bit to 1 in the BITMAP.SYS file to remember that this block is "in use". If you create an empty file, the system will allocate no blocks, whatever the cluster factor of the disk is.

Then, when you populate the file, the system will allocate a certain number of blocks and write to it. Each time a block is allocated to a file, the bitmap is updated and the header block too, with the first logical block number of the "data" part of the file (called the retrieval pointer) and the number of contiguous blocks in which your data are actually stored. If the file is contiguous, you'll get one retrieval pointer and a count. If it is fragmented, you'll get more than one.

Now, when you delete a file, the system will clear the bit in the BITMAP.SYS file which matches the header block of the file in INDEXF.SYS. The header is still here (and the data too) but the block is marked as "free", hence the message "file marked for delete".

When VMS needs a new block to write a file header, it will scan the INDEXF.SYS file and not the BITMAP one. So, when there are no more free blocks in INDEXF.SYS, you get your error message "index file is full".

The $ ana/disk/rep/nocon will clear in the INDEXF.SYS file all headers which bit in the BITMAP.SYS is cleared.

All this from top of my head. I gave a presentation to DECUS in 1982 on how I successfully recovered an RP06 badly initialised, "just" by filling the BITMAP.SYS with "1", I then dumped the indexf.sys file to recover its correct checksum (the dump program kindly told me "invalid checksum, should be blablabla" :-), then I patched the INDEXF.SYS file with the right checksum (the header of indexf.sys is within the file), then I did an $ ana/disk/rep/nocon on the disk and I got it back safe.

I still remember today the applauses at the end of my session...

To see a formatted header within the indexf.sys file, use the command:

$ dump/file_head/form/block=(start:7136,count=1) indexf.sys

where the start block is actually the first parameter of the file ID plus the offset from the beginning of the indexf.sys file to the header part (before the header part, there are a couple of other file management related blocks on which I will not talk now). The offset for my Alpha 7.3-1 and a RZ29 is 139.

So, to know the block number of the header of the file you are interested in, do a DIR/FULL of the given file, then add 139 (or your value) to the file ID and you'll get your entry.

Example:

DTL02> dir/full A.A

Directory DKA0:[000000]

A.A;2 File ID: (6997,7,0) Size: 1/9 Owner: [SYSTEM]

6997 + 139 = 7136

dump/file_head/form/block=(start:7136,count=1) indexf.sys

Dump of file DKA0:[000000]INDEXF.SYS;1 on 9-DEC-2003 09:07:01.69
File ID (1,1,0) End of file block 61814 / Allocated 61821

Virtual block number 7136 (00001BE0), 512 (0200) bytes

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (6997,7,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Variable
File organization: Sequential
Record attributes: Implied carriage control
Record size: 6
Highest block: 9
End of file block: 1
End of file byte: 8
Bucket size: 0
Fixed control area size: 0
Maximum record size: 255
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics:
Caching attribute: Writethrough
Map area words in use: 2
Access mode: 0
File owner UIC: [SYSTEM]
File protection: S:RWED, O:RWED, G:RE, W:RE
Back link file identification: (4,4,0)
Journal control flags:
Active recovery units: None
Highest block written: 1
Client attributes: None

Identification area
File name: A.A;2
Revision number: 2
Creation date: 9-DEC-2003 08:52:59.29
Revision date: 9-DEC-2003 08:52:59.41
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 9 LBN: 3730113

Checksum: 62456

D.
(where is my Aspirin? :-)

[Willem Grooters]

Didier,

I've got bags full of asperin, just in case ;-)

Really: thanks for the explanation. I knew this for the larger part, the only surprise was where the bitmap comes in on file deletion. Still it's good to understand the whole lot.

[John Gillings]

Although most of what Didier said is correct, I have to disagree with:

"may I suggest that you first give up with unused headers? Those who are marked for delete, for example. You should find plenty."

If you find any headers marked for delete, you should investigate why. They will typically only result from a system crash or power failure. An OpenVMS system should not accumulate junk files unless something is wrong!

If the disk is badly fragmented, you may also be able to recover headers by defragging the disk (but then that will require a few headers to be available, so it might not be possible).

As of V7.3-2, it makes a lot of sense to INIT all disks with very large (maximum possible) values for /MAXIMUM_FILES and /LIMIT. In the past, the proportion of the disk required to preallocate allocation and header bitmaps for maximum possible extensions was relatively large. Today it's little more than noise. (~32MB for the allocation bitmap and 2MB for the header bitmap) Existing disks can be setup for a higher limit with SET VOLUME/LIMIT, but the disk needs to be privately mounted. Do this now and you'll have much more flexible options for managing your disks in the future.

Unfortunately, even with dynamic volume expansion, there is no way to extend /MAXIMUM_FILES without a reINIT.

[Hein van den Heuvel]

Winfried, You are SOL. Sh*# Out of Luck.

You'll have to go find some files to delete and maybe run anal/disk just in case there is stale information as Didier suggests.
Apparently thre were more files created then expected (disk size divided by average file size).

Or... is this system per chance using excessive amounts on ACLs on the files? Lot's of ACEs/ACLs will cause single files to require multiple headers. I would recommend DFU (freeware) to analyze.

Didier, pretty close but not good enough.
There ARE index slots. They are managed in the "Index File Bitmap". This is stored within indexf.sys on offset 4*cluster-size + 1. It is pointed to by HM2$L_IBMAPLBN from the homeblock (vbn 2 in indexf.sys).
You kinda contradict yourself:
a) a file exist if it has "a bit set in the BITMAP.SYS file"
b) If you create an empty file, the system will allocate no blocks
An empty file has NO entry in BITMAP.SYS.

Also, bitmap.sys is pre-allocated on system boot / disk mount and each bit represents a cluster, not a block.

Files 'marked for delete' still exist. Their headers can not be used. A normal delete will indeed leave most file header data there, but will NOT 'mark for delete'. Just flipping the bit in the index file bitmap is enough. FCH$V_MARDEL = "Set if file is marked for deletion. If this bit is set, further access to the file is denied, and the file is physically removed from the disk after the last user has closed it."

All, Every self respecting VMS nut MUST have the 'VMS Files System Internals" book (Kirby McCoy Digital Press DB ISBN 1-55558-056-4). It details what I try to explain above.


Hein.

[Didier Morandi_1]

Now you understand why I wrote "before the header part, there are a couple of other file management related blocks on which I will not talk now".

I actually forgot about the bitmap block of the INDEXF.SYS file and got confused with BITMAP.SYS...

Thanks, Hein.

D.

(was not too bad after twenty years, was it? :-)