unable to create file on the disk

[megat razizul]

I using vms 7.3-2 version and attach SAN storage as their logical disk. We have DSA1 as one of logical drive. What is major issue here unable to create or copy file into DSA1. We already check disk size but it still have lot free size. The error as above.

HSIBD3::D_CERT:> sh def
DSA1:[000000]
HSIBD3::D_CERT:> CREATE testmeg.DAT
%CREATE-E-OPENOUT, error opening DSA1:[000000]TESTMEG.DAT; as output
-RMS-E-CRE, ACP file create failed
-SYSTEM-W-IDXFILEFULL, index file is full
HSIBD3::D_CERT:> sh dev dsa1:/full

Disk DSA1:, device type HSV110, is online, mounted, file-oriented device,
shareable, available to cluster, error logging is enabled, device supports
bitmaps (no bitmaps active).

Error count 0 Operations completed 372439
Owner process "" Owner UIC [FIELD,SYSTEM]
Owner process ID 00000000 Dev Prot S:RWPL,O:RWPL,G:R,W
Reference count 10671 Default buffer size 512
Total blocks 41943040 Sectors per track 128
Total cylinders 2560 Tracks per cylinder 128
Logical Volume Size 41943040 Expansion Size Limit 48029696

Volume label "ALPHASYS" Relative volume number 0
Cluster size 41 Transaction count 1144
Free blocks 27195710 Maximum files allowed 499321
Extend quantity 5 Mount count 1
Mount status System Cache name "_DSA1:XQPCACHE"
Extent cache size 64 Maximum blocks in extent cache 2719571
File ID cache size 64 Blocks in extent cache 533
Quota cache size 0 Maximum buffers in FCP cache 7549
Volume owner UIC [1,1] Vol Prot S:RWCD,O:RWCD,G:RWCD,W:RWCD

[labadie_1]

Welcome to the OpenVMS forum

The index file on your disk is full, so you can't create more files.

When you initialize a disk, you have 2 important qualifiers
- maximum_files
- headers

A file can use one or more headers, so both qualifiers are important.

Now, in order to be able to create some more files, you need to delete some.

Maybe you can move some files from this disk to another, or some directories ?

Something like
$ bac/lo/del dsa1: dsa2:<*...>/owner=origi
if you have some space on dsa2:

The real solution is to backup dsa1 (for example, when no files are opened)
then
$ init/maximum=500000/headers=1000000... dsa1:
then
backup/noinit...backup_disk: dsa1:


See the doc, "using backup"
http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/aa-pv5mj-tk.HTMl

[Jan van den Ende]

Megat,

to begin with:

WELCOME to the ITRC VMS forum!

On your problem, the ]error message says it all:

>>>
-SYSTEM-W-IDXFILEFULL, index file is full
<<<

Look at
>>>
Maximum files allowed 499321
<<<

Is this limit reached?

I am afraid your only ways out of this are:

- Init another drive with /maxfiles={at least double} and /headers = {max}
( look at help INIT for the max values at your version, as you already have max for /LIMIT )

- BACKUP/IMAGE/NOINIT DSA1: {new DSA#:}

As you having DSA suggests shadowing; perhaps you can split off one member, init that, and use as target.
Adapt your Startup sequence to use the new DSA whereever you are now using DSA1; and off you go.

(If DSA1 is your current System Disk, then also change SYSGEN's SHADOW_SYS_UNIT; and reboot).

Second potential cause:
DSA1: is badly fragmented.
Now the above also is the long-term solution, but the shortterm way out MIGHT be to find a file with many headers, and delete that (or move it to another drive).
But that will only help for a short time!

hth

Proost.

Have one on me.

jpe

[Jim_McKinney]

> Maximum files allowed 499321


Others have already written of addressing the long term needs of the disk by changin it's attributes - perhsps for the short term (or maybe even the long term) you might just delete some files and free up a few headers in the INDEXF.SYS file - or perhaps you could delete lots and lots and lots of files - do you really need to keep nearly 500,000 files on the disk?

[Hein van den Heuvel]

I don't think this problem is caused by /maxfiles being exhausted.
/headers is more likely to be the problem.

The disk has default init attributes:
1 million clusters, 1/2 million files.

There are 27M blocks free. 13M used.
If each file had just one cluster then there would be (41943040 - 27195710) / 41 = 359690 in use and still 150,000 available.

Maybe there are lots of empty files?
Maybe there is an excessive amount of ACLs on the files, requirign lots of extention headers?
Maybe [000000]INDEXF.SYS can not longer be expanded?

Megat,

To understand where the hurt comes from, get yourself the DFU tool and REPORT on the volume. http://www.digiater.nl/dfu.html
Add Report output as .TXT attachment to future reply if need be

To work around the problem you need to remove files from the volume.
Maybe you can combine a bunch of little files in a BACKUP or ZIP container?
Maybe create an LD container file to hold lots of little file?

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Jon Pinkley]

I agree with Hein, the most likely problem is that the indexf.sys file has been extended too many times, and it has filled its file header. INDEXF.SYS is one of the files that is not allowed to use an extension file header, so it is limited in the number of pieces the the file can be in.

If you have DFU V3.2, the following command will tell you a lot of useful info.

$ dfu indexf /anal sys$sysdevice

Disk and File Utilities for OpenVMS V3.2
%DFU-I-ANALDISK, Analyzing INDEXF and BITMAP...
%DFU-I-TOTAL, Maparea maps 135392 blocks in 4 fragments (6% used)
%DFU-I-FINDLBN, Largest free contiguous space 5104 blocks at LBN 12197656
%DFU-W-NOOPT, No optimization needed for INDEXF.SYS

%DFU-I-READY, INDEXF command ready
$

If you don't have DFU, you can at least look at the internal fragmentation of the indexf.sys file with:

$ dump/head/block=count:0 sys$sysdevice:[000000]indexf.sys

At the bottom of the report will be the retrieval pointers, each describing a contiguous piece of the file.

If you can not dismount the disk, your only option is to delete some files so the existing file headers can be reused.

Do download and install DFU as recommended by Hein.

If you can dismount the device clusterwide, and the disk has a reasonable amount of contiguous free space, then you may be able to recover by using the INDEXF defrag built into DFU V3.2.

$ dfu help indexf *

Disk and File Utilities for OpenVMS V3.2

DFU

INDEXF

Qualifiers


/ANALYZE

Displays information about the number of fragments in INDEXF.SYS
and the largest contiguous free space. If INDEXF.SYS can not be
defragmented a "%DFU-W-NOOPT" message will be displayed.

/DEFRAG

Starts a defragment operation on INDEXF.SYS. The disk must be
clusterwide dismounted before starting a defragment operation.

/EXTEND
/EXTEND=n (number of blocks)

Extends INDEXF.SYS with 'n' blocks in 1 new fragment. This is
equivalent to 'n' extra file headers. The disk must be clusterwide
dismounted before starting an extend operation.
The 'n' value will be rounded up to be a multiple of the device
cluster size.

/SHADOW_MEMBER
/SHADOW_MEMBER=device:

The physical member of the shadowset on which to perform an
EXTEND or DEFRAG operation. This qualifier is only required
in combination with /DEFRAG and /EXTEND. Example :

DFU> INDEXF/DEFRAG DSA0:/SHADOW_MEMBER=$1$DUA104:

/SHOW_POINTERS

Displays all mapping pointer information during the analyze phase.


/STATISTICS
/STATISTICS
/NOSTATISTICS (default)

Controls whether cpu and I/O consumption of the
command are shown.

/TRUNCATE

If possible the INDEXF.SYS file will be truncated back to
the end-of-file size. The disk must be dismounted to
perform this operation.


Topic? Exit
$

This is one reason for using the /headers=xxxxx switch when initializing a volume, it preextends the indexf.sys file so it will not need to be extended in small increments.

Good Luck,

Jon

[Robert Gezelter]

Megat,

If you have not installed DFU (or for some reason are reluctant to do so), the limited information needed at this instant can be obtained using the DUMP and DIRECTORY commands, to wit:

$ DIRECTORY/FULL DSA1:[000000]INDEXF.SYS

and

$ DUMP/HEADER/BLOCK=(START:0,END:0) DSA1:[000000]INDEXF.SYS

- Bob Gezelter, http://www.rlgsc.com

[Jon Pinkley]

After a bit more research, I now believe that MAXIMUM_FILES has been reached instead of the indexf.sys file header being full, since the error message for that is SYSTEM-W-HEADERFULL.

I have reproduced both types of errors using LDA devices.

If you really have 499321 files on this disk, as Hein has pointed out, there must be a large number of zero block files. (If each of these has 1 cluster allocated, that would use 499321*41 = 20472161 blocks, and only 14747330 have been used.)

So, find the zero sized files, and determine why they are being created, and delete the ones you don't need.

$ directory/size=alloc/sel=size:max:0 dsa1:[000000...]

If you really need these zero sized files, you will need to reinitialize your device with a larger value of /maximum_files and preferably a smaller /cluster_size. If you do reinitialize, I would recommend a cluster_size that is a power of 2 (i.e. 1,2,4,8,16,32,64...).

Jon

[Jon Pinkley]

Here's an example of the HEADERFULL error, and the fragmented INDEXF.SYS file.

Note the Map area words in use in the dump/header output (and the difference between Map area offset and Access control area offset).

$ cre lda3:[jon]abc.dat
%CREATE-E-OPENOUT, error opening LDA3:[JON]ABC.DAT; as output
-RMS-E-CRE, ACP file create failed
-SYSTEM-W-HEADERFULL, file header is full
$ sho dev lda3:

Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
$4$LDA3: (SIGMA) Mounted 0 TEST 8889 1 1
$ dump/header/block=count:0 lda3:[000000]indexf.sys

Dump of file LDA3:[000000]INDEXF.SYS;1 on 24-MAR-2008 22:02:38.80
File ID (1,1,0) End of file block 99 / Allocated 99

File Header

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (1,1,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Fixed
File organization: Sequential
Record attributes:
Record size: 512
Highest block: 99
End of file block: 100
End of file byte: 0
Bucket size: 0
Fixed control area size: 0
Maximum record size: 512
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics: Contiguous best try
Caching attribute: Writethrough
Map area words in use: 154
Access mode: 0
File owner UIC: [FILSYS]
File protection: S:RWED, O:RWED, G:RE, W:
Back link file identification: (4,4,0)
Journal control flags:
Active recovery units: None
File entry linkcount: 0
Highest block written: 99
Client attributes: None

Identification area
File name: INDEXF.SYS;1
Revision number: 1
Creation date: 24-MAR-2008 21:13:04.63
Revision date: 24-MAR-2008 21:13:04.63
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 3 LBN: 0
Count: 1 LBN: 5027
Count: 21 LBN: 5006
Count: 1 LBN: 9999
Count: 1 LBN: 8191
Count: 1 LBN: 8190
Count: 1 LBN: 4095
Count: 1 LBN: 4094
Count: 1 LBN: 9998
Count: 1 LBN: 8189
Count: 1 LBN: 4093
Count: 1 LBN: 9997
Count: 1 LBN: 8188
Count: 1 LBN: 4092
Count: 1 LBN: 9996
Count: 1 LBN: 8187
Count: 1 LBN: 4091
Count: 1 LBN: 9995
Count: 1 LBN: 8186
Count: 1 LBN: 4090
Count: 1 LBN: 9994
Count: 1 LBN: 8185
Count: 1 LBN: 4089
Count: 1 LBN: 9993
Count: 1 LBN: 8184
Count: 1 LBN: 4088
Count: 1 LBN: 9992
Count: 1 LBN: 8183
Count: 1 LBN: 4087
Count: 1 LBN: 9991
Count: 1 LBN: 8182
Count: 1 LBN: 4086
Count: 1 LBN: 9990
Count: 1 LBN: 8181
Count: 1 LBN: 4085
Count: 1 LBN: 9989
Count: 1 LBN: 8180
Count: 1 LBN: 4084
Count: 1 LBN: 9988
Count: 1 LBN: 8179
Count: 1 LBN: 4083
Count: 1 LBN: 9987
Count: 1 LBN: 8178
Count: 1 LBN: 4082
Count: 1 LBN: 9986
Count: 1 LBN: 8177
Count: 1 LBN: 4081
Count: 1 LBN: 9985
Count: 1 LBN: 8176
Count: 1 LBN: 4080
Count: 1 LBN: 9984
Count: 1 LBN: 8175
Count: 1 LBN: 4079
Count: 1 LBN: 9983
Count: 1 LBN: 8174
Count: 1 LBN: 4078
Count: 1 LBN: 9982
Count: 1 LBN: 8173
Count: 1 LBN: 4077
Count: 1 LBN: 9981
Count: 1 LBN: 8172
Count: 1 LBN: 4076
Count: 1 LBN: 9980
Count: 1 LBN: 8171
Count: 1 LBN: 4075
Count: 1 LBN: 9979
Count: 1 LBN: 8170
Count: 1 LBN: 4074
Count: 1 LBN: 9978
Count: 1 LBN: 8169
Count: 1 LBN: 4073
Count: 1 LBN: 9977
Count: 1 LBN: 8168
Count: 1 LBN: 4072
Count: 1 LBN: 9976
Count: 1 LBN: 8167
Count: 1 LBN: 4071

Checksum: 63907
$

[Jon Pinkley]

megat razizul,

HEADERFULL wasn't your problem, but for anyone that does get that, here is an example of fixing with DFU. This is the same disk as my last note. Note the Map area in use drops to 8 words.

$ dfu indexf /anal lda3:

Disk and File Utilities for OpenVMS V3.2
%DFU-I-ANALDISK, Analyzing INDEXF and BITMAP...
%DFU-I-TOTAL, Maparea maps 99 blocks in 77 fragments (99% used)
%DFU-I-FINDLBN, Largest free contiguous space 3133 blocks at LBN 5034
%DFU-I-MOVE, 74 blocks can be defragmented (74 fragments)

%DFU-I-READY, INDEXF command ready
$ dism lda3:
$ dfu indexf /defrag lda3:

Disk and File Utilities for OpenVMS V3.2
%DFU-I-MOUNTING, Busy mounting disk LDA3:...
%DFU-I-ANALDISK, Analyzing INDEXF and BITMAP...
%DFU-I-TOTAL, Maparea maps 99 blocks in 77 fragments (99% used)
%DFU-I-FINDLBN, Largest free contiguous space 3133 blocks at LBN 5034
%DFU-I-MOVE, 74 blocks can be defragmented (74 fragments)
Continue to modify INDEXF.SYS ? (Y/N) [N] : y
%DFU-I-MOUNTFOR, Busy remounting disk LDA3: /FOREIGN...
%DFU-I-STARTDFR, Now copying fragments to new location...
%DFU-S-COPIED, 1 blocks copied (fragment 4)
%DFU-S-COPIED, 1 blocks copied (fragment 5)
%DFU-S-COPIED, 1 blocks copied (fragment 6)
%DFU-S-COPIED, 1 blocks copied (fragment 7)
%DFU-S-COPIED, 1 blocks copied (fragment 8)
%DFU-S-COPIED, 1 blocks copied (fragment 9)
%DFU-S-COPIED, 1 blocks copied (fragment 10)
%DFU-S-COPIED, 1 blocks copied (fragment 11)
%DFU-S-COPIED, 1 blocks copied (fragment 12)
%DFU-S-COPIED, 1 blocks copied (fragment 13)
%DFU-S-COPIED, 1 blocks copied (fragment 14)
%DFU-S-COPIED, 1 blocks copied (fragment 15)
%DFU-S-COPIED, 1 blocks copied (fragment 16)
%DFU-S-COPIED, 1 blocks copied (fragment 17)
%DFU-S-COPIED, 1 blocks copied (fragment 18)
%DFU-S-COPIED, 1 blocks copied (fragment 19)
%DFU-S-COPIED, 1 blocks copied (fragment 20)
%DFU-S-COPIED, 1 blocks copied (fragment 21)
%DFU-S-COPIED, 1 blocks copied (fragment 22)
%DFU-S-COPIED, 1 blocks copied (fragment 23)
%DFU-S-COPIED, 1 blocks copied (fragment 24)
%DFU-S-COPIED, 1 blocks copied (fragment 25)
%DFU-S-COPIED, 1 blocks copied (fragment 26)
%DFU-S-COPIED, 1 blocks copied (fragment 27)
%DFU-S-COPIED, 1 blocks copied (fragment 28)
%DFU-S-COPIED, 1 blocks copied (fragment 29)
%DFU-S-COPIED, 1 blocks copied (fragment 30)
%DFU-S-COPIED, 1 blocks copied (fragment 31)
%DFU-S-COPIED, 1 blocks copied (fragment 32)
%DFU-S-COPIED, 1 blocks copied (fragment 33)
%DFU-S-COPIED, 1 blocks copied (fragment 34)
%DFU-S-COPIED, 1 blocks copied (fragment 35)
%DFU-S-COPIED, 1 blocks copied (fragment 36)
%DFU-S-COPIED, 1 blocks copied (fragment 37)
%DFU-S-COPIED, 1 blocks copied (fragment 38)
%DFU-S-COPIED, 1 blocks copied (fragment 39)
%DFU-S-COPIED, 1 blocks copied (fragment 40)
%DFU-S-COPIED, 1 blocks copied (fragment 41)
%DFU-S-COPIED, 1 blocks copied (fragment 42)
%DFU-S-COPIED, 1 blocks copied (fragment 43)
%DFU-S-COPIED, 1 blocks copied (fragment 44)
%DFU-S-COPIED, 1 blocks copied (fragment 45)
%DFU-S-COPIED, 1 blocks copied (fragment 46)
%DFU-S-COPIED, 1 blocks copied (fragment 47)
%DFU-S-COPIED, 1 blocks copied (fragment 48)
%DFU-S-COPIED, 1 blocks copied (fragment 49)
%DFU-S-COPIED, 1 blocks copied (fragment 50)
%DFU-S-COPIED, 1 blocks copied (fragment 51)
%DFU-S-COPIED, 1 blocks copied (fragment 52)
%DFU-S-COPIED, 1 blocks copied (fragment 53)
%DFU-S-COPIED, 1 blocks copied (fragment 54)
%DFU-S-COPIED, 1 blocks copied (fragment 55)
%DFU-S-COPIED, 1 blocks copied (fragment 56)
%DFU-S-COPIED, 1 blocks copied (fragment 57)
%DFU-S-COPIED, 1 blocks copied (fragment 58)
%DFU-S-COPIED, 1 blocks copied (fragment 59)
%DFU-S-COPIED, 1 blocks copied (fragment 60)
%DFU-S-COPIED, 1 blocks copied (fragment 61)
%DFU-S-COPIED, 1 blocks copied (fragment 62)
%DFU-S-COPIED, 1 blocks copied (fragment 63)
%DFU-S-COPIED, 1 blocks copied (fragment 64)
%DFU-S-COPIED, 1 blocks copied (fragment 65)
%DFU-S-COPIED, 1 blocks copied (fragment 66)
%DFU-S-COPIED, 1 blocks copied (fragment 67)
%DFU-S-COPIED, 1 blocks copied (fragment 68)
%DFU-S-COPIED, 1 blocks copied (fragment 69)
%DFU-S-COPIED, 1 blocks copied (fragment 70)
%DFU-S-COPIED, 1 blocks copied (fragment 71)
%DFU-S-COPIED, 1 blocks copied (fragment 72)
%DFU-S-COPIED, 1 blocks copied (fragment 73)
%DFU-S-COPIED, 1 blocks copied (fragment 74)
%DFU-S-COPIED, 1 blocks copied (fragment 75)
%DFU-S-COPIED, 1 blocks copied (fragment 76)
%DFU-S-COPIED, 1 blocks copied (fragment 77)
%DFU-I-NEWTOTAL, New Maparea maps 99 blocks in 4 fragments
%DFU-S-REWRTIF, INDEXF.SYS File header rewritten !
%DFU-I-RBDBITMAP, Updating BITMAP.SYS...
%DFU-S-DONE, all operations succesfully completed

%DFU-I-DISMNT, Volume dismounted

%DFU-I-READY, INDEXF command ready
$ mou/ov=id lda3:
%MOUNT-I-MOUNTED, TEST mounted on _$4$LDA3: (SIGMA)
$ dump/header/block=count:0 lda3:[000000]indexf.sys

Dump of file LDA3:[000000]INDEXF.SYS;1 on 24-MAR-2008 22:14:33.99
File ID (1,1,0) End of file block 99 / Allocated 99

File Header

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (1,1,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Fixed
File organization: Sequential
Record attributes:
Record size: 512
Highest block: 99
End of file block: 100
End of file byte: 0
Bucket size: 0
Fixed control area size: 0
Maximum record size: 512
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics: Contiguous best try
Caching attribute: Writethrough
Map area words in use: 8
Access mode: 0
File owner UIC: [FILSYS]
File protection: S:RWED, O:RWED, G:RE, W:
Back link file identification: (4,4,0)
Journal control flags:
Active recovery units: None
File entry linkcount: 0
Highest block written: 99
Client attributes: None

Identification area
File name: INDEXF.SYS;1
Revision number: 1
Creation date: 24-MAR-2008 21:13:04.63
Revision date: 24-MAR-2008 21:13:04.63
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 3 LBN: 0
Count: 1 LBN: 5027
Count: 21 LBN: 5006
Count: 74 LBN: 5034

Checksum: 30522
$

[Hakan Zanderau ( Anders]

Hi,

Made a little test using LogicalDisk.
Created a 10000 block disk.

When I reached the limit of "maximum files" I got:

%CREATE-E-OPENOUT, error opening $1$LDA200:TEST.TXT; as output
-RMS-E-CRE, ACP file create failed
-SYSTEM-W-IDXFILEFULL, index file is full

I made a new INIT with /MAXIMUM=nnn and got the same result when reaching the limit.

regards,

Hakan Zanderau
HA-solutions

[Hakan Zanderau ( Anders]

Sorry, didn't notice that Jon already reproduced the symptoms/errors.....

[Hein van den Heuvel]

I wrote "I don't think this problem is caused by /maxfiles being exhausted."

Sorry to have send you all on a wild goose chase. But I now think (helped with a scan of the f11x listings), that is has to be maxfiles after all.

I incorrectly recalled some several incident with unexpected trouble with indexf.sys expansion back in the early 90s.
But that was well before 7.3.
Thanks Jon for verifying/reproducing.

Still, the numbers don't seem to work for that, but maxfiles has to be the explanation, or the disk structure is broken ($ANALYZE/DISK?)

So my money is on lots of empty files causing trouble with the file headers being exhausted before the disk was full.

Hein.

[Jon Pinkley]

Just curious: Is anyone is aware of a free tool to increase /MAXIMUM_FILES without a reinitialization of the disk. I really think this functionality should be part of stock VMS, especially since we can now expand the size of a volume. I am a bit surprised that DFU doesn't allow this, as it seems that it would be relatively straightforward since it already knows how to rewrite the INDEXF.SYS file.

This is an outline of what I think would be involved in modifying DFU to allow this to be done in "offline" mode. That is still preferable to needing to reinitialize the volume and a backup/image/noinit from the old to the new disk, at least as a "quick fix". It wouldn't be able to work in every case, for example if the target volume was extremely fragmented, and the largest free extent is not large enough for the new index file bitmap plus the first 16 headers, as these must be in a contiguous extent.

First, the indexf.sys file should be checked to verify there is sufficient space in the map area for the largest retrieval pointer possible (FM2$C_FORMAT3, 64 bits or 4 16 bit words). If there is not sufficient map area, then the indexf.sys file will need to be defragmented first. The next step is determining the size needed for the new index file bitmap, using the formula int((new_maxfiles+4095)/4096) finding a contiguous piece big enough for that + 16 file headers, copying HM2$W_IBMAPSIZE-1 blocks starting at HM2$L_IBMAPLBN to the start of the new location. The block at HM2$W_IBMAPLBN + HM2$W_IBMAPSIZE - 1 may only be partially used, so all bits representing file headers > the original HM2$L_MAXFILES must be cleared, to indicate they are available for use. This block should then be written to the next available LBN in the new location. The remaining new blocks for the new index file bitmap would then need to be initialized (I haven't looked at what initialized does for the unused bits in the last block of the index file bitmap, but this should do the same thing as initialize does). The first 16 file headers need to be copied to the end of the new allocation (more could be copied, and the indexf.sys file defragmented at the same time, but it is not a requirement). Then the retrieval pointers describing the indexf.sys files itself need to be recomputed, adding the new extent and possibly removing or modifying the one that used to map the index file bitmap and the first 16 headers. Then the header checksum needs to be recomputed for the index file header, and the new modified index file header needs to be written to the primary and backup copies. Then the homeblock needs to have the HM2$L_MAXFILES, HM2$L_IBMAPLBN and HM2$W_IBMAPSIZE and homeblock checksums recomputed, then the homeblock and backup homeblocks need to be written. I am probably missing a few things... but it is doable, and in my opinion should be easier than the DFU INDEXF /DEFRAG was.

Jon

[Guenther Froehlin]

Not easy! MAXIMUM_FILES sizes the bitmap for file header slots used in INDEXF.SYS. This bitmap occupies some virtual blocks at the low part of INDEXF.SYS. Increasing this area would require quite some shuffling within the current virtual block layout of the current INDEXF.SYS file.

/Guenther

[Jon Pinkley]

Guenther Froehlin,

Maybe I used the wrong term for the difficulty of implementing this. However, DFU can defrag the indexf.sys file, and does this while the disk is mounted /foreign. If it can do that, it seems it already has a lot of the underlying routines available to do many of the things required to change the size of the index file bitmap.

Did you see errors in the proposed method of doing this?

BTW, the proposed solution was based on info in chapter 2 of "VMS File System Internals" Kirby McCoy 1990, chapter 2 "Files-11 On-Disk Structure", so there may be some changes that I didn't take into account.

I wasn't suggesting an "online" fix.

Jon

[Hein van den Heuvel]

Hmm, it might not be too tricky.

It would have to shuffle the whole indexf.sys up to create space, but that's just work.

The good news is that relative addressing is used. So an 'early' file like the MFD (000000.DIR) which has fid 4,4,0 is and would still be at VBN block: 4 * cluster + int((maxfiles+ 4095) / 4096)

Unlike say and RMS indexed file I do not think the VBN for any header block is stored in a header. Just ID's, which are relative.

Try it on an MD or LD device!
See if you can keep DFU and ANALYZE/DISK happy.

Cheers,
Hein.

[Jon Pinkley]

RE:"Did you see errors in the proposed method of doing this?"

Well I see I left out one important step: updating the BITMAP.SYS file to account for the changes to LBNs associated with the portions of the INDEXF.SYS file that were moved.

RE:"It would have to shuffle the whole indexf.sys up to create space, but that's just work."

We don't have to shuffle all the blocks in the indexf.sys file, just the retrieval pointers. The minimum that would need to be moved would be the LBNS (and any adjacent LBNs in the clusters involved) that are part of the index file bitmap and the first 16 file headers. The rest of the LBNS associated with indexf.sys would not have to move, although it may be necessary to shuffle the retrieval pointers in the map area. In this respect, it is similar to inserting something into a linked list, we are not required to move all the elements following the part we are removing or inserting. Of course, it might be nice to just go ahead and defrag the indexf.sys file while this is being done, but that isn't a requirement unless it is already too fragmented to allow for the new retrieval pointer.

Oh, and if anyone is interested in how I created such a fragmented indexf.sys, I am attaching the command procedure I used. It is only a template, since it has LDA3 hardcoded, but it shows one way to get a really fragmented indexf.sys file.

Basic method: initialize with small /header value and cluster of 1, create directory for files, allocate all available space less one block, create empty files until error, increment extent count, delete filler file, redo until more extensions have occurred than single file will hold retrieval pointers for, dump info about fragmented state and exit.

Jon

[megat razizul]

Guys
Thanks a lot for your respond for this information. Right know Iâ m trying to install DFU on VMS. Anyway, this is the output when I execute this command â DIRECTORY/FULL DSA1:[000000]INDEXF.SYSâ
Total of 483404 files, 0 blocks.

Directory DSA1:[SYS0.SYS$STARTUP]

TCPIP$SHUTDOWN_LIST.DAT;1
0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.CDE$DEFAULTS.SYSTEM.APPCONFIG.APPMANAGER.C.DECWINDOWS_APPS]

DXCARDFILER.;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.COMPAQ.NODES.HSIBD1.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;51 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.HP.NODES.HSIBD3.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;53 0
LOG.;52 0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.SYS$LDR]

VMS$SYSTEM_IMAGES.DATA;1
0
VMS$SYSTEM_IMAGES.TEMPLATE;1
0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.SYSEXE]

CLU$SYNCH.DAT;1 0
UCX$SERVICE.DAT;1 0
UCX$UCP.EXE;1 0

Total of 3 files, 0 blocks.

Directory DSA1:[SYS0.SYSEXE]

TEST.DAT;1 0
UCX$MGT_THRESHOLDS.BAK;391
0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSMGR]

MOUNT_DISK.COM;1 0
MOUNT_DISK_ORI.COM;1
0
TCPIP$V51_CONVERSION.FLG;1
0
TPU$WORK.TPU$WORK;1
0

Total of 4 files, 0 blocks.

Directory DSA1:[SYS0.SYSMGR.DT.TRASH]

.TRASHINFO;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.TCPIP$LPD]

TCPIP$TELNETSYM_HSIPRT014.LOG;3
0
TCPIP$TELNETSYM_HSIPRT109.LOG;3
0
TCPIP$TELNETSYM_TESTPRINT.LOG;3
0

Total of 3 files, 0 blocks.

Directory DSA1:[SYS0.TCPIP$XDM.WORK]

10_10_4_250_0.OUT;2
0
10_72_101_10_0.ERR;1
0
10_72_101_10_0.OUT;1
0
16_170_181_17_0.ERR;2
0
16_170_181_17_0.OUT;2
0
172_16_1_10_0.OUT;2
0
172_16_1_3_0.OUT;44
0
172_16_1_43_0.ERR;4
0
172_16_1_43_0.OUT;4
0
219_95_69_138_0.OUT;1
0
CONSOLE_HSI_HICT_GOV_MY_0.ERR;9
0
CONSOLE_HSI_HICT_GOV_MY_0.OUT;9
0
HSI-PC0035_HSI_HICT_GOV_MY_0.OUT;1
0
LBL3256_NORTHAMERICA_CERNER_NET_0.OUT;1
0
RUFUS_COMPAQ_COM_0.OUT;633
0

Total of 15 files, 0 blocks.

Directory DSA1:[SYS0.WBEM.WEB.IM]

CERTMM.PEM;1 0
CLIENTCA.PEM;10 0
FILEMM.PEM;1 0

Total of 3 files, 0 blocks.

Directory DSA1:[TCPIP$SNMP]

TCPIP$HR_MIB.ERR;5 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.CDE$DEFAULTS.SYSTEM.APPCONFIG.APPMANAGER.C.DECWINDOWS_APPS]

DXCARDFILER.;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.COMPAQ.NODES.HSIBD1.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;51 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.HP.NODES.HSIBD3.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;53 0
LOG.;52 0

Total of 2 files, 0 blocks.

Directory DSA1:[VMS$COMMON.SYS$LDR]

VMS$SYSTEM_IMAGES.DATA;1
0
VMS$SYSTEM_IMAGES.TEMPLATE;1
0

Total of 2 files, 0 blocks.

Directory DSA1:[VMS$COMMON.SYSEXE]

CLU$SYNCH.DAT;1 0
UCX$SERVICE.DAT;1 0
UCX$UCP.EXE;1 0

Total of 3 files, 0 blocks.



Output for â $ DUMP/HEADER/BLOCK=(START:0,END:0) DSA1:[000000]INDEXF.SYSâ

Dump of file DSA1:[000000]INDEXF.SYS;1 on 28-MAR-2008 11:10:04.39
File ID (1,1,0) End of file block 499626 / Allocated 499626

File Header

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (1,1,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Fixed
File organization: Sequential
Record attributes:
Record size: 512
Highest block: 499626
End of file block: 499627
End of file byte: 0
Bucket size: 0
Fixed control area size: 0
Maximum record size: 512
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics: Contiguous best try
Caching attribute: Writethrough
Map area words in use: 72
Access mode: 0
File owner UIC: [1,1]
File protection: S:RWED, O:RWED, G:RE, W:
Back link file identification: (4,4,0)
Journal control flags:
Active recovery units: None
File entry linkcount: 0
Highest block written: 499626
Client attributes: None

Identification area
File name: INDEXF.SYS;1
Revision number: 503
Creation date: 18-JUN-2003 14:32:48.57
Revision date: 16-MAR-2008 18:51:57.28
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 82 LBN: 0
Count: 41 LBN: 1025
Count: 41 LBN: 20972853
Count: 164 LBN: 20971828
Count: 74907 LBN: 8446
Count: 63673 LBN: 6666272
Count: 172487 LBN: 6749666
Count: 72693 LBN: 6970410
Count: 17343 LBN: 5116923



Dump of file DSA1:[000000]INDEXF.SYS;1 on 28-MAR-2008 11:10:04.39
File ID (1,1,0) End of file block 499626 / Allocated 499626

Count: 14760 LBN: 5501175
Count: 12546 LBN: 7932106
Count: 10660 LBN: 6925966
Count: 9061 LBN: 6938143
Count: 7708 LBN: 7916690
Count: 5002 LBN: 4129151
Count: 5002 LBN: 4138417
Count: 5002 LBN: 4147232
Count: 5002 LBN: 5983827
Count: 5002 LBN: 5989034
Count: 5002 LBN: 5994405
Count: 5002 LBN: 5999448
Count: 5002 LBN: 6005311
Count: 3444 LBN: 6010805

Checksum: 757

[Hein van den Heuvel]

>> Maximum files allowed 499321
:

:
>> Highest block written: 499626

So indeed, the maximum number of file headers is in use.

fwiw... the 'count:' in the LBN map adds up:
$ perl -ne "$t+=$1 if /nt: (\d+)/}{print $t" tmp.tmp
499626

>> trying to install DFU on VMS.
How long could that take? Should be seconds. Let us know if you have trouble installing/using.

When you get it going. Use the REPORT command. Look for the lines:
"Header count :"
"Free headers :"
"Empty files :"

>> Total of 483404 files, 0 blocks.

Looks more like $DIRECTORY/SIZE output to me.

But those '0' blocks everywhere are very odd!?
Can you dismount/remount or reboot if need be?
$ANALYZE/DISK ??

Hein.

[Jon Pinkley]

megat,

Use the following command:

$ directory/size=alloc/sel=size:max:0 dsa1:[000000...]/total /output=sys$scratch:dsa1_files.lis

Then edit sys$scratch:dsa1_files.lis and find the directory that had the 480000+ empty files. Look in that directory and determine what they are, and why they are being created. They are your problem. Delete the ones you don't need; that will solve your immedate problem. Find and eliminate what is creating them to prevent the problem from reoccurring.

We now know for sure that running out or mapping pointers for indexf.sys was not the problem.


Map area offset: 100
Access control area offset: 255
Map area words in use: 72
Map area
Retrieval pointers
Count: 82 LBN: 0
Count: 41 LBN: 1025
Count: 41 LBN: 20972853
Count: 164 LBN: 20971828
Count: 74907 LBN: 8446
Count: 63673 LBN: 6666272
Count: 172487 LBN: 6749666
Count: 72693 LBN: 6970410
Count: 17343 LBN: 5116923
Count: 14760 LBN: 5501175
Count: 12546 LBN: 7932106
Count: 10660 LBN: 6925966
Count: 9061 LBN: 6938143
Count: 7708 LBN: 7916690
Count: 5002 LBN: 4129151
Count: 5002 LBN: 4138417
Count: 5002 LBN: 4147232
Count: 5002 LBN: 5983827
Count: 5002 LBN: 5989034
Count: 5002 LBN: 5994405
Count: 5002 LBN: 5999448
Count: 5002 LBN: 6005311
Count: 3444 LBN: 6010805

But it does appear that your disk is getting fragmented, since the extents are getting smaller and smaller, with the last one being only 3444 blocks.

Jon

[Hein van den Heuvel]

Jon wrote> "But it does appear that your disk is getting fragmented, since the extents are getting smaller and smaller, with the last one being only 3444 blocks."

Hmmm maybe not. Here is what the XFC does:

" extend the file by 15% of the remaining available header space, until 90% of the
maximum headers have been allocated. After revert to allocating 1% of MAXFILES each time we invoke the routine until we reach MAXFILES."

The code also limits the extend to 20% of the free space

The 8 * 5002 are the under 10% left allocations of 1% rounded up to a cluster.
The 3444 is the remainder.

:-)

Hein.

[Jon Pinkley]

Hein is absolutely correct. The XQP (that's what Hein meant) would have limited the extents' sizes to what is seen even if there were large contiguous chunks of free space.

Reference: [V732.F11X.LIS]EXTIDX.LIS (listing file from [F11X_3.SRC]EXTIDX.B32)

Thanks for the correction.

Jon

[megat razizul]

Hi

I canâ t install DFU since it point to same logical disk at indext.sys. Any solution to solve it? The error massage as below


The following product will be installed to destination:
HP VMS DFU V3.2 DISK$ALPHASYS:[VMS$COMMON.]

Portion done: 0%...10%

%PCSI-E-OPENOUT, error opening DISK$ALPHASYS:[SYS0.SYSCOMMON.][SYSEXE]DFU.EXE; as output
-RMS-E-CRE, ACP file create failed
-SYSTEM-W-IDXFILEFULL, index file is full
%PCSI-E-OPFAILED, operation failed
Terminating is strongly recommended. Do you want to terminate? [YES] y
%PCSI-E-CANCEL_WIP, termination resulted in an incomplete modification to the system
%PCSI-E-S_OPCAN, operation cancelled by request
%PCSIUI-E-ABORT, operation terminated due to an unrecoverable error condition

[Martin Vorlaender]

You can install DFU to an alternate disk/directory by using

$ PRODUCT INSTALL DFU /DESTINATION=disk:[dir]

HTH,
Martin