Operating System - OpenVMS
1748201 Members
3167 Online
108759 Solutions
New Discussion юеВ

Re: unable to create file on the disk

 
Hakan Zanderau ( Anders
Trusted Contributor

Re: unable to create file on the disk

Hi,

Made a little test using LogicalDisk.
Created a 10000 block disk.

When I reached the limit of "maximum files" I got:

%CREATE-E-OPENOUT, error opening $1$LDA200:TEST.TXT; as output
-RMS-E-CRE, ACP file create failed
-SYSTEM-W-IDXFILEFULL, index file is full

I made a new INIT with /MAXIMUM=nnn and got the same result when reaching the limit.

regards,

Hakan Zanderau
HA-solutions
Don't make it worse by guessing.........
Hakan Zanderau ( Anders
Trusted Contributor

Re: unable to create file on the disk

Sorry, didn't notice that Jon already reproduced the symptoms/errors.....
Don't make it worse by guessing.........
Hein van den Heuvel
Honored Contributor

Re: unable to create file on the disk

I wrote "I don't think this problem is caused by /maxfiles being exhausted."

Sorry to have send you all on a wild goose chase. But I now think (helped with a scan of the f11x listings), that is has to be maxfiles after all.

I incorrectly recalled some several incident with unexpected trouble with indexf.sys expansion back in the early 90s.
But that was well before 7.3.
Thanks Jon for verifying/reproducing.

Still, the numbers don't seem to work for that, but maxfiles has to be the explanation, or the disk structure is broken ($ANALYZE/DISK?)

So my money is on lots of empty files causing trouble with the file headers being exhausted before the disk was full.

Hein.
Jon Pinkley
Honored Contributor

Re: unable to create file on the disk

Just curious: Is anyone is aware of a free tool to increase /MAXIMUM_FILES without a reinitialization of the disk. I really think this functionality should be part of stock VMS, especially since we can now expand the size of a volume. I am a bit surprised that DFU doesn't allow this, as it seems that it would be relatively straightforward since it already knows how to rewrite the INDEXF.SYS file.

This is an outline of what I think would be involved in modifying DFU to allow this to be done in "offline" mode. That is still preferable to needing to reinitialize the volume and a backup/image/noinit from the old to the new disk, at least as a "quick fix". It wouldn't be able to work in every case, for example if the target volume was extremely fragmented, and the largest free extent is not large enough for the new index file bitmap plus the first 16 headers, as these must be in a contiguous extent.

First, the indexf.sys file should be checked to verify there is sufficient space in the map area for the largest retrieval pointer possible (FM2$C_FORMAT3, 64 bits or 4 16 bit words). If there is not sufficient map area, then the indexf.sys file will need to be defragmented first. The next step is determining the size needed for the new index file bitmap, using the formula int((new_maxfiles+4095)/4096) finding a contiguous piece big enough for that + 16 file headers, copying HM2$W_IBMAPSIZE-1 blocks starting at HM2$L_IBMAPLBN to the start of the new location. The block at HM2$W_IBMAPLBN + HM2$W_IBMAPSIZE - 1 may only be partially used, so all bits representing file headers > the original HM2$L_MAXFILES must be cleared, to indicate they are available for use. This block should then be written to the next available LBN in the new location. The remaining new blocks for the new index file bitmap would then need to be initialized (I haven't looked at what initialized does for the unused bits in the last block of the index file bitmap, but this should do the same thing as initialize does). The first 16 file headers need to be copied to the end of the new allocation (more could be copied, and the indexf.sys file defragmented at the same time, but it is not a requirement). Then the retrieval pointers describing the indexf.sys files itself need to be recomputed, adding the new extent and possibly removing or modifying the one that used to map the index file bitmap and the first 16 headers. Then the header checksum needs to be recomputed for the index file header, and the new modified index file header needs to be written to the primary and backup copies. Then the homeblock needs to have the HM2$L_MAXFILES, HM2$L_IBMAPLBN and HM2$W_IBMAPSIZE and homeblock checksums recomputed, then the homeblock and backup homeblocks need to be written. I am probably missing a few things... but it is doable, and in my opinion should be easier than the DFU INDEXF /DEFRAG was.

Jon
it depends
Guenther Froehlin
Valued Contributor

Re: unable to create file on the disk

Not easy! MAXIMUM_FILES sizes the bitmap for file header slots used in INDEXF.SYS. This bitmap occupies some virtual blocks at the low part of INDEXF.SYS. Increasing this area would require quite some shuffling within the current virtual block layout of the current INDEXF.SYS file.

/Guenther
Jon Pinkley
Honored Contributor

Re: unable to create file on the disk

Guenther Froehlin,

Maybe I used the wrong term for the difficulty of implementing this. However, DFU can defrag the indexf.sys file, and does this while the disk is mounted /foreign. If it can do that, it seems it already has a lot of the underlying routines available to do many of the things required to change the size of the index file bitmap.

Did you see errors in the proposed method of doing this?

BTW, the proposed solution was based on info in chapter 2 of "VMS File System Internals" Kirby McCoy 1990, chapter 2 "Files-11 On-Disk Structure", so there may be some changes that I didn't take into account.

I wasn't suggesting an "online" fix.

Jon
it depends
Hein van den Heuvel
Honored Contributor

Re: unable to create file on the disk

Hmm, it might not be too tricky.

It would have to shuffle the whole indexf.sys up to create space, but that's just work.

The good news is that relative addressing is used. So an 'early' file like the MFD (000000.DIR) which has fid 4,4,0 is and would still be at VBN block: 4 * cluster + int((maxfiles+ 4095) / 4096)

Unlike say and RMS indexed file I do not think the VBN for any header block is stored in a header. Just ID's, which are relative.

Try it on an MD or LD device!
See if you can keep DFU and ANALYZE/DISK happy.

Cheers,
Hein.
Jon Pinkley
Honored Contributor

Re: unable to create file on the disk

RE:"Did you see errors in the proposed method of doing this?"

Well I see I left out one important step: updating the BITMAP.SYS file to account for the changes to LBNs associated with the portions of the INDEXF.SYS file that were moved.

RE:"It would have to shuffle the whole indexf.sys up to create space, but that's just work."

We don't have to shuffle all the blocks in the indexf.sys file, just the retrieval pointers. The minimum that would need to be moved would be the LBNS (and any adjacent LBNs in the clusters involved) that are part of the index file bitmap and the first 16 file headers. The rest of the LBNS associated with indexf.sys would not have to move, although it may be necessary to shuffle the retrieval pointers in the map area. In this respect, it is similar to inserting something into a linked list, we are not required to move all the elements following the part we are removing or inserting. Of course, it might be nice to just go ahead and defrag the indexf.sys file while this is being done, but that isn't a requirement unless it is already too fragmented to allow for the new retrieval pointer.

Oh, and if anyone is interested in how I created such a fragmented indexf.sys, I am attaching the command procedure I used. It is only a template, since it has LDA3 hardcoded, but it shows one way to get a really fragmented indexf.sys file.

Basic method: initialize with small /header value and cluster of 1, create directory for files, allocate all available space less one block, create empty files until error, increment extent count, delete filler file, redo until more extensions have occurred than single file will hold retrieval pointers for, dump info about fragmented state and exit.

Jon
it depends
megat razizul
Occasional Advisor

Re: unable to create file on the disk

Guys
Thanks a lot for your respond for this information. Right know I├в m trying to install DFU on VMS. Anyway, this is the output when I execute this command ├в DIRECTORY/FULL DSA1:[000000]INDEXF.SYS├в
Total of 483404 files, 0 blocks.

Directory DSA1:[SYS0.SYS$STARTUP]

TCPIP$SHUTDOWN_LIST.DAT;1
0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.CDE$DEFAULTS.SYSTEM.APPCONFIG.APPMANAGER.C.DECWINDOWS_APPS]

DXCARDFILER.;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.COMPAQ.NODES.HSIBD1.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;51 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.HP.NODES.HSIBD3.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;53 0
LOG.;52 0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.SYS$LDR]

VMS$SYSTEM_IMAGES.DATA;1
0
VMS$SYSTEM_IMAGES.TEMPLATE;1
0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSCOMMON.SYSEXE]

CLU$SYNCH.DAT;1 0
UCX$SERVICE.DAT;1 0
UCX$UCP.EXE;1 0

Total of 3 files, 0 blocks.

Directory DSA1:[SYS0.SYSEXE]

TEST.DAT;1 0
UCX$MGT_THRESHOLDS.BAK;391
0

Total of 2 files, 0 blocks.

Directory DSA1:[SYS0.SYSMGR]

MOUNT_DISK.COM;1 0
MOUNT_DISK_ORI.COM;1
0
TCPIP$V51_CONVERSION.FLG;1
0
TPU$WORK.TPU$WORK;1
0

Total of 4 files, 0 blocks.

Directory DSA1:[SYS0.SYSMGR.DT.TRASH]

.TRASHINFO;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[SYS0.TCPIP$LPD]

TCPIP$TELNETSYM_HSIPRT014.LOG;3
0
TCPIP$TELNETSYM_HSIPRT109.LOG;3
0
TCPIP$TELNETSYM_TESTPRINT.LOG;3
0

Total of 3 files, 0 blocks.

Directory DSA1:[SYS0.TCPIP$XDM.WORK]

10_10_4_250_0.OUT;2
0
10_72_101_10_0.ERR;1
0
10_72_101_10_0.OUT;1
0
16_170_181_17_0.ERR;2
0
16_170_181_17_0.OUT;2
0
172_16_1_10_0.OUT;2
0
172_16_1_3_0.OUT;44
0
172_16_1_43_0.ERR;4
0
172_16_1_43_0.OUT;4
0
219_95_69_138_0.OUT;1
0
CONSOLE_HSI_HICT_GOV_MY_0.ERR;9
0
CONSOLE_HSI_HICT_GOV_MY_0.OUT;9
0
HSI-PC0035_HSI_HICT_GOV_MY_0.OUT;1
0
LBL3256_NORTHAMERICA_CERNER_NET_0.OUT;1
0
RUFUS_COMPAQ_COM_0.OUT;633
0

Total of 15 files, 0 blocks.

Directory DSA1:[SYS0.WBEM.WEB.IM]

CERTMM.PEM;1 0
CLIENTCA.PEM;10 0
FILEMM.PEM;1 0

Total of 3 files, 0 blocks.

Directory DSA1:[TCPIP$SNMP]

TCPIP$HR_MIB.ERR;5 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.CDE$DEFAULTS.SYSTEM.APPCONFIG.APPMANAGER.C.DECWINDOWS_APPS]

DXCARDFILER.;1 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.COMPAQ.NODES.HSIBD1.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;51 0

Total of 1 file, 0 blocks.

Directory DSA1:[VMS$COMMON.HP.NODES.HSIBD3.SVCTOOLS.SPECIFIC.CA.DATA.AUTO]

LOG.;53 0
LOG.;52 0

Total of 2 files, 0 blocks.

Directory DSA1:[VMS$COMMON.SYS$LDR]

VMS$SYSTEM_IMAGES.DATA;1
0
VMS$SYSTEM_IMAGES.TEMPLATE;1
0

Total of 2 files, 0 blocks.

Directory DSA1:[VMS$COMMON.SYSEXE]

CLU$SYNCH.DAT;1 0
UCX$SERVICE.DAT;1 0
UCX$UCP.EXE;1 0

Total of 3 files, 0 blocks.



Output for ├в $ DUMP/HEADER/BLOCK=(START:0,END:0) DSA1:[000000]INDEXF.SYS├в

Dump of file DSA1:[000000]INDEXF.SYS;1 on 28-MAR-2008 11:10:04.39
File ID (1,1,0) End of file block 499626 / Allocated 499626

File Header

Header area
Identification area offset: 40
Map area offset: 100
Access control area offset: 255
Reserved area offset: 255
Extension segment number: 0
Structure level and version: 2, 1
File identification: (1,1,0)
Extension file identification: (0,0,0)
VAX-11 RMS attributes
Record type: Fixed
File organization: Sequential
Record attributes:
Record size: 512
Highest block: 499626
End of file block: 499627
End of file byte: 0
Bucket size: 0
Fixed control area size: 0
Maximum record size: 512
Default extension size: 0
Global buffer count: 0
Directory version limit: 0
File characteristics: Contiguous best try
Caching attribute: Writethrough
Map area words in use: 72
Access mode: 0
File owner UIC: [1,1]
File protection: S:RWED, O:RWED, G:RE, W:
Back link file identification: (4,4,0)
Journal control flags:
Active recovery units: None
File entry linkcount: 0
Highest block written: 499626
Client attributes: None

Identification area
File name: INDEXF.SYS;1
Revision number: 503
Creation date: 18-JUN-2003 14:32:48.57
Revision date: 16-MAR-2008 18:51:57.28
Expiration date:
Backup date:

Map area
Retrieval pointers
Count: 82 LBN: 0
Count: 41 LBN: 1025
Count: 41 LBN: 20972853
Count: 164 LBN: 20971828
Count: 74907 LBN: 8446
Count: 63673 LBN: 6666272
Count: 172487 LBN: 6749666
Count: 72693 LBN: 6970410
Count: 17343 LBN: 5116923



Dump of file DSA1:[000000]INDEXF.SYS;1 on 28-MAR-2008 11:10:04.39
File ID (1,1,0) End of file block 499626 / Allocated 499626

Count: 14760 LBN: 5501175
Count: 12546 LBN: 7932106
Count: 10660 LBN: 6925966
Count: 9061 LBN: 6938143
Count: 7708 LBN: 7916690
Count: 5002 LBN: 4129151
Count: 5002 LBN: 4138417
Count: 5002 LBN: 4147232
Count: 5002 LBN: 5983827
Count: 5002 LBN: 5989034
Count: 5002 LBN: 5994405
Count: 5002 LBN: 5999448
Count: 5002 LBN: 6005311
Count: 3444 LBN: 6010805

Checksum: 757
Hein van den Heuvel
Honored Contributor

Re: unable to create file on the disk

>> Maximum files allowed 499321
:

:
>> Highest block written: 499626

So indeed, the maximum number of file headers is in use.

fwiw... the 'count:' in the LBN map adds up:
$ perl -ne "$t+=$1 if /nt: (\d+)/}{print $t" tmp.tmp
499626

>> trying to install DFU on VMS.
How long could that take? Should be seconds. Let us know if you have trouble installing/using.

When you get it going. Use the REPORT command. Look for the lines:
"Header count :"
"Free headers :"
"Empty files :"

>> Total of 483404 files, 0 blocks.

Looks more like $DIRECTORY/SIZE output to me.

But those '0' blocks everywhere are very odd!?
Can you dismount/remount or reboot if need be?
$ANALYZE/DISK ??

Hein.