Reclaim disk space

[rison so_1]

After deleting a 16M block file, the 16M block is not release as free disk space.

Had a look at "Show device /file" and found no process with empty file name.

Any idea ?

[Joseph Huber_1]

Is the disk cluster mounted ? Maybe another cluster-member has the file open. In this case find it out by doing show dev/file from within SYSMAN set to environment/cluster.

Otherwise do
$ set volume/rebuild device, and if this does not reclaim free space,
$ analyze/disk device
to see if the file is lost or marked for delete,
then eventually followed by
$ analyze/disk/repair

[Hein van den Heuvel]

Welcome to the OpenVMS Forum.

I think the starting point should be that VMS is working properly, that the space is being released (because 25 years experience tells us so), but that you need help understanding why is does not appear to be released.

To help with that, we would first need to know what command you use to determine the space is released or not. SHOW DEV ?

Did you delete the file because free space was near 0? Then some process may have been 'lurking' to gobble up whatever space became released. (Some log file gone wild?).

As asked, could it be that the file is still open on an other node? SHOW DEV/FIL (on each node) would be a good way to tell, but it would not show an 'empty file name'.
(What exactly do you mean by that?)

Is there (heavy) concurrent access going on?

Just in case there really is something suspect with VMS, which version are you running on what box? Patches? Special software like a defragger running?

hth,
Hein.

[Willem Grooters]

IIRC, if a file is opened for WRITE by another process, would it not be impossible to delete the file in the first place? And AFAIK, "process" is in that contaxt be read as "Process in the cluster". But I may be mislead by wishfull thinking ;-)
Anyway - if it is possible to delete the file, it would be "marked for deletion" and actually be deleted not earlier than there are no channels opened to the file. Having said that, it is feasable the space is not yet reclaimed.
It could be that is the file is INSTALLED, the system keeps a channel open so would need to INSTALL REMOVE the file first.
Another possibility is that a program stalls IO's to disk because the disk does not contain sufficient free space, and now this limitation is gone, it will shower them all at once, taking all freed space. In the past, I've seen similar behaviour by OPCOM writing to operator terminals that were stalled; once the reason for stall was removed, all messages were purged to the device; I can imagine an image to do the same.

SHOW DEVICE/FILES will show all files that have a channel open; no process name means the system has a channel open (typically INSTALLED files), so if the file shows up as such, INSTALL REMOVE might do the trick (as said)

[Mike McKinney]

You didn't say what the file was...

This happens frequently when you do an install - the old image remains in memory (and is mapped to the drive) until reboot. Also, if you were to delete a DCL procedure which is opened by a batch, the file will appear deleted, but will still be there. The easiest way to determine if this is the case is to do an $analyze/disk_structure/repair $device:

If the file shows up as in the output as a result, then you know that it's locked by a process. Try rebooting and see if the space is cleaned up. If not $set def $device:[syslost] and see if it shows up there.

[Willem Grooters]

Is this the Windows corner bleating?

REBOOT IS NOT AN OPTION, unless there is really and absolutly no alternative and weverything else has been tried. In some cases you must, but in most cases, you simply don't have to.

1. First of all: find out what is keeping the space occupied. SHOW DEVICE/FILES, ANALYZE/SYSTEM for examination. On reboot, this information will be lost. Forever.
2. This will learn you if there are other ways to get around it: If a file is still in use by another process, stop that process locking the file (for instance: exit the program). Stop all user processes, if needed (STIP/ID=, not the best soclution but you may be forced to do so); If the file is installed: INSTALL REMOVE will fre the channels. If a batch program: DELETE/ENTRY.
3. If you are that desparate in disk space that 16Mb is worth the reboot, it is worthwhile to clean up your disk in other ways: Purge, delete temporary files etc. Make the calculation: 16Mb is NOTHING if you hace big disks with a default clustersize - you may loose more room as slack space (allocated but never used)
4. If all fails and you really need to reboot, do it as a last resort, if all other possibilities have failed.

That's the VMS way of solving problems.

Willem

[rison so_1]

I'm runing OpenVMS 7.3-1 on ES45 and DS20E with a resonable update patch and no special software running.

Yes, we have a cluster and I've checked both node for file locking and found no process has the file open.

Also have performed "Analyze/disk/repair" and still not missing the 16M block.

This 16M block is a RMS file which stores archive record and it was accidently deleted by our programmer( I do have a backup and I reauire the disk space to restore the backup ).

This file is not INSTALLED and is not a heavely use file.

[Peter Quodling]

Also have performed "Analyze/disk/repair" and still not missing the 16M block.

This 16M block is a RMS file which stores archive record and it was accidently deleted by our programmer( I do have a backup and I reauire the disk space to restore the backup ).

Your disks are that "tight" that you don't have 16MB spare... bzzzt. bad move. get more disk, archive data, do what needs to be done. Running systems close to the edge on disk, memory etc, is fraught with danger.

Do a show dev/files of the device that the file was on? Does the file in question still show up? (and what "owns it")

I would also concur with the previous statement, that something else may have "absorbed" the vacant space when it was freed up. (PArticularly as you are running so close to the limit)

Previous experience (about 18 years ago, as I recall) showed a client VAX with 8 disks all running at 99.5% capacity, and running like a dog. Some purging, deleting,restarting of log files, a few parameter changes and an autogen, and the system ran like a rocket. (client was a meat broking firm - I had steak dinners for the rest of the year...)

Q

[Hein van den Heuvel]

rison,
We may have a communication problem here.
Based on your last reply I am starting to believe you are not worried about getting the 16M blocks free space, but you want the file back that use to live in those 16M blocks, but which was accidently deleted.

Furthermore, you believe that no further allocations were done on the disk.

Am I on the rigth track?

In that case there is some chance that you can reconstruct the file. google/search for 'undelete' and 'vms' there are tools.

As an initial step you could search INDEXF.SYS/FORM=NON/NUM/WIN=1 for the file name in question.
This may (or might not!) get you the file header for the deleted file, which you can then dump and interpret using examples, or the 'guide to vms filesystem internals'.

$ cre aapnootmies.tmp
test
Exit
$ dir/file aapnootmies.tmp

Directory U$1:[HEIN]

AAPNOOTMIES.TMP;1 (37756,16,0)

Total of 1 file.
$ dele aapnootmies.tmp;
$ sea [000000]indexf.sys/form=non/num/win=1 aapnootmies
37895 (d<
SOH>ú¼úAAPNOOTMIES.TMP;1 ¤òOD¤C=D¤
ô}]
$ dump/head/bloc=(sta:37895,coun=1) [000000]indexf.sys
:

Be warned though that VMS re-allocates from a deleted file cache in LIFO order, so any allocation (file extent) would likely have gobbled up valuable data.

Restore is your better option!

Hein.

[Hein van den Heuvel]

Oh... forgot to mention... the first tool to try to help you with this should probably be "DFU" from the OpenVMS freeware. It has an UNDELETE function built in.

Also... (Getting carried away here with my speculations, but other readers might find this interesting...) On the lifo theme....

In the prior reply I deleted a file with ID = (37756,16,0)
Now if I create a new file I'll get the same ID back with a new sequence number:
$ cre newfile.tmp
more test
$ dir/file newfile.tmp
NEWFILE.TMP;1 (37756,17,0)
$ dump/head newfile.tmp:
Map area... LBN: 6127092
$
$ x=6127092
$ show sym x
X = 6127092 Hex = 005D7DF4

This is the exact same LBN as was visible in my prior dump:
:
0000005D 7DF48008 20202020 20202020 ..ô}]... 0000C0
:

To prove you can use those LBN's and that they are not 'cleared' on delete:
$ dele newfile.tmp;
$ dump/bloc=(count=1,start='x) sys$disk:
:
Logical block number 6127092 (005D7DF4), 512 (0200) bytes
:
0000FFFF 00747365 74206572 6F6D0009 ..more test..... 000000

See, the data is still there.

And now for a miraculous resurection / primitive undelete:

$ set vol/nohigh sys$disk
$ copy nl: test.dat /allo=9
$ set file/end test.dat
$ dump/blo=count=1 test.dat
Dump of file U$1:[HEIN]TEST.DAT;1 on 8-SEP-2005 21:44:13.32
File ID (37756,18,0) End of file block 9 / Allocated 9
Virtual block number 1 (00000001), 512 (0200) bytes

0000FFFF 00747365 74206572 6F6D0009 ..more test..... 000000
$ set file/attr=(ebk=1,ffb=12) test.dat
$ type test.dat
more test
$

Not for the faint of heart, when a serious file is lost.
Notice the NOHIGHWATERMARKING... or else...

Cheers,
Hein.

[Phil.Howell]

Would this happen if the rms file has global buffers (unlikely if it's an archive, but you never know)
Phil

[Hein van den Heuvel]

Phil wrote:

"Would this happen if the rms file has global buffers (unlikely if it's an archive, but you never know)"

Hmmm, this is sooo wrong on sooo many levels. It expresses and perpetuates an undeserved fear of the unknown: the rms global buffers.

1) "Would what happen?"
1A) Accidental deletes for file with global buffers? Sure!
1B) Free space not becoming visible after a delete of a file with global buffers? Possibly, but not more or less so because of those global buffers.
Those matters are utterly unrelated. Global buffers have NOTHING what so ever to do with file deletes.
They are NOT mapped file sections, but they are global buffers used to buffer IO to an from teh file into. They could readily continute to exist for a deleted file (allthough that would be pointless and it would represent a bug in rms)

2) While I agree you will not see global buffers on an archived file, why would you not want global buffers on an a file which receives archive records? Those could receive records from mutliple concurrent processes. Those could have frequent queries that can benefit from 'warmed up' buffers.
Anyway, it is a moot point as per above.

cheers,
Hein.

[Robert Gezelter]

rison so,

I would rate Peter's comments about "are you sure that nobody else grabbed the space" far more seriously than I would consider a reboot.

Reboot is unlikely to clear anything. I might, in some cases, consider dismounting and remounting the volume, but certainly not as an early course of action.

You said that you ran an ANALYZE/DISK/REPAIR. Check directory [SYSLOST] for the file. If the errant user removed the directory entry for the file, but did not actually issue a DELETE on the file, it may be considered a lost file. ANALYZE/DISK/REPAIR should have reported that it found a "lost file", but the message may have scrolled off of the screen.

If the file is in [SYSLOST] on the affected disk, then it can be put back in its correct location using the RENAME command.

More information would be extremely helpful. I hope that the above is helpful.

- Bob Gezelter, http://www.rlgsc.com

[comarow]

On all nodes of the cluster, do a show dev/files
and see if anyone has the file open.

You can set the output to a file and save the results.

show dev/file/out = t.t disk
search t.t file
That will kill it.

Check to see if it is installed.
install>
list

Those are the two most obvious reasons.

Otherwise the file will be marked for delete and you'll have to clean it up after a reboot, now it is marked or delete.


Good luck.

Bob Comarow

[rison so_1]

First, Thanks for all your help.


I've tried to dismount the disk on the weekend and I was able to dismount the disk which tells me no user/process is locking any file on the disk.

After mounting the disk, the 16M block still not showing as free disk space.

Can reboot really fix this problem ?

[Antoniov.]

Hi Rison,
how did you determine to loss space?

Antonio Vigliotti

[Willem Grooters]

Dismount - on one node or for all the nodes in the cluster?
The only way you can be absolutly sure nobody uses the disk anymore is to dismount the disk on all nodes so it can no longer be accessed from ANY node. Dismounting it from one node may leave others a possibility to access the disk.
Little, VERY little chance reboot would solve the problem.

As stated by others and me previously, FIRST analyse ALL your system in the cluster, to the utmost extent. Not just SHOW DEV/FILES, ANALYZE/SYSTEM to check process information on open channels, locks etc. Try DFU as well to report on MFD.
If you can successfully dismount the disk from all nodes, mount it privately (on one) and execute $ ANALYZE/DISK/REPAIR and analyze it's outcome (including [SYSLOST]).

Last: like Hein said: if you expect the file to be in exactly the same space: forget it. The best you can get is restore the file in one contiguous file. (Or you must restore the file as the first on a newly initilaized disk, but even that would be impossible since, AFAIK, you cannnot control the location where a file is restored on disk).

[Peter Quodling]

If you have "disconnected" all I/O by dismounting, flushed all "half done bits etc" with an anal/disk/repair, then the remaining answer is that some other application has "eaten your disk space.

It may, of course be something that "ate it" in one hit, so try a disk-wide dir/sel=siz=min=16MB (or something like that) to see if something has "eaten it" in one chunk.

failing that, a dir/since= to see what has been created since, and whether it adds up to 16MB.

q

[Willem Grooters]

A few more questions amnd hints:
16MB: is that the used size (DIR/SIZ) or the allocated size (DIR/SIZE=ALLOCATED)? In the latter case, you may have to loog for a larger size, depending on disk clustersize aned EXTENTSIZE parameter of the file itself. Since that is deleted, you won't be able to tell ;-(

No matter what: In the vast majority of situation, a file of that size will NOT be allocated in one chunk of blocks (AKA contiguous) but in a number of extents that can bre spread over the disk. So if you delete the file, the allocated space is freed for use. But you won't see as one 16Mb chunk of free space. One way to really find out is to $SHOW DEV/FULL immediately before and after deletion, and compare "free blocks". After deletion there should be 32M blocks more (more or less, since space might be occupied directly).
Only if a file has the attribute "Contiguous", deletion of a 16Mb file will free a chunk of 32M blocks that will show up as such.

(do a $ DUMP/HEADER/BLOCK=(COUNT:0) on a big file, you may see multiple retrieval pointers (this is a 18K block file):
Count: 8721 LBN: 11179503
Count: 843 LBN: 11244672
Count: 7086 LBN: 11275755
Count: 1653 LBN: 11282847
I have seen files with hundreds....(due to bad EXTENT attribute)))

So it is very well possible the blocks are freed, but in a number of chunks, (much) smaller then 32M blocks.

[Hein van den Heuvel]

>> Can reboot really fix this problem ?

No.

Please explain to use how you determine 16M 'missing' free space.
Did you know exactly what was there before and after?
Is this all the space you have?
Where exactly does that backup and accidental delete enter the story?

Regards,
Hein.

[John Gillings]

Hi Rison,

It looks like all the obvious possible reasons have been exhausted. Please log a case so we can look at this deeper.

[comarow]

If it's the system disk, I can send you an old Stars article,
How to recover space on the system disk.
Either post your email address, or send me an email to Bob.C@hp.com

[rison so_1]

Since this disk only stores archive data.

I've decided to Init the disk and restore all data.

Thank you for all your support.

[rison so_1]

Thank you!

[Peter Quodling]

Don't forget to assign points.