topic Re: Output of "show dev d" seems not correct in Operating System - OpenVMS

Output of "show dev d" seems not correct

Ricky Pardede — Mon, 05 Jul 2010 11:40:18 GMT

Hi All,

I suspect I have untraced disk usage on my system. It is on the system disk.

From "show dev $1$DGA100/size=byte/full" :
Total size 100.00GB
Free space 30.16GB
=> means around 70 GB is used.

Meanwhile, from "dir $1$DGA100:[000000...]*.* /size=unit=byte/total" :
Grand total of 3025 directories, 91108 files, 21.73GB
=> it says disk usage is only 21.73 GB

I can not find any folders that take disk usage excessively.

Can anyone suggest why "show dev d" indicates high disk usage on my system ?
Why the output is so much different with "dir /total"

Thanks a lot guys.

Re: Output of "show dev d" seems not correct

Jan van den Ende — Mon, 05 Jul 2010 12:12:36 GMT

Ricky,

multiple reasons, but you very probably already get much better agreement if you do those measurements again after
$ SET VOLUME/REBUILD
That will clear (most importantly) any remaining incorrectly closed extended files.
A next step would be
$ ANAL/DISK/REPAIR
That will (most importantly) clear any references to INSTALLed files that were deleted before deINSTALL, or upon INSTALL/REPLACE will still being referenced (ANAL/DISK will report those as Delete Pending)
Furthermore, in an active system some disk space is already "reserved" for quick allocation when wished for. That amount gets multiplied by the number of nodes in a cluster.
Furthermore,
$ dir $1$DGA100:[000000...]*.*
reports only ONE version of any file, if you need to see them all, then
$ dir $1$DGA100:[000000...]*.*;*
or
$ dir $1$DGA100:[000000...]
would give more relevant info.

... and the actual situation can easily be reversed for the system disk, because you see many directories twice!
Each [SYS*]SYSCOMMON.DIR is an alias for [000000]VMS$COMMON.DIR, and all files in there are counted (at least!) twice.

hth.

Proost.

Have one on me.

jpe

Re: Output of "show dev d" seems not correct

Joseph Huber_1 — Mon, 05 Jul 2010 12:27:11 GMT

I recommend to install and use the DFU utility for this purpose: its REPORT command show all relevant usage/allocation/free data.

Re: Output of "show dev d" seems not correct

Kris Clippeleyr — Mon, 05 Jul 2010 12:48:44 GMT

Hi,

DIRECTORY/SIZE gives the "size used" not the "size allocated". That also might explain (partially) the difference you've calculated.

Regards,
Kris (aka Qkcl)

Re: Output of "show dev d" seems not correct

Hein van den Heuvel — Mon, 05 Jul 2010 13:00:53 GMT

Ricky,

This this OpenVMS. It is correct.
Certainly for common, simple, operations such as "SHOW DEV"

So when you write "Output of "show dev d" seems not correct" you should have been thinking "what am I missing in my understanding of the system"

In this case the verification seemed flawed.
The command issued, only reports the USED bytes, not the ALLOCATED bytes.
Change to : /size=(alloc,unit=byte)
Also change to : /size=(all,unit=byte)

Try this on a small directory first, not the whole disk.

Of course it could also be that there files on the disk which are not in any directory, and other tricky situations, but for now let's just assume the difference is in allocated versus used because allocated was not reported / an unknown concept.

Joseph: use the DFU utility
YES! for serious investigation, but this is just basics.

Jan: $ SET VOLUME/REBUILD ... $ ANAL/DISK/REPAIR
YES! for serious investigation, but this is probably just basics.

Jan: *.* reports only ONE version of any file
NO! *.*; reports only one.

Jan: situation can easily be reversed for the system disk
YES!, but let's keep it simple for now.

Good luck!
Hein

Re: Output of "show dev d" seems not correct

Jan van den Ende — Mon, 05 Jul 2010 14:10:06 GMT

@Hein:

>>>
Jan: *.* reports only ONE version of any file
NO! *.*; reports only one.
<<<

Oops! My bad! Thank for the correction.

Proost.

Have one on me.

jpe

Re: Output of "show dev d" seems not correct

Suresh Hegde — Tue, 06 Jul 2010 03:42:45 GMT

Ricky,

I think, i had this situation too.

I believe this might be because of the two different links (VMS$COMMON and SYSCOMMON) for same directory SYS$SYSDEVICE:[SYS0.SYSCOMMON] in system disk.

For example:

When i do directory for single file from my login directory, i get like below...

1.
>dir [-...]ABS$BACKUP.EXE/size/grand/total

Grand total of 2 directories, 2 files, 2378 blocks.
>

2.

>dir [-...]ABS$BACKUP.EXE/size

Directory SYS$SYSDEVICE:[SYS0.SYSCOMMON.SYSEXE]

ABS$BACKUP.EXE;1 1189

Total of 1 file, 1189 blocks.

Directory SYS$SYSDEVICE:[VMS$COMMON.SYSEXE]

ABS$BACKUP.EXE;1 1189

Total of 1 file, 1189 blocks.

Grand total of 2 directories, 2 files, 2378 blocks.
>

Hope this helps you understand the difference.

/Suresh

Re: Output of "show dev d" seems not correct

Jon Pinkley — Tue, 06 Jul 2010 04:58:01 GMT

@Hein,

Although I agree with you that the most likely cause of the discrepancy is that the command was displaying used instead of allocated space, OpenVMS 8.3 (Alpha) still occasionally gets confused. The following shows the results of analyze/disk/repair after I tried to backup about 4 million blocks (2 GB) of files to a device that was showing over 5 million block free. I was surprised when the operation failed with an insufficient space error. This is all I saved, which occurred after the error.

$ show dev DISK$BJLARC01

Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
$1$DGA3305: (SIGMA) Mounted 0 BJLARC01 2089392 1 2
$ anal/disk/rep disk$bjlarc01
Analyze/Disk_Structure/Repair for _$1$DGA3305: started on 1-MAR-2010 03:36:14.93

%ANALDISK-I-OPENQUOTA, error opening QUOTA.SYS
-SYSTEM-W-NOSUCHFILE, no such file
%ANALDISK-W-FREESPADRIFT, free block count of 2089392 is incorrect (RVN 1);
the correct value is 0
$

One of the two cluster nodes had been recently rebooted, and the disk mounted with the following mount command:

$ mount/system/noassi/rebuild/subsystem/wind=80 $1$dga3305: bjlarc01/nowrite

but the mount failed because the disk wash mounted for write on the other node. I possibly had recently deleted files from the disk, I can't say for sure. When I mounted it manually, I probably used that exact command without the /nowrite (cut & pasted from our mount_disks.com file)

What is odd is that this disk has very little activity, it is just used to archive old backup journal files. Perhaps the inactivity in some way contributed to the free space drift. Other disks that were frequently modified did not have the problem.

The reason I saved the output was because it was something that happens infrequently. And I wasn't able to reproduce the problem (but I didn't try real hard).

If /size=allocated is used, and there are no alias files on the disk, then the numbers should add. Up.

I just tried on a disk with a relatively small number of large files, no aliases, and in this case, the numbers free + allocated = total (as expected)

$ dir/size=allocated disk$data4:[000000...]/grand

Grand total of 11 directories, 12719 files, 215333504 blocks.
$ show dev disk$data4/ful ! output edited to remove non-relevant info

Disk DSA2200:, device type Generic SCSI disk, is online, mounted, file-oriented
device, shareable, available to cluster, error logging is enabled, device
supports bitmaps (no bitmaps active).

Total blocks 251658240 Sectors per track 128
Logical Volume Size 251658240 Expansion Size Limit 2147475456
Free blocks 36324736 Maximum files allowed 12549313
$ write sys$output 251658240-36324736
215333504
$ write sys$output 251658240-36324736-215333504
0
$

Jon

Re: Output of "show dev d" seems not correct

Steve Reece_3 — Tue, 06 Jul 2010 05:12:36 GMT

Hi Ricky,

As others have commented, the most likely cause for this (I'm assuming that it's not a system disk) is that the space you're reporting is the used disk space rather than the allocated space. It works like this:

I have a disk of 36GB. Because I want to save space in the index files and bitmaps and such like, I initialize the disk with a cluster factor equivalent to 5MB. That means that the smallest space that I'm able to allocate to a file is 5MB - no matter what the contents of the file are. So, if I have a 2KB file it will have 2KB used of 5MB allocated. If I have a 6MB file it'll have 6MB used of 10MB allocated. Only files that are nice multiples of 5MB will use all of the space allocated to them. The rest of the space will be allocated but not actually used.

If you have any BIG log files that are still open for write (e.g. SET HOST 0/LOG) then this will also allocate space but not use it until the file is closed. On a DIR/SIZ=ALL you'll see things like 0/1778 or 0/9234567 with the used figure on the left and allocated on the right.

An ANALYZE/DISK pass across the device might uncover some lost files or files that the system thinks should be deleted but haven't been and stuff like that, but it sounds like the disk was just badly set up to start with compared with the present use of it.

Re: Output of "show dev d" seems not correct

Jon Pinkley — Tue, 06 Jul 2010 05:20:53 GMT

@Suresh,

You make a valid point; that directory /grand will include the contributions from alias files or alias directories, thus overstating the total size that directory reports. However, in this case, Ricky was wondering why show device disk was showing a higher utilization than directory /grand. Alias directories (as exist on the system disk) will make the output of directory [000000...] larger, not smaller. That's why the most likely cause is the discrepancy between directory/size=allocated vs. the default /size=used.

Steve just provided a good explanation of why allocated size is always .ge. used size.

Jon

Re: Output of "show dev d" seems not correct

Ricky Pardede — Tue, 06 Jul 2010 08:01:48 GMT

@all : Thanks for mentioning about used and allocated. I didn't aware on this concept.
I believe my case was caused by this wrong reading.

By checking "allocated", I have found a directory that has high usage of allocated space. It's smsc$root:[log]

XXXX:SYSTEM> dir /size=(alloc,unit=byte)/total
Directory SMSC$ROOT:[LOG]
Total of 2656 files, 59.59GB

Then, I try to copy this whole directory to another one but in different disk.
The result :
XXXX:SYSTEM> dir /size=(alloc,unit=byte)/total
Directory SMSC_TRC0:[LOG]
Total of 2599 files, 287.56MB

I was surprised with the comparison result.

The original takes allocated 60GB, and the copied takes allocated just 288MB.

Can please suggest me which related setting I need to adjust on $1$DGA100 ?
Or may be I need to Jan suggestion :
SET VOLUME/REBUILD
ANAL/DISK/REPAIR ?

Thank a lot.

Re: Output of "show dev d" seems not correct

Ricky Pardede — Tue, 06 Jul 2010 08:30:05 GMT

I suspect the system disk $1$DGA100 needs to repair.

I deleted the whole files in smsc$root:[log], but still it is showing allocated space 60GB.

XXXX:SYSTEM> dir /size=(alloc,unit=byte)/total
Directory SMSC$ROOT:[LOG]
Total of 58 files, 59.40GB

I am not sure what's the cause.
Can help to suggest me the safest step to try troubleshooting ?

Thanks a lot.

Re: Output of "show dev d" seems not correct

Joseph Huber_1 — Tue, 06 Jul 2010 08:41:28 GMT

Leave out the /total, and do a dir/size=all to see which files remained.
If there are none, then a rebuild and analyze/repair are the way to recover.
rebuild will free allocated/cached blocks.
analyze/repair finds files which are not in any directory, and puts them into [syslost] directory.

Re: Output of "show dev d" seems not correct

Joseph Huber_1 — Tue, 06 Jul 2010 08:51:19 GMT

and by the way:
[
dir /size=(alloc,unit=byte)/total
Directory SMSC$ROOT:[LOG]
Total of 58 files, 59.40GB
]

shows there ARE still files, not just allocated
Maybe You did not delete all versions of files ?

Re: Output of "show dev d" seems not correct

Ricky Pardede — Tue, 06 Jul 2010 09:13:53 GMT

@Joseph :
Some files can't be deleted because it's locked by another process.

So you suggest me to execute :
anal/disk/repair $1$DGA100 ?

To execute this, I think I need to shutdown all applications and running queue.
Isn't it ?

Re: Output of "show dev d" seems not correct

Joseph Huber_1 — Tue, 06 Jul 2010 09:47:20 GMT

[
Some files can't be deleted because it's locked by another process.
]

No, in this case a /repair will not change the situation.
Only a stop of the application locking the files will unlock them, then delete.

Re: Output of "show dev d" seems not correct

p.balamurugan — Tue, 06 Jul 2010 10:12:32 GMT

show dev d/full. you need to see the cluster_size. For example if cluster_size=200 and you are file is only 10. As per VMS it will allocate the 200 block and content may be 10 block. To reduce this while initialize you need to give the cluster_size parameters. So your disk usage will not be wasted. Here I pasted the example.

25MAY.LIS;1 6/16(it is allocated space and this value will be taken from cluster_size). In the above example you could see size of the file is 6 block. But allocated is 16 block. So you need to reduce the cluster_size value. your issue will be resolved.

Re: Output of "show dev d" seems not correct

Shriniketan Bhagwat — Tue, 06 Jul 2010 10:31:37 GMT

Hi,

If your requirement is to have larger cluster factor and want to reduce the allocated but unused blocks of the file, then you can use the $ set file/truncate command. This command truncates the file at the end of the block containing the end-of-file (EOF) marker and releases allocated but unused blocks of the file. Try this on some huge file which has larger unused and allocated blocks. Truncate the file and then check the free space on the disk. This could be better approach than deleting the file (in my opinion).

Regards,
Ketan

Re: Output of "show dev d" seems not correct

Jan van den Ende — Tue, 06 Jul 2010 10:47:00 GMT

@Ketan,

not exactly.

SET FILE/TRUNC reduces the file down to the disk cluster containing the EOF.
Then again, so does a normal close of a file, as well as SET VOLUME/REBUILD.
What you refer to is only slightly related: If a file needs to grow, the chunk of growth is the EXTEND size (can be set per file, if not uses the volume's setting) - rounded up to the next cluster size border.
If a file is NOT closed appropriately (program crash, process crash, system crash) then the extended chunk is not rounded down.

hth

Proost.

Have one on me.

jpe

Re: Output of "show dev d" seems not correct

Joseph Huber_1 — Tue, 06 Jul 2010 11:24:17 GMT

The cluster size is completely irrelevant here:
the OP has 58 files summing up to a total of 59.6 GB, hold locked by an application.
Even if each of the file has once cluster allocated unused, that's nothing compared to the total size.