Operating System - OpenVMS
1753427 Members
4952 Online
108793 Solutions
New Discussion юеВ

Re: VAX/VMS V7.1 Large Directory Files

 
oldskul
Advisor

Re: VAX/VMS V7.1 Large Directory Files

And how do you get paragraph breaks in your responses? Copy/Paste from someplace else? :) Tks.
Hoff
Honored Contributor

Re: VAX/VMS V7.1 Large Directory Files

To get a paragraph break using the Safari or Firefox browsers, add a blank line.

You may be using Microsoft tools and Microsoft Internet Explorer on Microsoft Windows?

If so, then I might guess you've Microsoft Office involved here somewhere, and potentially cutting and pasting and possibly composition, as that package tends to do wacky things with line wraps around cut-and-paste. (I don't even use Microsoft Windows or Office, and I've hit that case.) Use Wordpad in its stead; as your compose pane, and cut-and-paste from there.

Yes; there are various problems here. The planned ITRC UI "upgrade" will improve the forums from "rank disaster" all the way up to "corporate-level UI embarrassment", too.

This ITRC forum stuff works "normally" (read: no worse than usual) on Mac OS X using Safari browser, and using cut-and-paste from the "gun-slit" text entry box as needed. (And you can drag the text box bigger in Mac Safari.)
Andy Bustamante
Honored Contributor

Re: VAX/VMS V7.1 Large Directory Files


I once supported an browser based application that created and deleted html files on a regular basis. The developers used a single directory and we purged files after 30 minutes. This was fine on small sites, larger customers wound up with directory entries in the 1000's of blocks where a delete pass would saturate the disk for hours. (Try $MONTOR DISK/ITEM=QUE).

Breaking the logical into multiple directories and having a batch job update the logical on the fly led to a world of improvement. This let me delete one smaller directory that wasn't in customer use. You can spread the logical over multiple disks or license DEC-RAM if you need more I/O bandwidth.

I couldn't get approval to deploy the update to 7.2, so I updated a customer in conjunction with other travel as a "test" case. They posted how a review and mentioned how much faster the system was on a customer mailing list and suddenly the policy was how quickly can we deploy.
If you don't have time to do it right, when will you have time to do it over? Reach me at first_name + "." + last_name at sysmanager net
tsgdavid
Frequent Advisor

Re: VAX/VMS V7.1 Large Directory Files

THANK YOU, THANK YOU, THANK YOU!

to John Gillings for this information and your suggested delete procedure.

>>>
But to really see poor performance, consider the DELETE command. You start by deleting the files at the beginning of the file, until you empty the first bucket. Since empty buckets aren't allowed, you then copy the 2nd to the first, and cascade to the end of the file. Now you proceed to empty the 2nd bucket (which is now the first), and repeat copying the entire directory every dozen or so files. That's why it takes so long.
>>>

We have had a problem related to this for a LONG time with a cleanup procedure that runs every day. One directory takes almost 5 hours to delete the older files. This should cut this time drastically!

Dave
Hein van den Heuvel
Honored Contributor

Re: VAX/VMS V7.1 Large Directory Files

Dave, you have not been paying attention.. we've known and written about how to fix this for decades! :-).

You may also want to get out DFU DELE/DIRECTORY.
Mind you, you may need to first save away what you would like to retain in a fresh directory.


In comp.os.vms search for "hein delete sort"
And you'll find the skeleton script below.

Btw... Now that the XQP shuffles up and down with ACP_MAXREAD it could potential add value during the shuffle by peaking into the buffers. If it 'sees' the opportunity to compact the whole buffer by a block or so then it could do so and reduce the total blocks to be shuffled for that case and future shuffles.
Possible drawbacks
1) some slack / fill space is nice for files which come and go.
2) if the move gets interrupted the result will be messy / messier.

Hmmm, this is probably best left as a conscious decision as implemented by DFU DIRECTORY/COMPRESS/FILL versus an automatic, uncontrolled action by the XQP.

fwiw,
Hein


http://groups.google.com/group/comp.os.vms/search?group=comp.os.vms&q=hein+delete+sort&qt_g=Search+this+group

$if p1.eqs."" then goto help ! Hein van den Heuvel, DIGITAL, Jan-1995
$directory/out=sys$Login:delete.tmp/col=1 'p1
$sort /key=(pos:1,siz:-1,desc) sys$Login:delete.tmp sys$Login:delete.tmp
$open/read/error=clean_up sorted sys$Login:delete.tmp
$init:
$read/end=clean_up sorted f
$l2 = f$len(f)
$if f$loc(";",f).eq.l2 then goto init
$files = f
$loop:
$read/end=done sorted f
$l1 = f$len(f)
$if f$loc(";",f).eq.l1 then goto loop
$if l2.lt.200
$ then
$ files = files + "," + f
$ l2 = l2 + l1 + 1
$ else
$ delete 'p2' 'files'
$ files = f
$ l2 = l1
$endif
$goto loop
$done:
$delete 'p2' 'files'
$clean_up:
$close/nolog sorted
$delete sys$Login:delete.tmp;,;
$exit
$help:
$type sys$input
Usage:
P1 (required) = wild card file spec, with optional DIR selection qualifiers
P2 (optional) = delete command qualifiers (/LOG or /CONFIRM)
$exit

RBrown_1
Trusted Contributor

Re: VAX/VMS V7.1 Large Directory Files

Since I am also stuck at V7.1 (on old hardware too!), your question interested me.

I threw together the attached command file to give some rough measure of speed of mass file creations and deletions. Whether or not I can find a newer version of VMS running on similar old hardware is a different question.

Other replies on this topic seem to indicate that the operation measured by this command file is not fixable by an upgrade anyway. Oh well.

Have fun!
RBrown_1
Trusted Contributor

Re: VAX/VMS V7.1 Large Directory Files

Evidently, ITRC forums did me a favour by declining to include my attachment.
Hoff
Honored Contributor

Re: VAX/VMS V7.1 Large Directory Files

Gazillions of versions of the file 0.txt are close to the tightest you can pack stuff into a directory. If you want to fall off the directory cache, differing and (much) longer filenames are the usual choice.

And directories and individual files will make for stinky application databases, and no amount of tweaks to caching nor to ODS will help with that; not to the degree that using the proper tools would.

There are folks that have posted here that have stashed hundreds or thousands of blocks of filenames into a single directory on a daily basis, for instance, and who ever looks at that cruft? Stacking up gazillions of log files is a common problem, and that then leads to a look at compression or data reduction, or both, or just shutting off the log file creation during normal operations.

Yes; they've had problems with using a directory as a database.
P Muralidhar Kini
Honored Contributor

Re: VAX/VMS V7.1 Large Directory Files

John,

>> Note we've been promised a DCL DELETE/TREE for a few decades now.
>> Maybe it's in V8.4?
Yes, its is.
OpenVMS V84 provies the DELETE/TREE option.

Regards,
Murali
Let There Be Rock - AC/DC
oldskul
Advisor

Re: VAX/VMS V7.1 Large Directory Files

There really isn't a resolution to this as I only asked for information, which I did receive. Thank you everyone.