Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

House keeping for file version in Openvms

 
shiva27
Frequent Advisor

House keeping for file version in Openvms

Hi All,

In VMS is it possible to keep the housekeeping for file versions ? As you know in VMS Max. file version is 32767 , after that if we try to create the same file, it won't allow. so we've last option to purge the file and rename file to version 1. So doing this things manually every week, Can we put any script so that once file version reached to "a.txt;32767" then automatically script will purge this file and rename to version 1 as well.

we are using one monitoring tool and it generate the new logs files with same name every 1 hr , I've already kept file housekeeping to Purge/keep=5 a.txt
But this is not enough to fix this issue , Vendor still checking on their script.So by the time is there any way to fix this.

Note:same script is running on HP-UX w/o any issue because no file version limit concept there as in VMS.
13 REPLIES 13
Steven Schweda
Honored Contributor

Re: House keeping for file version in Openvms

I would not wait until version 32767 has been
created before I did something.

> Can we put any script [...]

The best place to handle this may be in the
procedure/program which creates the log file,
rather than in some periodic thing which may
not run when you need it.

Craig A
Valued Contributor

Re: House keeping for file version in Openvms

I think you first need to work out whether you NEED to keep these files historically.

I prefer to housekeep logfiles by suffixing the file with _YYYYMMDDHHMMSSHH of the creation date of the file.

So OVERNIGHT_BATCH.LOG would become
OVERNIGHT_BATCH.LOG_2008072801012356

Then, if you want to housekeep those files you can use something like:

$ BACKUP/LOG/VER/DEL *.LOG_* /BEFORE="-90-" LOGS.BCK/SAVE

I echo the above that you do not want to let file version numbers get out of control. Anything over a couple of hundred and you want to start having a closer look.

HTH

Craig A
Robert Gezelter
Honored Contributor

Re: House keeping for file version in Openvms

shiva,

Both Steve and Craig have good points.

One can automatically limit the number of retained versions of a file by using SET FILE/VERSION_LIMIT. However, the effect of automatic purging files does have significant dangers with regards to losing data by accident.

One can also explicitly specify a non-zero version number and overwrite the previous file. This is not generally done for the same reason; it can lead to a variety of accidents. Often, the reason for this is that while the application is writing (and re-writing) version 1 of the file, someone accidentally edits the file and saves (rather than quits) the edit, creating version 2.

Caution is strongly advised.

- Bob Gezelter, http://www.rlgsc.com
Heinz W Genhart
Honored Contributor

Re: House keeping for file version in Openvms

Hi shiva

In your forum profile we can see the following:

I have assigned points to 1 of 140 responses to my questions.

Maybe you will find the time to assign points to those people who helped you solving your problems. This link could help you with assigning points http://forums13.itrc.hp.com/service/forums/helptips.do?#28

Back to your current problem. All the answers from steven, craig and bob are correct. If you still want to use a procedure which solves your problem, then see attached file. It is a subroutine. P1 is the name of the file with the high file version. The Subroutine renames all those files. If you have for example the following files:

x.x;300
x.x;299
x.x;298

and you start the Subroutine wit x.x as P1 then

x.x;298 will be renamed to x.x;1
x.x;299 will be renamed to x.x;2
x.x;300 will be renamed to x.x;3

Hope that helps

Regards

Geni
John Gillings
Honored Contributor

Re: House keeping for file version in Openvms

Note that renaming files is always subject to timing windows if files are created during the RENAME operation.

Assuming no interference, the simplest way to reset a range of file versions to a contiguous range starting from 1 is:

$ RENAME sourcename.type;* sourcename.TMP;
$ RENAME sourcename.TMP;* sourcename.type;

(UPPER CASE characters and punctuation all appear exactly as written, lower case characters replaced according to your file name).

This will change some dates dates in the header.
A crucible of informative mistakes
Steve Reece_3
Trusted Contributor

Re: House keeping for file version in Openvms

The whole issue is surrounded by the quesiton of what do you really want to achieve here?
If you have 32767 log files, that's going to be a log of hunting if you try and find an issue with the job.
You're also likely to be creating a lot of fragmentation on the disk where these files exist if you continually delete files. If you're looking at a physical disk that also contains data files and you're expecting performance, think again (I know this from experience!)

So, potential resolutions:
- set up a script that looks for versions of the file and then renames them, as you're suggesting.
- set up a script that zips the files up into a dated zipfile. If you do this with the move file into zip archive then you won't have lots of files dotted around but you'll still be able to unzip the zip archive if you need to troubleshoot a problem.
- Put rename and purge logic into the start of the script that generates the log file.
- Create the log file on the null device so that it doesn't really create a log file at all?
- Submit a second job from the job creating the log file which deletes the log file after, say, one minute? Rubbish for troubleshooting, though you can always try and do a search for an error status before doing the delete.

Depending upon how quickly the log files are being created, you might always try something as simple as:
$ PURGE LOGFILE.LOG/KEEP=3
$ RENAME LOGFILE.LOG;-2 LOGFILE.LOG;1
$ RENAME LOGFILE.LOG;-1 LOGFILE.LOG;2
$ RENAME LOGFILE.LOG; LOGFILE.LOG;3

Steve
shiva27
Frequent Advisor

Re: House keeping for file version in Openvms

Thanks All for your quick help.

As there is weekly reboot of servers (business requirement),i will put below options in system startup script as suggested by Steven.
$ PURGE LOGFILE.LOG/KEEP=3
$ RENAME LOGFILE.LOG;-2 LOGFILE.LOG;1
$ RENAME LOGFILE.LOG;-1 LOGFILE.LOG;2
$ RENAME LOGFILE.LOG; LOGFILE.LOG;3

Warm regards,
Shiva
Steve Reece_3
Trusted Contributor

Re: House keeping for file version in Openvms

It will be advisable to run this manually before you get to the next reboot, just in case there is anything else that causes problems.
You'll also need to issue the commands at a time when the log file isn't open - otherwise VMS won't let you rename the file!
Steve
Craig A
Valued Contributor

Re: House keeping for file version in Openvms

Steve

>You'll also need to issue the commands at a >time when the log file isn't open - >otherwise VMS won't let you rename the file!

I think RENAME is possible, just certain other operations aren't.

Craig A
Richard W Hunt
Valued Contributor

Re: House keeping for file version in Openvms

I don't claim that this works for everyone, but in a U.S. Dept. of Defense shop, I get a lot of log-file retention requirements. The versioning problem crops up all over the place for me.

What I do for those files that I simply MUST keep (e.g. Error log, Security Journal, User Accounting, Operator log, and a few others), I built a script that searches for files matching the template in a little F$SEARCH loop. For each file, I look at the file's attributes to get the creation date. I convert the creation date to COMPARISON time and remove all punctuation by using string subtraction. I then tack the remaining string as a time-tag on the file. E.g. OPERATOR.LOG will become OPERATOR_2009080723513614.LOG (or whatever). Depending on how often I do a particular set of "trimming" I might reduce the size of the time-tag to as short as yymmdd for the daily cleanups.

That's phase I. Do that at least daily, do it more often as a low-priority batch job if you like. Do it for ALL files. (You can rename an open file if you have adequate privileges for the script doing the dirty deed. The file was opened by its creator in a way that won't lose track of it while it is still open, so you don't kill anything.) When a new version of the file is generated, there is NO extant version so you just restart at version 1 for the new file.

Phase II is that monthly, I use the ZIP program that is part of the stuff that is found under the [...SVCTOOLS.RCM] subdirectory. Give it a wildcard template for file selection to have it zip up last month's files. You can script this, again using COMPARISON time to extract date parts for YEAR and MONTH to develop a unique, month/year-tagged zip file to hold your stuff. You can easily get anywhere from 65% to 95% compression for typical system files such as I named earlier.

HINT: How to get last month's month-number and year number? Take today's date. Extract the DAY, which will be something from 1-31. Convert it to integer, add 1 to it, then build a string equal to a delta-time of that number of days followed by a dash. (Which means an integer number of days.) Then in an F$CVTIME() function, take todays date and subtract from that the delta-time string you just computed. The resultant date will be the last day of the previous month. From this date, you can extract the year and month easily using F$CVTIME() with "YEAR" and "MONTH" operations.

Phase III is that if our government guys ask a question, I use the UNZIP program found under the [...SVCTOOLS.BIN] subdirectory and I can then find relevant files according to date of creation. (And, of course, the names allow me to know the subject matter contained in each unzipped file.) Wildcard searches for a month's worth of files with the /LOG /WINDOW=0 options will tell me if I have any hits - and which file they are in. When done, just delete the unzipped files and retain the zip container.

Phase IV is that once the stuff gets old enough, we either back it up to a long-term retention tape (if we still might need it) or we just discard it based on creation dates of the ZIP files. What this gives us is a rolling window of the critical files in one-month slices, date-tagged, and therefore easy to search if you have an idea of that which you seek by date. Right now, I've got this running to give me a two-year rolling log of everything, with one-month granularity for the containers.

This IS NOT for the faint of heart in DCL scripting and is NOT for the sites that are not blessed with enough space to support that much retention. In my case, the retention WAS required and I got enough disks allocated to support the requirement. Your system's free space will govern your retention limits.

This is how you rein in profligate file versions.
Sr. Systems Janitor
Hein van den Heuvel
Honored Contributor

Re: House keeping for file version in Openvms

Shiva wrote...
> script as suggested by Steven.
>$ PURGE LOGFILE.LOG/KEEP=3
>$ RENAME LOGFILE.LOG;-2 LOGFILE.LOG;1
>$ RENAME LOGFILE.LOG;-1 LOGFILE.LOG;2
:

A litte DCL/file-name-parsing trick.
Rename works mutliple version from teh top down, when you specify an input wildcard.
With an explicit version it default to the current version. Now if you make the output version ";" = "The latests", then it will use that, for each file version, effectively reverse-ordering, from the lowest possible version: ;1 for a clear target, or one higher than any remaining version if you have left-overs.

Repeat twice. Once to renumber downwards. Once to renumber 'properly' according to original order:


$ pur/keep=3 y.cob
$ ren y.cob;* x.cob;/log
%RENAME-I-RENAMED, Y.COB;9 renamed to X.COB;1
%RENAME-I-RENAMED, Y.COB;8 renamed to X.COB;2
%RENAME-I-RENAMED, Y.COB;7 renamed to X.COB;3
$ rena x.cob;* y.cob;/log
%RENAME-I-RENAMED, X.COB;3 renamed to Y.COB;1
%RENAME-I-RENAMED, X.COB;2 renamed to Y.COB;2
%RENAME-I-RENAMED, X.COB;1 renamed to Y.COB;3


Hein.
The Brit
Honored Contributor

Re: House keeping for file version in Openvms

I've never seen that one before.

Sneaky!

John Gillings
Honored Contributor

Re: House keeping for file version in Openvms

>I've never seen that one before.

Just look up a few replies...

>$ RENAME sourcename.type;* sourcename.TMP;
>$ RENAME sourcename.TMP;* sourcename.type;

(maybe I should have explained how it works?)
A crucible of informative mistakes