BACKUP /SINCE= /BEFORE= not doing it right?

[Art Wiens]

I'm trying to clean up a directory that has way too many files in it (~30,000). They have been quietly accumulating for a few years. I thought an "easy" way to solve this was to use BACKUP with the /SINCE and /BEFORE to select files based on the year they were created and put them in a saveset. My command was eg.:

$ backup disk:[dir.subdir.subsubdir]*.*;* -
/before=01-jan-2005 /since=01-jan-2004 -
differentdisk:[dir]files_from_2004.bck/save/block=32768 -
/log

It backed up all the files in the subsubdir, not just the ones created in 2004.

Why would it have done that? I checked some of the "wrong" files' dates and they don't fit the criteria eg.:

$ dir/date=(cre,modi) disk:[dir.subdir.subsubdir]xxx.xxx;2

Directory disk:[dir.subdir.subsubdir]

xxx.xxx;2 17-APR-2005 07:39:49.85 17-APR-2005 07:39:49.97

VAX/VMS v6.2

Cheers,
Art

[Uwe Zessin · Accepted solution]

Can you show the dates of DIR, SUBDIR and SUBSUBDIR? Starting with V6.2, the BACKUP utility picks up all files if a directory has been changed.

[Art Wiens]

$ dir/date=(cre,modi) disk:[000000]dir.dir

Directory disk:[000000]

dir.DIR;1 15-SEP-2002 17:49:59.55 15-SEP-2002 17:49:59.60

$ dir/date=(cre,modi) disk:[dir]subdir.dir

Directory disk:[dir]

subdir.DIR;1 15-SEP-2002 17:50:20.13 15-SEP-2002 17:50:20.20

$ dir/date=(cre,modi) disk:[dir.subdir]subsubdir.dir

Directory disk:[dir.subdir]

subsubdir.DIR;1 12-APR-2004 15:37:39.42 12-APR-2004 15:37:39.47

So since subsubdir was created in 2004, it picked up everything in it is what your saying?

I guess it's time to cook up some DCL to do it another way :-(

Art

[Art Wiens]

Can I use /EXCLUDE=something to my advantage here?

Art

[Uwe Zessin]

No, I don't think /EXCLUDE will help on a date problem. I'd pick up the FILE utility and change the directory's creation and revision dates a little into the past and retry so see if my guess is right.

http://vms.process.com/scripts/fileserv/fileserv.com?FILE

[Ian McKerracher_1]

Hello Art,

I cannot see what is wrong with your backup command but I would be tempted to try it again but include the "/CREATED" input file-selection qualifier. Maybe backup requires a more explicit instruction in this instance.

Regards,

Ian

[Antoniov.]

Art,
try adding /creat qualifier.

$ backup disk:[dir.subdir.subsubdir]*.*;* -
/before=01-jan-2005 /since=01-jan-2004 -
/creat -
differentdisk:[dir]files_from_2004.bck/save/block=32768 -
/log

Antonio Vigliotti

[Art Wiens]

/CREATE made no difference. Resulting saveset and logfile were identical.

So I guess the points go to Uwe ... I bet this "feature" was hotly debated 10 years ago!

I don't think I'll try the FILE suggestion but thanks for that. I'm afraid that once I had that kind of power, I'ld use it for evil and not good!! ;-)

Oh well, a little DCL goodness to make Friday go a little faster.

Cheers,
Art

[Ian McKerracher_1]

Hello Art,

Uwe is absolutely right with his comment. I found the following extract in the OpenVMS FAQ document (section 5.42.3) and it appears as though your problem could be correctable.

"As part of the OpenVMS V6.2 change, the /SINCE commandâ without
the specification of the =BACKUP keywordâ selected more files than
it should have. This is a bug. This bug has been remedied in the
OpenVMS BACKUP source code and in some of (all of?) the BACKUP
ECO kits."


Regards,

Ian

[Guy Peleg]

This looks like a bug. I do not see a
reason why /SINCE and /BEFORE can't be
combined.

If you have a valid support contract please
escalate this.

Regards,

Guy Peleg
OpenVMS Engineering

[Antoniov.]

Art,
if backup looks at directory date, you can exclude all directories in your backup.
e.g.
$ backup disk:[dir.subdir.subsubdir]*.*;* -
/before=01-jan-2005 /since=01-jan-2004 -
/crea/excl=*.dir; -
differentdisk:[dir]files_from_2004.bck/save-
/block=32768/log
This could work.

Antonio Vigliotti

[Uwe Zessin]

It's not a bug unless we are talking about different things or VMS eng. has changed their mind again!

It used to be a design decision to enable a correct restore in case the directory structure has been altered. It is described in the V6.2 release notes or new features guide - can be a bit hard to find because it is mentioned together with Pathworks. I don't have the material handy, so I can't point you to the correct section, sorry.

[Jan van den Ende]

Art,

looks like Ian hit the target.

So, if you still need to do this, I guess you need to apply the 6.2 BACKUP patches, and try again.
But I currently would not know how or where to get those patches....

Proost.

Have one on me.

jpe

[Guy Peleg]

Uwe,

I do not think VMS engineering changed
their mind....I was giving my opinion.

In my opinion if a user bothered and
spcified /since & /before BACKUP should
not ignore it. Maybe we need a new qualifier, this requires more thinking that's for sure but again I think it should be possible.

Guy

[Art Wiens]

I happen to have collected quite a few patches before the .au site "went away" (is it still really there?). I found vaxback03_062 (circa 18-MAR-1999) and from the ECO summary:

"Problems Addressed in VAXBACK01_062:
...
A new qualifier was added to invoke pre-OpenVMS V6.2 /SINCE behavior.

After enhancements to INCREMENTAL restores was done, feedback from customers and Customer Support Centers (CSCs) indicated that not everyone was completely pleased with the new /SINCE behavior.

Old style (pre-OpenVMS V6.2) /SINCE behavior can be invoked by adding the new /NO_INCREMENTAL qualifier to the BACKUP command line.

Note that this change was implemented differently in OpenVMS V7.1. Check the OpenVMS V7.1 release notes for that implementation.
..."

The system in question does not have any of the backup patches installed so I can't try the workaround. Please remember, I "inherited" all these old systems ;-) .

I did start the DCL using:

$ source_file = f$search("disk:[dir.subdir.subsubdir]*.*;*",1)
$ source_file_year = f$cvtime(f$file_attributes("''source_file'","cdt"),,"year")

as a basis for determining the year the file was produced.

Just a bit more error handling and we'll make that VAXstation smoke! ;-)

I am in the process of installing RCM and will decide what to do with patching once all the systems' info have been gathered.

Cheers,
Art

[Antoniov.]

Art,
/excl=*.dir seems to work on my V7.3-2 system.

Antonio Vigliotti

[Aaron Lewis_1]

Art, I had a similar issue with backup taking all the files instead of just the ones in the specified date range. What I finally figured out is that BACKUP, by default looks at the 'REVISED' date, and since I had recently run a utility to resequance all the version numbers on all files on the drive, it was backing up everything. I added the '/CREATED' switch to the BACKUP command, and everything returned to normal.

[Art Wiens]

Wow! How long would you think it should take to process ~30,000 files determining the year they were produced and copying them to an MSCP served disk if they weren't from the current year? How about 13hrs!!!

Accounting information:
Buffered I/O count: 2531433 Peak working set size: 2216
Direct I/O count: 1288895 Peak page file size: 18638
Page faults: 5390454 Mounted volumes: 0
Charged CPU time: 0 09:29:59.74 Elapsed time: 0 13:04:18.66

VAXstation 4000-90 80MB reading from a local SEAGATE ST15150N 4GB drive. Of the ~30,000, it was split about half it didn't have to copy vs those it did. Files are small text files 0-8 blocks each.

Of course it ran so long that the backup would have also been competing for some of those 13 hours.

Hope I didn't "break it" ;-)

Cheers,
Art

[Uwe Zessin]

That's about 1.56 seconds per file.

But the system must have been busy like hell:
9:29 hrs CPU of 13:04 hrs elapsed.

You also have 5+ million page faults!!

[Art Wiens]

I guess in between 3 lexicals, an IF decision, invoking WRITE for every file processed (to a logfile) and possibly BACKUP, there was lots of " cipherin' " to be done.

The VAXstation was like the Intel 386 ... who would ever need more processing power than this? Then they came out with the 4000 model 96 which usually had a cd drive as well. All the girls wanted to know you then!! ;-)

Cheers,
Art

[John Gillings]

Art,

Another approach is to use the RENAME command with your /BEFORE and /SINCE time window into a new, temporary directory. Then use BACKUP *.* from there.

DIR/NOHEAD/NOTRAIL/OUTPUT might also be of interest.