PURGE/KEEP=../BEFORE=..

[Jan van den Ende]

Hi all,

Today we got into a discussion of the behavior of the title subject.

Does it:

a. Purge down to /KEEP= versions, but refuse to delete any files after /BEFORE

or

b. Scan for any file /BEFORE, anf trim those down to /KEEP.

We did some testing.

$ DIR/DAT X.X
X.X;7 22-JAN-2007 15:08:10;00
X.X;6 22-JAN-2007 15:07:50.00
X.X;5 22-JAN-2007 15:06:40.00
X.X;4 17-JAN-2007 12:10:00.00
X.X;3 17-JAN-2007 12:08:00.00
X.X;2 17-JAN-2007 11:55:00.00
X.X;1 16-JAN-2007 16:00:00.00
Total of 7 files

(well, not exactly, but this is the gist of it)

$ PURGE /BEFORE=19-JAN/KEEP=6
(a. should leave 7 files, b. six)

$ DIR/TOT X
Total of 7 files.
Seems to point to a.

$ PURGE /BEFORE=19-JAN/KEEP=3
Now we expect 1 file gone, but:
$ DIR/TOT X
Total of 5 files. !!!! ;1 & ;2 gone
Only TWO files /BEFORE left

Repeat: No change

$ PURGE /BEFORE=19-JAN/KEEP=2
version ;3 also gone!
Only ONE file /BEFORE left !!!

$ PURGE/BEFORE="15:07"/KEEP=2
AGAIN, only ONE before left!


Anybody seen this before?
Anybody any explanation?

We tentatively concluded this was a little bug in the algoritm, but we await your vision.

Proost.

Have one on me.

jpe

[Jan van den Ende]

Sorry, I forgot:

This is on 7.3-2

Proost.

Have one on me.

jpe

[Hoff]

Odd. I'd tend to simplify the command -- this would not be the first odd behavior from PURGE. A workaround, of course. (That, or file or directory version limits.)

Lob a bug report at HP -- whomever within OpenVMS engineering owns DCL also owns this decision, and they own the vision around how this was intended to work, and the decision around whether the code or the documentation will change. If nothing else, the existing documentation could be clarified around the ordering and the intent.

I'd half wonder if the file selection order might (also) be dependent on the order of the qualifiers. There have been weirder things implemented.

[Jan van den Ende]

Hoff,

we also wondered about the order of qualifiers.
As far as we investigated that, there is no difference.

But it still seems strange, that the actual numeber of kept versions is one lees than the specified (for b.)
And it definitely seems we can discard a.

Proost.

Have one on me.

jpe

[Doug Phillips]

Odd indeed. Doesn't look like any expected result.

It seems reasonable to me that (b.) is the way it should work; consider only those files meeting the /before=date criteria (exclude all other files from consideration) and operate only on those files. I think /before /since should always be the primary selectors.

(a.) doesn't make sense to me.

Compare this to PURGE/SINCE=date/KEEP=n for expected behavior. You would expect anything older that SINCE=date to be ignored. So with your example, PURGE/SINCE=22-JAN/KEEP=2 we should see only X.X;5 deleted, and that's the way it works.

I recall not too long ago, a copy /since /before bug that caused no files to be selected regardless of the dates, even though a dir with the same dates would show the files.

[Jon Pinkley]

Jan,

In your example, it seems that algorithms a. and b. are reversed, i.e. wouldn't a. leave 6 and b. leave 7?

I submitted a problem report in January 2002 for VMS 7.2-2, because I did not feel that purge/keep=(n>1)/before=date did the correct thing.

Call Reference #: C020125-17

I don't think I am allowed to post their entire response, but here is part of it:

"The /BEFORE selects only those files dated prior to the specified date. This sets the starting point for the Purge command and then follows the /KEEP qualifier. This behavior has been this way since before the V6.1 time frame and the entire logic of the facility would need to be rearranged to implement what is eventually determined to be the correct behavior. The PURGE command itself is used so often that the possibility of introducing a situation where files get deleted that were not intended to be deleted is too big a risk for the limited gain."

We ended up using a command procedure to do what we wanted, which was equivalent to skipping the n highest versions of the file, then deleting all other versions of the file if and only if they matched the /before=date.

In any event, what Purge does when /keep is not specified, should be identical to what it does when /keep=1 is specified. Also, /keep=1 should be consistent with /keep=2, and that does not seem to be the case when /before is used.

[Robert Gezelter]

Jan,

I don't have an opportunity to check this out throughly at this instant (a quick break from dealing with re-caulking a shower faucet to stop a leak), however a quick comment about the behavior.

Whatever the behavior is, it should be consistent. In this case, I am referring to how /BEFORE and /AFTER behave, singly and in concert. I would re-run your test cases, and see if the behavior differs for any range.

If it does differ, then it should really not be considered a documentation problem, but a real issue.

- Bob Gezelter, http://www.rlgsc.com

[Martin Hughes]

Tested this on V5.5-2H4, the results were more like you would expect;

$ dire/date x.x

X.X;34 6-DEC-2006 10:49:23.00
X.X;33 1-DEC-2006 09:57:55.00
X.X;32 1-DEC-2006 09:57:29.00
X.X;31 1-DEC-2006 09:56:18.00
X.X;30 1-DEC-2006 09:55:31.00
X.X;29 16-NOV-2006 16:27:29.00
X.X;28 16-NOV-2006 16:25:59.00
X.X;27 16-NOV-2006 16:19:07.00
X.X;3 15-DEC-2006 13:02:25.38
X.X;2 17-AUG-2005 16:22:06.04
X.X;1 13-APR-2005 12:28:16.78
Total of 11 files.

$ purge/before=1-dec-2006/keep=5 x.x
$ dire/date x.x

X.X;34 6-DEC-2006 10:49:23.00
X.X;33 1-DEC-2006 09:57:55.00
X.X;32 1-DEC-2006 09:57:29.00
X.X;31 1-DEC-2006 09:56:18.00
X.X;30 1-DEC-2006 09:55:31.00
X.X;29 16-NOV-2006 16:27:29.00
X.X;28 16-NOV-2006 16:25:59.00
X.X;27 16-NOV-2006 16:19:07.00
X.X;3 15-DEC-2006 13:02:25.38
X.X;2 17-AUG-2005 16:22:06.04
Total of 10 files.

This logic is as per "a." in the original post.

Seems like a bug with later versions. However the logic is supposed to work, the bottom line is files should never be deleted unintentionally.

[Doug Phillips]

Martin's example and Jon's Call response both describe the (b) logic;

>>
b. Scan for any file /BEFORE, and trim those down to /KEEP.
<<

which is how the /before and /since qualifiers work with all other commands.

i.e.

$ PURGE /KEEP=n /BEFORE=time

The command should act as if the "/BEFORE=time" files did not exist.

Jan's test results don't seem to all comply with that logic, though, and I think that was what he was questioning.

[Jan van den Ende]

Jon,

well spotted. Upon re-reading the original statement does look a bit awkward.
So, it HAS been noted before!

Bob,

agreed, it DOES seem like a real issue!

Martin,
So it DID work as expected once!
And I coudn't agree more with your bottom line!

Doug,
you got the point.
Software should behave like what is reasonable to expect, but ANY ambiguity should be decided upon by the ocumentation, which MUST be as correct as the code should be!

Proost.

Have one on me.

jpe