Mark a DIRECTORY as NOBACKUP?

[Jan van den Ende]

Situation:

VMS 7.3-1
Occasionally BACKUP does not fit in one tape, and next morning an operator has to supply a continuation tape and resume BACKUP.
This is a 24 hour shop, most of activity during office hours, but non-negligeable activity at night also.

The consequences: Far less consistent backups as would have otherwise been possible, and severe performance degradation in the early office hours.

Investigation showed the following:
Also during the night a batch job collects all kind of mutations and formats them in "reports" in a format acceptable to another (non-VMS) system.
Those reports are collected into one dedicated directory, and when reporting is done, they get tranferred to the other system. If that is successfull, they are deleted.
Unvariably the tape-full situation happened when this temporary directory had much data in the backup.
And the value of having it in the backup is nill....

BACKUP without /IMAGE but with /EXCLUDE would:
- make the backup procedure much more complicated (now it just processes all whole disks one at a time)
- take much more time

The simple solution looks to be set the files /NOBACKUP.

I do not know the exact programs that make the extractions, they are from a 3rd party.

We tried setting the .DIR file to /NOBACKUP, which is happily accepted, but BACKUP still backups the files (which would be a minor nuisance) AND the data, which is what we try to avoid.



Any ideas anyone?


Proost.

Have one on me.

jpe

[Uwe Zessin]

Can you do a dumb:
$ set file/nobackup ...

just before the BACKUP starts? It won't hit open files I guess, but should turn down the amount at least a little bit.

[Jim_McKinney]

> setting the .DIR file to /NOBACKUP

If the goal is to exclude the data in the files that are "pointed to" by the directory then you'd need to set the NOBACKUP attribute on each of those file individually.

[Jan van den Ende]

Uwe,
Trouble is, there ARE no files yet when BACKUP starts.
And they may already be gone when the BACKUP of THAT drive starts (which is the situation when things go well).

We HAVE already questioned the correctness of the whole setup, since the "moment" of backup of related files is streched for hours, so integrety of a restored system is questionable anyhow.
But the applications are being transfered "ASAP", so any investment in the system will be overruled.
I am rather pessimistic about the duration of ASAP, but it is "politically incorrect" to say so. :-(

Thanks anyway Uwe, but no solution.

Proost.

Have one on me.

jpe

[Walter Miller_1]

I thought the first thing backup did was read indexf.sys and any files created after that scan would not be saved during an image backup. This would make Uwe's suggestion seem appropriate.

[Jim_McKinney]

> during the night a batch job collects all kind of mutations and formats them in "reports"

Perhaps you could modify the batch job code to set the nobackup attribute on each file as it moves them into your target directory?

[Jan van den Ende]

Jim,


If the goal is to exclude the data in the files that are "pointed to" by the directory then you'd need to set the NOBACKUP attribute on each of those file individually.

Exactly that was the intention: hoping it would be ingeretid. Not so. :-(
Ane ACE to trigger that would also be welcome, but I know not of it.

Second idea:
something along those lines is the ugly, hard way to get (near) there. It might well be where this will lead to, if this collective has no more elegant solution.

Walter:
but those scans are performed per disk, therefor, rather spaced in time.


Proost.

Have one on me.

jpe

[David B Sneddon]

Jan,

Is it possible to set up a "scratch" disk for
these files and not backup that disk? You could
also use it for other junk.
Just a thought...

Dave

[John Gillings]

Jan,

Directory files are ALWAYS backed up, regardless of backup attributes. This is required to ensure incremental restores are done correctly.

At the moment there is no way to specify that files created in a particular directory should have the NOBACKUP attribute by default. It has been requested in the past. If you log a case and lodge a formal request for an enhancement, you will increase the weight of the request. Since NOBACKUP is ignored on directory files, the attribute could be used to specify a default for files within the directory.

In the mean time, you could place an ACE on the directory like:

(IDENTIFIER=BACKUP_PROCESS,OPTIONS=DEFAULT,ACCESS=NONE)

and grant the identifier BACKUP_PROCESS to the process that does your backup. Make sure you remove BYPASS privilege. This should prevent the backup process from reading the files. You'll get error messages which will look ugly, but it should achieve what you want. To reduce the number of errors you could block access to the directory from the backup process itself perhaps?

Other than that, you'll need a /EXCLUDE list.

[Steven Schweda]

> Since NOBACKUP is ignored on directory
> files, the attribute could be used to
> specify a default for files within the
> directory.

I've wished for this many times, but I'm not
a profit center.

[Volker Halle]

Jan,

would it be possible to create and mount a LD container file (set /NOBACKUP) and have the temporary files be created in a directory on that 'disk' ?

Volker.

[Jan van den Ende]

David,
that would require investing in an extra disk. Because the current dusk connectivity is exhausted, that would require adding an extra controller (which I do not even know if a slot is available). See above, no way!

John,
request coming up.
Setting NO access to the DIR file will not help much. It is not only BYPASS, which the account does not have, but also READALL, which it DOES have, and VERY probably needs.
I don't think I even DARE try without.

Volker,
THAT might be an idea! I will pass it on, and if they do it and it works, I definitely will report!

Proost.

Have one on me.

jpe

[Chris Davis_9]

Hi Jan,

You say that a batch job produces the report files in this directory.

Would it not be possible to either:

1. Modify the batch job to set the /NOBACKUP flag on the files after creation

2. Have another job run (batch or detached) which sets the /NOBACKUP flag on any files in

the directory. This could be set to start at a specific time, run until a specific time, checking every 30 seconds, 1 minute or whatever.

In our setup we have some detached jobs running every 30 seconds or so to check if there is any work for them to do.

Some check specific database files for header records showing that a batch of work is

complete, allowing us to separate printing tasks from data entry programs.

Some check specific pathworks directories for files of a certain specification. This

allows us to automatically process files submitted by PC users.

a simple

$LOOP:
$ set file /nobackup [dir.subdir]*.*;*
$ wait 00:01:00
$ goto loop

...either running constantly as a detached job or submitted to start at a specific time
and with a time check in it should do the job.

Regards

Chris

[Robert_Boyd]

If you really wanted to get carried away with a process that set the files to /NOBACKUP status as each one was put in place and can't modify the batch script that creates them:

1. Create an identifier like NO_BACKUP_DOORBELL

2. set appropriate access control entries on the directory to trigger an alarm for successful writes

3. Create a detached process to listen for security alarms for the NO_BACKUP_DOORBELL identifier. When an alarm comes in, set the proper file(s) to /NOBACKUP.

It would obviously be much simpler to be able to alter the create so that the script does this directly rather than having to write something fancy to clean up after the script.

Robert

[Jan van den Ende]

All,

I entered an enhancement request in Advocacy.

As soon as it gets a number I will report here again, asking for your "I agree"'s.

In the meantime, the "watching procedure" road was chosen.
Ugly, but workable.

Proost.

Have one on me.

jpe

[Klaes-Göran Carlsson]

Hi all

Could a backup/before be a solution?

[Jan van den Ende]

Hi all,

if you think you might have some future use for this:

http://www.hpuseradvocacy.org/node/1987

Add your "I agree" and report here, please.

Proost.

Have one on me.

jpe

[Jan van den Ende]

ping

Proost.

Have one on me.

jpe

[Duncan Morris · Accepted solution]

I have added my support on the advocacy site.

[Ian Miller.]

been there, done that :-)

[Jan van den Ende]

ping

Proost.

Have one on me.

jpe

[John Abbott_2]

Added my support.

Regards,
John.

[Anton van Ruitenbeek]

Also add my support and possible suggestions.

AvR

[Willem Grooters]

I had to move two applications off one disk, and this would have made my life a lot simpler.
So I gree on this.
But it should be sufficient, I think, to mark the directory as "nobackup", causing all subsequent files be "invisible" for backup.