Incremental backup

[geir_2]

Hi,

Sorry for all questions, but I am working with DCL script for incremental backup, and have some problems. The idea is:

I want to take full backup on monday, and the next day I want to take incremental backup, until next monday(weekly backup again, I also think that following qualifiers is usefull /since=backup, /record, /image,
/since="-7-"
/norewind( I want to take incremental backups on the same tape), etc. So far so god.

But I also want to have some control. If the tape drive contains wrong volume, the backup should abort, and the operator informed about it.

The script should also control the weeknumber (1-4), and I assumed that each tape should be initialized with nodename + weeknumber. How is it possible to control the weeknumber?? The DCL script is a typical batch job, and should be a generic job/file.

Today we take full backup every day, but if someone delete a file, it's not possible to recover it, because the tape will overwrite it with new contents. So it's better to have some weeks to recover the file if we do some mistakes. I also think that the tape is big enough to contain full backup + incremtnal backup for one week.


So if it sounds familiar to some of you, and you could share the experience with me it would be great. Thanks

geir

PS:
If some of you have better idea or weel done DCL-script, I would appreciate it.

[Joseph Huber_1]

Controlling the tape volume label:
label = f$getdvi(tape, "VOLNAM")
if label .nes. expected_label then goto errexit

What means weeknumber(1-4) ? Do You want to rotate 4 tapes, one for a week ?
Then easiest is to keep file containing the number, every Monday increment this number, and use (number mod 4)+1 as part of the tape label:
open/read wn weekno.dat
read wn weeknos
close wn
weekno = f$integer(weeknos) + 1
open/write wn weekno.dat
write wn weekno
close weekno
weekno = (weekno.and.3)+1 !modulo 4 + 1
! use weekno to build tape-label


Note, backup tapes are ANSI labelled tapes, and the label is restricted to 6 characters !
So You can't combine nodename and weeknamuber for the tape label, need to develop a different scheme for Your environment.

[Joseph Huber_1]

On the Freeware CD #5 , in directory [SRH_mumble], there is a funny program by Steven Hoffman to calculate a weekly tape-set name. Look into source file DSBD.C .

[Willem Grooters]

FIRST ISSUE TO SOLVE:



If I'm right, you will use the same tape every day.

WRONG. PERIOD.

You'll need to cycle your tapesets - at least one month. in your case it would mean 4 FULL backups, and 6 for incremental (one tape per day) if you cycle incrementals per week. You can choose to cycle week-sets per 4 weeks as well, so you'll have a full backup + all incrementals on a weekly basis.
You could think of a 13-week cycle - requiring even more tapes. It's up to the level of persistance you require.

I would NOT recommend to have all incrementals on one tape. If you loose your tape, you'll loose your backup. Use one tape a day. It's safer, it's easier to administer, it's easier to control.

If you would really need to save tape and have all backup files on one, then make TWO tapes - on separate hardware, with the same procedure. Chances that you lose two tapes in one go, are less.

Next is the matter of control.
Label your tapes in advance, and mark them.
Note, within your cycle, which tape is used on which day in the cycle, and keep this in a file so your procedue can find out what tape should be used. If you cycle your tapes, you don't have to re-label them.

Third: the commands can best be given by the real system managers ;-) but à know that I use:

$ BACKUP/IMAGE/RECORD for a full disk backup
$ BACKUP/SINCE=BACKUP for incremental backup

ALWAYS SPECIFY /JOURNAL and /VERIFY

/JOURNAL will tell you which files are backed up, and where. /VERIFY will give you SOME assurance your backup is good.

/IGNORE=INTERLOCK MAY seem a good idea but it can render your backup useless if it contains files that are interraleted, so your data can be incomplete and, worse, corrupted between the files. If you have an indexed file, BACKUP may render itcorrupt when data is stored and updated dureing backup.

If you backup databases, check if there is an opproturnity to make a hot-standby backup to disk first, and have that backup stored on tape. Often, programs do not have to be stopped during that time.

Willem

[Lawrence Czlapinski]

geir:
Our AUTOSUBMIT.COM of backups, checks whether it is a weekend or holiday.
WEEKLY BACKUPS: You might consider 5 weekly backups. Week 1 is always first weekly backup of a month, etc. Some months you may have 5 weekly backups. The advantage of this method is that you don't need a log to know which weekly to tape to use next or which week is on which tape. For weeklies we use nnjWwt where
nn is a 2 letter node abbreviation
j is job number (some nodes have multiple backup jobs)
4th character is W for weeklies
5th character is the week number
6th character t is the tape number (we have some nodes with multiple tapes)
DAILY BACKUPS:
METHOD 1. 4-6 DAY ROTATION: For dailies, you can use 4-6 tapes (depending on whether you backup on weekends) nnMON, nnTUE, nnWED, etc. skipping the day you do weeklies and any days you don't do a backup (like weekends). This is the simplest way.
METHOD 2. 31 DAY ROTATION: For a 31 day tape rotation, we label nnjddt where dd is the day of the month. You can run an autosubmit.com from your daily_mgr.com file if you use one. The autosubmit.com can calculate what label to use for a given day.

METHOD 3. OTHER NUMBER OF DAYS ROTATION: You can use 2 - any number of tapes you want but you need to keep a record of what the next tape to be used is. You can either use a system logical $DEFINE/SYS BACKUP_SET_MAX max_number (or similar) for the max_number in the tape rotation or have BACKUP overwrite the tape labels. If you use a system logical, you can increase the number of tapes if you need to and still use an autosubmit.com and have the tape labels match the paper labels.
DISADVANTAGE:
1. You need a log to know what tape label was used on what day. When someone needs a file from a certain date, you need to look up which tape the data is on.
2. You need to keep track of which tape is to be used next.
Lawrence

[geir_2]

Hi,

Thanks to all of you, for great answers. One more questions:

If I start incremental backup on tuesday, and some user create a file on wedensday, and delete the same file on thursday. On saturday the same user want to recover the same file. No tape backup will contain the file,(because incremental backup will overwrite the same tape, with new contents for each backup)and I will have trouble if the same user ask me to recover the specific file. Some idea, how to solve the problem?

I also think we should avoid 31 backup tapes, if possible.

Thanks

Geir

[Joseph Huber_1]

geir, NO, don't overwrite the incremental backup, that's completely against the intent of incremental.

Your original scenario was exactly the one which makes recovering accidently deleted or overwritten files possible (except for files created and deleted on the same day of course).
Every Monday You have a full backup, every day You add an incremental, not overriding the ones before. So You can restore from the incremental saveset of the day when the good file still existed; more precisely, the creation or modification day.

With the 4 or 5 tapes rotating You have a full month to recover.

[geir_2]

Hi,

Thanks for the answer. My problems is that we have 4 servers that I want to backup. I hoped that I could avoid new tapes for each incremental backup. (6 x 4 = 24 tapes only for incremental backup)+ 4 week backup tapes. That the reason why I said that it possibly would be better if I used the same tape for incremental backup tuesdag - to sunday ( with qualifier the /norewind, then it's possible to have 6 saveset + week backup on the same tape volume. So somebody have different suggestions, I will appreciate the solutions.

Geir

[Ian Miller.]

Tapes are much cheaper than the value of the data you are backing up. Try and convince the bean counters of this.

[Joseph Huber_1]

Mybe just a misunderstanding, but I didn't propose to use a different tape for each daily incremental. A days incremental is probably only in the percent order. Only the full backup should be separated from the incremental tape volume. How much You distribute the incremental savesets should be calculated/decided by:
The amount of data changing over a week ?
How many tape drives available ?
Manpower available to change tapes ?

Some rised this question of security in case of adding several backups on one tape cartridge: what is the probability that a tape is destroyed AND the disk backed up within the same week fails ? If the value of Your data is high enough, then use as many tapes as possible.

And depending on the data-change rate, how about doing the daily backup to a separate disk, backup this disk to tape once a week together with the full backup ?
For the price of a few SDLT cartridges You can buy a 250GB disk

[Willem Grooters]

I double Joseph. backup to disk is a magnitude faster than backup to tape - and at least a lot more reliable (less tear & wear). Worthwhile investigating!

Willem