Backup with append feature

[sen_ux]

Hi All,

Can someone suggest a HP-UX native method to backup large (> 10G) files with append feature. I have large DB exp backups. I want those files to be backed up to a tape everyday.It should be appended each day. Can be overwritten after a week backups.


Thanks
SEN

[Dennis Handly]

This is very dangerous unless you have a commercial product that will do that for you.
fbackup(1m) will not do that. tar/cpio/pax (with mt(1)) are very error prone and dangerous.

[DeafFrog]

Hi Sen ,

I have been using compress command to export file , this reduces the size atleast by 50% in most of the cases.you can use fbackup to transfer single file greater than 10G.Again , i would go with whad Dennis said.

Regards,

[Bill Hassell]

Appending backup sessions will eventually lead to catastrophic situation because someone make a mistake. To append anything to a tape requires exact records of what is currently on the tape, then issuing the correct mt command to forward-space-files (fsf). If you have 3 backups on the tape, you must issue the fsf 3 command. If someone forgets to follow the instructions, the entire contents can be erased and only one backup will exist. If you do not take extrodinary steps to verify each backup, keep accurate records and accurately label the tapes, you WILL lose data someday, and it could be dozens, even hundreds of GB of data, never to be recovered.

It is a very bad idea to try to save money on tapes by appending. I am assuming that your data is far more valuable than a few tapes, especially when you must recover files from your backup.

[Terry.giblin]

Dear Sen,

Backup, backup, backup.

That being said, I have been faced with a similar problem and after a lot of arguement and discussion, we did design a backup using the append option. In my case I think it was for the Oracle redo logs.

Tapes are expensive and storage is a nightmare.

Speak to your manager and the DBA's and get their opinion and approval. How critical is the data, we are taking about.

Then speak to your SA's and Operators, because after all, they are the ones who are in charge of the backups.

We eventually choose the internal 'dat or exebyte' tape drive to use. The tapes were small and cheap and no one, apart from the engineer normally used the tape drive.

The backup script does take a little thinking about and testing.

Just one possibility, to consider.

Terry Giblin

[Steven Schweda]

> [...] To append anything to a tape requires
> exact records of what is currently on the
> tape, [...]

Are you serious?

In my experience, to append anything to a
tape requires advancing to the end of what's
currently on the tape, which can be done by
reading the tape, rather than by consulting
any other "exact records" of anything. I'll
admit that this is approximately easy and
reliable when using decades-old technology,
such as, say, VMS BACKUP, with its
/[NO]REWIND option, but a pretty simple shell
script on a UNIX (-like) OS should be able to
handle the job, too.

Restricting a tape to holding only one file
is not very far from restricting a disk to
holding only one file. The absence of good
software is principally what makes one more
dangerous than the other. (I claim.)


> We eventually choose the internal 'dat or
> exebyte' tape drive to use. [...]

If I were serious about retaining my data,
then I'd probably choose a tape technology
from a family which was designed for computer
use, like say DLT (and its descendants)
rather than one designed for audio/video use
and later adapted for computer use.
Half-inch tapes are less cute, but they do
seem to get mangled less than the cuter
(gossamer) 4mm and 8mm tapes with their
complicated helical-scan tape transport
mechanisms. Call me old-fashioned, but all
my Exabyte 8mm drives are in my junk pile.
(And that's my disused-junk pile, not the
pile of junk I'm still using.)

[Dennis Handly]

>Bill: Appending backup sessions will eventually lead to catastrophic situation because someone make a mistake.

Right, I learned that the hard way decades ago in high school, using IBM JCL.

>To append anything to a tape requires exact records of what is currently on the tape

Actually EOF marks or a convention of two EOF marks in a row.

>Steven: which can be done by reading the tape

Yes but the software that would read it and the software that writes may be separate.

>I'll admit that this is approximately easy and reliable when using decades-old technology, VMS BACKUP

This would be that commercial product that's made for it, that I mentioned above.

>but a pretty simple shell script on a UNIX (-like) OS should be able to handle the job

I'd trust a program better than a simple script.

>The absence of good software ...

Exactly.

[Terry.giblin]

It is true, I have been lucky, I have only ever do this type of backup on one site, in my whole professional career.

And I would not recommend, only as a last restore, any backup is better than no backup at all.

There are only so many hours in the day you can do backups, with limited tape drives, sometimes you don't have any other option.

"Needs must" and "the customer is always right", also comes to mind.

Not everyone lives in a perfect world, sometimes we have to do the best with what we have got and to make the best use of them where possible.

The site in question was multi-vendor sites, and which type of 'hardware and software' you are using is very important.

If everything in life was so simple.

Terry Giblin

[Hakki Aydin Ucar]

in the first place ,I think you can use Data Protector backup utility if you have.
and besides, which type of tape media you use ? when it comes to DAT you need at least DDS4 to support 40GB compressed mode.
if you use ultrium tape media your limits are better..

[Dennis Handly]

Another important issue is how do you get the data off of the tape when you need to restore?
You would need to know the position of the last? backup.