massive FLAW in fbackup?

[Steve Post]

I like fbackup. It seems to work fine. But I just stumbled upon a massive flaw in it.

Perhaps I'm doing something wrong? PLEASE tell me I'm wrong and why. Here's the flaw:

I have a massive sDLT tape. I can probably store 40 backups from "fbackup" on it.
....Well....maybe not.
You see. It looks like I can only store ONE and ONLY one backup per tape.

This isn't true is it? I can't believe such a nice utility would have such a glaring hole. It's like having a school bus that's empty except for one seat.

PS: I'm not talking about hpux OS, or patches, or trying omniback, or dataprotector, or tar, or cpio, or some fancy perl script, or dd, or mirroring disks, or breaking mirrors, or anything else. I just want fbackup to let me backup my system without wasting tapes.

Steve

[Pete Randall]

Steve,

The man page, in the WARNINGS section, explicityly states that

"fbackup should not be used with no-rewind devices, for example, /dev/rmt/0mn."

I would think that precludes putting multiple backups on one tape! What a waste.


Pete

[Jeff_Traigle]

That's the way it works. One backup per tape. No exceptions. A flaw? Not really. Just the way that particular tool was designed to work. There are others (e.g. tar, as you mentioned) that will allow you to manipulate the tape with the mt command to write multiple archives to a tape.

[James R. Ferguson]

Hi Steve:

Sorry, no cigar. 'fbackup' by design, rewinds a tape and writes from the beginning. In part, this occurs so that an index file of what's on the tape can be written to the beginning. This index is built based on the sorted, merged inclusions and/or exclusions of the graph file driving the backup.

The layout of an 'fbackup' tape also includes markers to allow very rapid tape positioning to aid in finding files for recovery.

When a copy of a file is about to be written to a tape, a checksum for the file is computed. Upon completion of the transfer of data to the tape, the checksum is computed again. Should the values differ, the current tape copy is marked "bad" and a new copy is attempted. This occurs 'maxretries' times as controlled by the 'fbackup' 'config' file. (see the 'fbackup' manpages for more information). This mechanism insures that you backup a consistent, static image at a point in time.

While it is true that small backups may waste tape, the ability to do high-speed recovery; examine a tape's index without having to read (sequentially) every tape block; and the assurance that if you extract a file for recovery that you have a static version of the data; in my opinion, far outweigh any tape "waste".

Regards!

...JRF...

[A. Clay Stephenson]

This ins't a flaw; it's a feature -- but it's a well-documented feature --- and, it's free.

Do yourself a favor and move up to a much better solution such as Data Protector and backups become something that you can (almost) forget and restores become point-and-click.

[Steve Post]

Ok. It's a feature. It's fast to recover. But it has a price. That price is I can only save 20 gigs on a 320 gig tape.

And I can also conclude that incrementals act the same? ie ONE gig on a 320 gig tape?

fine. At least I got my answer.

[A. Clay Stephenson]

Yes, incrementals need a new tape as well. See how feature rich fbackup is?

You could use tar but be aware that even the patched version of tar imposes an 8GB limit on the size of individual files.

I've used OmniBack/Data Protector for many years and the use of one product to manage all your backups (UNIX and PC's) is very appealing AND DP has a feature that does allow the use of full and incremental backups on the same medium (or you can choose to not allow it).

[Steve Post]

I tried out Dataprotector. It does the job perfectly.

I could setup a cpio backup routine. But any files over 2gig's get thrown out. I have big files. So idea is no good.

And, like you said, tar has an 8gig limit. I know I have ten 11 gig files. So tar's out too.

Fbackup would work. But I don't want to waste tapes.

So either I HACK around my problem, or use Data Protector. I'll use data protector. I don't like the idea of building some half-baked, home-grown routine for production work.

Steve

[A. Clay Stephenson]

If you are going the DP route then you should really look very hard at getting a tape library so that you can fully automate the process. Last year, we purchased a used StorageTek DLT library on eBay for our dedicated SOX lab for < $1K; the original identicaL unit was well over $150k. The $1k model has functioned flawlessly.

[Rick Garland]

If you prefer the free solution, look at using vxdump and dump utilities. Can script these to work with incremental backups as well as full backups.

[Bill Hassell]

Some background on why multiple backups on one tape is a very bad idea. To use your school bus analogy, multiple backups are like a school bus with doors every few rows. You can't tell if the bus is full, half full or just a few students. And to get the students off the bus, you have to open each door to see who's there. Any seasoned sysadmin (that created multiple backups per tape) will tell you of horror stories (like yours) where there were supposed to be 6 backups on the tape but there appear to be only 3. Or 40 backups on one tape were instantly wiped out because an operator used the wrong device file. Or the tape that has 40 backups has no label and no one knows which tape has which backup.

AS you can see, it is a clerical and operator nightmare. Back in the days of 100 Mbyte workstations, tar and cpio made sense as well as backing up a few megs at a time by appending. Not too much to track so the risk was low. But those techniques don't scale and the impact of bad backups is massive, affecting many gigs of data. And as most of use have found out, systems grow, people change jobs and there are never enough details to pass onto the new hires.

fbackup is the closest you can get to a commercial backup tool like DataProtector. In a commercial or production environment, the accuracy and reliablitlity of the backup is of primary importance. So record keeping is a critical part of the strategy. Rather than create a massive database (like DataProtector), fbackup stores all the details in the index on the front of the tape. And this index intelligently spans multiple tapes. Each successive tape has the entire index plus a marker for each previous tape. This allows restoring any file on a 5 tape backup in just a few minutes by starting with the last tape.

[Florian Heigl (new acc)]

Wow, I didn't yet know fbackup is no friend of no-rewind devices.

now I still agree on the point 1 backup/tape for occasional use of fbackup, and that's what it's intended for, at least from my point of view.

let me step ahead and take the chance of announcing my recently compiled bacula binaries of hp-ux.

- single-shot reliable filesystem backup: fbackup
- no-cost enterprise grade (or getting there, by user base) backup tool: bacula.


http://deranfangvomen.de/~floh/bacula/

Best wishes,
florian

[Tim Medford]

My pet peeve has always been that you cannot use fbackup to create an image on disk, and THEN move that image to tape. The original target of the image must be the tape.

fbackup definitely has it's limitations, but overall I think it's a good, reliable utility. And hey, at least it's free!

My 2 cents.

[Patrick Wallek]

Actually that is not true. fbackup can write to a file.

You can do something like:

# fbackup -f /dir/somefile.fbackup -i /home

That will create a backup of /home in the file somefile.fbackup.

[Tim Medford]

Patrick - What I'm saying is you cannot later take that disk image (created with fbackup) and archive it to tape.

We wanted to write our original backups to disk and then LATER write them to tape. Then use frecover against that tape at some later time.

This does not work from everything I've tried.

[Patrick Wallek]

Ahh...I see what you are saying. Once you move the fbackup file to tape, you'd have to restore that file to disk and then restore the file you want out of the fbackup image. Essentially a 2 stage restore process.

I haven't tried it, but I would think something could be done with dd and writing the fbackup image to tape.

[Steve Post]

I should provide just a TWEENY bit more of information.

I have TWO servers. The primary box has the 320 gig tape. I WAS planning on running fbackup on one server (200 gigs), then fbackup via secure shell to for the 2nd server (20more gigs). One tape. One backup. But for two servers.
See? 200 + 20 < 320. But I would have both backups one on tape.
Now that I can't do that.
I would have one tape with 200 gigs on it.
And a second tape with 20 gigs on it.
I would call this....a waste of space.

But I agree this limitation of fbackup is not that bad. It's just a different tool to use. I wouldn't mow my lawn with a scissors. I wouldn't shave with a lawn mower. So fbackup appears to be a great way to backup a box. And a bad way to backup two servers onto one tape.

As far as backing up more than one thing onto a tape? That's a different, opinionated discussion. I got my answer.

[Tim Medford · Accepted solution]

Steve - Sorry to keep beating a dead (or dying) horse here....but maybe you've been following the conversation I've been having with Patrick.

What you could do is use fbackup to create 2 separate archives on disk (if you have room). You could then use tar, dd, cpio or something to get those fbackup disk images on a single tape.

Of course, if you ever need to recover you'll need to first restore the fbackup image to disk using the same utility (tar, dd, cpio), and THEN you can use frecover against the disk image.

Just a thought. Ok, I'm done.