Disappointed with backup !!

[YMJ]

Hi again

I have opened a thread few days ago (see http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=846390 )concerning knowing the size of total data before doning the backup.

the problem is I have DSS3 disk drive that can store 12 GB (24 GB compressed). with the command "du -ks" the total size 11044372 KB or 11 GB. Doing the full backup with sam requires two tapes which disapponited me.

Two question:
1- how can I check the total data size with sam after the backup?
2- Is there a difference between the backup with sam and the fbackup.

THANKS

[harry d brown jr]

Why use sam to perform backups? I don't of anyone that uses that, and I'm not even sure what sam is doing under the covers.

I bet you could EASILY "tar" your data. Give it a try.

btw, 11044372 KB does NOT equal 11 GB, it's more like 10.53GB.

Also, are you doing hardware compression?

live free or die
harry d brown jr

[Stuart Whitby]

I'm confused about the kind of hardware you're actually talking about here.

DSS3 disk drive? Or are you talking about a DDS3 tape drive? I have no idea how SAM backup works, or how it would work copying it to another disk, but I understand tape, so I'm going to make the assumption that's what you're talking about and try to sound intelligent :)

The DDS3 tape drive can store 12GB native, but compression depends on the kind of data you're backing up. You'll *typically* be able to get twice the normal amount of data onto a tape drive, but there are no guarantees. Back up near empty database files with compression and you may get well over 100:1 compression. Back up jpg or mpgs and you'll be lucky to get 12:10.

There are a few other problems you'll be hitting too. Tape media tends to be different lengths, so you can't guarantee a set amount on each tape. You'll also find that the backup will need to store its own information, so while the physical data in the files may be 10.5GB, you've also got a pile of pathnames, permissions, owner info and other metadata stored on there as well which may take up the other 1.5GB. The smaller the files you're backing up, the bigger the percentage of tape data is taken up with this. Also, each file will take up a block on tape, and if your blocksize is 64KB, then a 1KB /etc/hosts file will still take up 64KB on tape. Take the blocksize down and you slow down the drive speed by asking the drive to insert more block marks, which also take up more space (but nowhere near 32K - we're talking tiny amounts here).

I'd go with the suggestion to use tar. Depending on what kind of system you're backing up, I'd go for a script like the following (untested):

#!/bin/ksh

tar cvf /dev/rmt/0cbn /usr
tar cvf /dev/rmt/0cbn /export
tar cvf /dev/rmt/0cbn /etc

and so on. Or better, go for a copy of Amanda. Haven't used it, but it's free and sounds fairly good. Or get a decent commercial backup solution such as NetWorker/NetBackup/Data Protector etc, which will get you through those blockmarks faster to get your data back a whole lot quicker.

Stuart.

[Bill Hassell · Accepted solution]

SAM uses fbackup and does not do anything special. Now as is true with all backup programs, there are headers for each file and for fbackup, a compete index of everything that was saved. This can be a very large file if your system has 100's of thousands of files. This index is very desirable but not found in tar or cpio or and of the classic Unix backups. Ineed, fbackup is the closest you can get to a commercial-quality backup program. Not only does every tape have an inde, but any file can be retrieved in just a few minutes fbackup has error recovery in case of a bad spot, it has changer capability, and most important, it supports all file sizes (hundreds to thousands of Gb).

So the first thing to do is to earn the command line options--and most important, always use a config file (-c config-file). Otherwise, you'll waste a lot of tape and the backup will take a lot longer. The config file looks like this:

blocksperrecord 512
records 64
checkpointfreq 1024
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

Then use the -i and -e options as needed and try your backup again. Note also two very useful commands:

frecover -V - -f /dev/rmt/0m

(for a header listing)

frecover -I /tmp/myindex -f /dev/rmt/0m

For a complete index. And as always, very that the tape device file you are using reports "best density" using the lssf command.

[YMJ]

Hi all

Sorry, DDS3 not DSS3, it is just mistyping.

the reason for using fbackup or sam is the indexing ,as Bill said,specially when your data is huge, the indexing is important. tar or cpio doesn't do that.

but what makes me confusing is my total data is 11 GB and can't fit into 24 GB compressed tape. Bill I can't imagine the rest of the space will be used for indexing and still not enough. unless sam or fbackup doesn't do compression. As far as I know, It does do compression by default.

my server is old HP 9000, with 4 processors and 400 MB memory. HP-UX version 10.

[john korterman]

Hi,
are the data absolutely stable? It could be that fbackup makes more than a single attempt at backing up one or more of your files....

Fbackup checks the timestamp of each file before and after copying it to tape. For a big file, it also checks after having written some data to tape. If the timestamp has changed, the maxretries parameter decides if a new attempt should be made: either until a successful copy is achieved, or the limit for maxretries is reached. Fbackup consumes some tape in doing this, as it does not rewind after a failed attempt, but rather continues from where it was.
This makes it difficult to foresee the amount of tape consumption for fbackup.

Do you see any warnings concerning this behaviour - particularly for huge oracle files?

regards,
John K.

[harry d brown jr]

you can also use GNU tar: http://hpux.cs.utah.edu/hppd/hpux/Gnu/tar-1.15/

and with the --verbose option you can build your own "index" and you can use --gzip to compress the data on the tape.

live free or die
harry d brown jr

[Bill Hassell]

> Bill I can't imagine the rest of the space will be used for indexing and still not enough. unless sam or fbackup doesn't do compression. As far as I know, It does do compression by default.

Very important: Neither SAM nor fbackup (or tar or cpio, etc) do compression. You enable compression by using the correct device file. When you run fbackup, what is your -f parameter? Then run lssf against that device file. For example:

lssf /dev/rmt/0m
stape card instance 2 SCSI target 0 SCSI LUN 0 at&t best density available at ad
dress 10/12/5.0.0 /dev/rmt/0m

The above indicates that this particular device will use compression ("best density").


> my server is old HP 9000, with 4 processors and 400 MB memory. HP-UX version 10.

400 megs is very, very small. There is a possibility that with such small memory, the extra reader processes (2 by default) and the amount of shared memory used by fbackup (which is proportional to the total number of files that are backed up) is causing serious paging (swapping). What this does is to stop processes for long periods of time. And for any modern tape drive (DDS, DLT, etc) this causes the tape to stop, backup and get ready to record more data. This causes significant delays (10:1 to 100:1) and excess wear on the tape drive.

I assume that you can't add more RAM or upgrade from the obsolete version of HP-UX. You can try using tar which has a lower footprint in RAM and should be able to keep the tape drive busy.

I would suggest downloading the tapeinfo program from: ftp://contrib:9unsupp8@hprc.external.hp.com/sysadmin/programs/tapeinfo/
This program will report on the tape's capacity, the compression ratio (read and write) and the actual data transmission to/from the tape.

[Michael D. Zorn]

What about Omniback? We got it as part of the package, but never seemed to need it. Our syetem isn't that complicated, and fbackup works fine for us.

Is anyone using Omniback (and is it still supported - or an HP product)?

[Pete Randall]

Yes, HP still has OmniBack, though it's now called DataProtector. However, in this case, with only 11Gb of data to handle, DP seems like overkill to me.

http://h18006.www1.hp.com/products/storage/software/dataprotector/index.html



Pete

[Marlou Everson]

I used fbackup for years and it works fine. Make sure that you are using DDS3 tapes in your DDS3 tape drive. I could always tell when Operations put a DDS2 tape in because the fbackup wrote to more tapes and took longer.

Marlou