fbackup file > 2GB, but wont fit on DDS2 Tape

[Gus Mestousis]

Same issue different problem now... As I said before in my last post "fbackup files greater than 2GB" it is a DDS3 Drive, writing to a DDS2 tape, capacity is 4GB/8GB compressed. The file is slightly larger than 3 GB. When I do an ls -al it comes up as 3809393609 in size. The fbackup prompts me for a second tape. I should be able to fit this all on one DDS2 tape, should I not? Is there any additional option I need to pass to fbackup? I know I cant use tar, because the files is greater than 2GB. Any ideas?

Thanks again!

[Michael Tully]

Hi,

I am not sure what the problem is exactly, but... let try something first... From what I
recall from your last post the file was in
compressed mode. Suggest you get better compression first by uncompressing and using 'gzip' as it is far more efficient, then placing it onto the tape.

# uncompress file.Z
# /usr/contrib/bin/gzip file
# fbackup -i filepath -f /dev/rmt/0m

You will find that using software compression
tools are far better than attempting to use
hardware ones.

-Michael

[Printaporn_1]

Hi,

1. Are you use compress device file ?
2. Does this file was accessed during backup time ?

[Gus Mestousis]

I uncompress the file, and then try to gzip it, but this is what I get:

# gzip backup.tar
backup.tar: Unknown error

here are the details on the file:

-rw-rw-rw- 1 root sys 7777423360 Jan 25 14:14 backup.tar

Any ideas?

[John Palmer · Accepted solution]

The 'standard issue' gzip in /usr/contrib/bin doesn't support files > 2Gb either!

I recommend that you download gzip 1.3.1 from the porting centre. This does support large files.
http://hpux.cs.utah.edu/hppd/hpux/Gnu/gzip-1.3.1/

Regards,
John

[John Palmer]

One other point more relevant to your original problem is that if by default, fbackup doesn't write large enough blocks to make efficient use of the tape.

Due to this wastage, it's quite possible that it can't write a 3.8Gb compressed file to a 4Gb capacity tape.

There have been several posts recently that recommend settings to configure fbackup for performance, particularly setting the blocksperrecord parameter which defines the tape blocksize. A quick search of the forums will show you these.

Regards,
John

[harry d brown jr]

You probably have to make some kernel changes to allow you to handle that size of file. What are your values for shmmax, maxtsiz, and maxdsiz??

Mine are, for 11i:

maxdsiz -------> 0x40000000
maxdsiz_64bit -> 0x40000000
maxtsiz -------> 0x4000000
maxtsiz_64bit -> 0x40000000
shmmax --------> 0x40000000

Also, use the "gzip -9 filename" method.

live free or die
harry

[Gus Mestousis]

For HP-UX 11.00

maxdsiz -------> 1073741824
maxdsiz_64bit -> 1073741824
maxtsiz -------> 67108864
maxtsiz_64bit -> 1073741824
shmmax --------> 1073741824

[James R. Ferguson]

Hi Gus:

The 'fbackup' configuration parameters are enumerated in the man pages for 'fbackup'. A default set is used if the 'config' file is absent, and are non-optimal.

I don't have any magic values for you to use. As for 'blocksperrecord' I find that values of 64 or 128 with a 'checkpointfreq' of 128 or 256 accordingly work well.

For more , an excellent document is #B00000196 "Understanding Fbackup/Frecover and Optimizing Backup Performance" (attached for your convenience).

Regards!

...JRF...

[Bill Hassell]

If you are using fbackup without the optional config file, that is might be your problem. The default parameters (shown in the man page and the same for SAM) are designed for 1/2" magtape, not for DDS or DLT drives.

Create the following file:

blocksperrecord 256
records 32
checkpointfreq 1024
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

Save it as something like: fbackup.config

Now, always run fbackup with -c

There are several advantages:

- A larger record size will be used which more closely matches modern tape capabilities

- A larger block size will be used to minimize I/O channel overhead with smaller blocks

- Checkpoint frequency and setmarks require a large amount of tape and when too small, cause a lot of wasted tape.

- Multiple reader processesd (6 in this case) means that multiple files will be opened at the same time to allow for overlapping I/O and thus reduce data starvation (which slows the tape down significantly).