topic No Compression, DDS-3 in Operating System - HP-UX

No Compression, DDS-3

Tom Dawson — Mon, 30 Sep 2002 12:41:19 GMT

All,

I'm having a problem with tape compression on a D330. Actually, it looks
like I'm not getting any compression. I have two D330's, both with the
same hardware configuration. Both have a total of approximately 20 Gb of
disk space used. One server will fit a full ( fbackup ) backup onto one
DDS-3 tape but the other server goes to two DDS-3 tapes. Looking at the
stderr from fbackup, the tape swap comes at about half-way through the
backup. It's not just the last few files rolling over onto the second
tape, about half the data goes on the second tape.

I have two tape drives that I have interchanged between these servers.
One is a Single DDS-3:

8/16/5.1.0 tape HP C1537A SureStore DAT24

The other is a DDS-3 AutoLoader:

8/16/5.1.0 tape HP C1557A SureStore DAT24x6

I get the same results with both drives on either server. I have, of
course, tried new tapes. And I have ensured the tapes are rewound before
the backup starts. The backup in question is a full/cold/weekly backup
where the Oracle database has been cleanly shutdown and there is no
activity on the system. I have increased the "blocksperrecord" and
decreased the "maxretries" in /etc/sam/br/fbackup_config:

Original /etc/sam/br/fbackup_config:

flo1q05 /etc/sam/br # cat fbackup_config.20020915
blocksperrecord 32
records 32
checkpointfreq 32
readerprocesses 2
maxretries 5
retrylimit 5000000
maxvoluses 100

Current /etc/sam/br/fbackup_config:

flo1q05 /etc/sam/br # cat fbackup_config
blocksperrecord 128
records 32
checkpointfreq 32
readerprocesses 2
maxretries 1
retrylimit 5000000
maxvoluses 100

Both /etc/sam/br/fbackup_config files had a "chgvol" entry when the
AutoLoader was attached:

chgvol /etc/sam/br/chgvol

The "AutoLoader Options" on the DAT24x6 is set at 7. But I have the
problem even when using the single drive DAT24.

The server experiencing the compression problem is a "make_tape_recovery"
clone of the "good" server. I have manually removed the special files
associated with the external tape drive:

rmsf -v -H 8/16/5.1.0

and then reboot the server to ensure that the proper device files
are built. I have tried using both /dev/rmt/1m and /dev/rmt/c2t1d0BEST
as device files. The fbackup first goes to the external drive,
/dev/rmt/c2t1d0BEST, and then to the internal drive.

I have reached the limits of my knowledge of what to do to resolve this
( lack of ) compression problem. Does anyone have any suggestions?

Thanks,
Tom

Re: No Compression, DDS-3

Dietmar Konermann — Mon, 30 Sep 2002 12:46:15 GMT

Tom,

the problem sounds really weird. I would start trouble-shooting using the "tapeinfo" tool which can be used to dump compression statistics of tape drives. It typically shows what amount of data went via SCSI to DC and what from DC to media.

Please let me know if you need this tool.

Regards...
Dietmar.

Re: No Compression, DDS-3

Leif Halvarsson_2 — Mon, 30 Sep 2002 12:57:46 GMT

Hi

Is it possible to swap the two drives to get an idea if it is a drive or configuration problem.

Re: No Compression, DDS-3

Vincent Farrugia — Mon, 30 Sep 2002 13:03:55 GMT

Hello,

Are the servers backing up identical data? What is the "good" server backing up?

Vince

Re: No Compression, DDS-3

A. Clay Stephenson — Mon, 30 Sep 2002 13:22:56 GMT

Actually this makes perfectr sense. The problem does not move with the tape drive but rather stays on the 'bad' server. The 'problem' is not really a problem but rather a characteristic of the data that you are trying to backup. A DDS3 tape has a native capacity of 12GB; the 24GB capacity ASSUMES a 2:1 compression ratio. The only capacity that you can depend on is the native capacity; anything over that is just 'gravy'. Some data will compress far more that 2:1 and other data might only compress 1.1:1 - it just depends.

If you are using the cXtYdZBEST device node then that is instructing the drive to use compressed mode.

Re: No Compression, DDS-3

Leif Halvarsson_2 — Mon, 30 Sep 2002 13:39:25 GMT

Hi
I agree with Clay in that compression ratio is hightly data dependant but it seems strange, if Tom is right, he get almost no compression at all. I assume the data on the disk is not already compressed. Even with my "worst" data I get at least a 1.3-1 compression ratio.

Re: No Compression, DDS-3

Rodney Hills — Mon, 30 Sep 2002 13:43:03 GMT

I had a similar event occur just recently. It turned out the tape drive itself was having errors. This causes the system to re-position the tape and writing blocks multiple times.

This made it appear that compression wasn't going on.

Check your
-- Rod Hills

Re: No Compression, DDS-3

Dietmar Konermann — Mon, 30 Sep 2002 13:51:50 GMT

Data compression seems to be the key here... that's why I recommend dumping the DDS internal statistics using tapeinfo. You get all needed info at once.

Example output:

...
Data Retrieved from Drive Compression Logs
Kbytes to DC = 581483
Kbytes from DC = 596966
Kbytes to Tape = 325867
Kbytes from Tape = 341350
Compression ratio (Read) = 1.75 : 1
Compression ratio (Write) = 1.78 : 1

Re: No Compression, DDS-3

Tom Dawson — Mon, 30 Sep 2002 17:42:13 GMT

Dietmar,

Thanks, I do have the tapeinfo tool and I've found its' documentation at docs.hp.com. When I tried to execute it, I was asked to install a license. Is this licenses software? I had the impression it was part of the STM suite.

Leif,

As I've been testing this scenario over the past two months, I've swapped the drives back and forth many times. The problem stays with the "bad" server.

Vincent,

The "bad" server is a production server and the "good" server is an acceptance server for the same Oracle DB/Application. The data on the acceptance server was actually copied from the production server a few months ago. Over time it has certainly changed, and log file growth is different. But basically, they have the same db files.

Clay,

Thanks, but as I mentioned above, the data ( at least the data structure ) is about the same on both servers. The Oracle dbf files are static, so we don't see growth in that area. Obviously, the data within the dbf files changes/grows. I do see significant compression on the "good" server, so I still feel I should be seeing at least some compression on the "bad" server.

I'll experiment with tapeinfo as time permits this week. And I'll post an update to let everyone know what I find.

Thanks again!
Tom

Re: No Compression, DDS-3

Tom Dawson — Mon, 07 Oct 2002 11:05:37 GMT

All,

Tapeinfo helped me find the real problem. Using tapeinfo
through the week on incremental backups, I found that I
was actually getting less that 1:1 compression at the
tape drive! Then I ran a full backup on Saturday, running
tapeinfo before the backup to clear the compression stats
and then running it again after the backup to capture the
compression stats:

2002-10-05 10:35:34 tapeinfo - after fbackup
Manufacturer = HP
Product Id = C1537A
Product Revision Level = L812
Manufacturing Date Code = 9

Data Retrieved from Drive Capacity Logs (kBytes)
Max Capacity Partition 0 = 11704284
Rem Capacity Partition 0 = 0
Max Capacity Partition 1 = 0
Rem Capacity Partition 1 = 0

Data Retrieved from Drive Compression Logs
Kbytes to DC = 17656900
Kbytes from DC = 0
Kbytes to Tape = 11882728
Kbytes from Tape = 0
Compression ratio (Read) = n/a
Compression ratio (Write) = 1.49 : 1

1.49:1 is okay, but not what I expect.

I ran similar backups on my other servers and got compression
stats of better than 2:1. So some research found my real
problem. Our DBA has been taking exports of the Oracle data-
base each night and compressing them ( no problem there... ).
And then as we run the weekly full backup, he copies all the
dbf files and compresses them ( again, I don't have a problem
with that ). But it appears he doesn't delete them! I had
compressed export and dbf files going back more than two
months. About 40% of the data on the server was in the form
of compressed files. Consequently, those files will not
obtain much compression at the tape drive. So, on Sunday I
ran a full backup skipping the directories that contain all
the compressed files. I then ran tapeinfo after that backup
and got the following compression stats:

2002-10-06 04:19:14 tapeinfo - after fbackup
Manufacturer = HP
Product Id = C1537A
Product Revision Level = L812
Manufacturing Date Code = 9

Data Retrieved from Drive Capacity Logs (kBytes)
Max Capacity Partition 0 = 11624031
Rem Capacity Partition 0 = 11622047
Max Capacity Partition 1 = 0
Rem Capacity Partition 1 = 0

Data Retrieved from Drive Compression Logs
Kbytes to DC = 12246354
Kbytes from DC = 3
Kbytes to Tape = 4432244
Kbytes from Tape = 0
Compression ratio (Read) = n/a
Compression ratio (Write) = 2.76 : 1
Manufacturer = HP

So now I just need to get with the DBA and coordinate a
schedule for removing these compressed files on a regular
basis.

The tapeinfo that was installed on my 11.11 system required
a license. HP Support provided a copy of tapeinfo that does
not require a license. I've attached the shar file. It is
freely distributable.

Thanks to all for your help,
Tom