topic Re: DDS4 DAT drive capacity ? in Operating System - HP-UX

DDS4 DAT drive capacity ?

Jerry_109 — Sat, 14 May 2005 15:29:10 GMT

Hello HP,

I was under the impression the DDS4 DAT drive could hold 40G of compressed data? I was only able to load about 15G via :
tar -cvf - . | gzip > /dev/rmt/0mn

Please advise. Thanks

HP-UX B.11.11 9000/800/A500-7X

**********************************************

# /etc/ioscan -nfC tape
Class I H/W Path Driver S/W State H/W Type Description
=====================================================================
tape 0 0/0/1/0.1.0 stape CLAIMED DEVICE HP C5683A
/dev/rmt/0m
********************************************
Jerry Sims
415 565-1630

Re: DDS4 DAT drive capacity ?

Tony Scully_2 — Sat, 14 May 2005 15:53:04 GMT

Jerry,

I thought about 40Gb too.

Can you post

lssf /dev/rmt/0mn

Tony

Re: DDS4 DAT drive capacity ?

Jerry_109 — Sat, 14 May 2005 15:56:24 GMT

root[/root]
# lssf /dev/rmt/0mn
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t no rewind best density available at address 0/0/1/0.1.0 /dev/rmt/0mn
root[/root]

Re: DDS4 DAT drive capacity ?

Victor BERRIDGE — Sat, 14 May 2005 16:02:29 GMT

Hi Jerry,
when you say 15 GB do you mean you tar.gz file?
Which you have just compressed on the fly before writing it to tape...
the HP C5683A DDS4 drive is given for "up to " 20/40 uncompressed data which seams to no more be the case (gzip) but notice up to is no guarantee it will...

All the best

Victor

Re: DDS4 DAT drive capacity ?

Tony Scully_2 — Sat, 14 May 2005 16:11:59 GMT

Victor,

You make a good point, the capacity is given as upto 40Gb, which means that you should be able to get about 40Gb of reasonably (2:1 is the usual ratio) compressable data onto the tape.

In this case Jerry has compresed the data with the gzip prior to writing to tape, so maybe 20Gbis more reasonable.

It is possible to use mksf to create a new device file with compression on (see man mksf, stape section -c option) but I'm not sure if that will help. I have seen this make a big difference on Berkely device files for LTO2 drives.

What other device files have you got (lssf /dev/rmt/*)?

Tony

Re: DDS4 DAT drive capacity ?

Victor BERRIDGE — Sat, 14 May 2005 16:20:35 GMT

Hi Tony,
Yes lets say Jerry put a DDS4 tape in the drive, and uses the previous command, only in that mount point there is 70% of binaries, what estimation you believe the tar.gz file be in size? (not talking of transfer to tape yet...)

All the best
Victor going to bed...

Re: DDS4 DAT drive capacity ?

Tony Scully_2 — Sat, 14 May 2005 16:27:24 GMT

Victor,

Exactly -- depends on how much those files can be 'squeezed' and if they are binaries, then 'not much'.

Have a good night.
Tony.

Re: DDS4 DAT drive capacity ?

Michael Schulte zur Sur — Sat, 14 May 2005 17:03:38 GMT

Hi,

try
tar -cvf - . | gzip | wc -c
to see how much you are trying to put on the tape.

greetings,

Michael

Re: DDS4 DAT drive capacity ?

Bill Hassell — Sun, 15 May 2005 20:58:35 GMT

The DDS4 capacity is 20Gb - period. There is no way to increase the density. So your 15Gb file takes most of the tape. Now marketing (across the board for all type of tape drives, any manufacturer) decided many years ago that the hardware compression feature should be marketed but early on, testing was done on commercial operating systems with lots of highly compressible data. Thus, your 20Gb DDS4 drive would have been rated at 40Gb to 160Gb. But these numbers are meaningless as they depend on the hardware compression algorithm and the actual data used.

By creating a file with 100 million spaces and another file with 100 million random characters, the tape drive will seem to store 100Gb or about 20Gb, depending on which file you store on the tape. However, the bit patterns stored on the tape are still just 20Gb. If the file is compressible, then the data is encoded for more efficiency. In your case, you have defeated the compression hardware by supplying already compressed data to the tape drive. In your tar|gzip combination, you did not send 40Gb to the tape. Instead, you sent perhaps 40Gb to gzip and the output of gzip was significantly less, but the tape can't compress already compressed data.

So for a truly efficient operation, you can drop gzip and let the hardware do the compression. To see exactly how compressed the data is, run the tapeinfo utility which is available from:

ftp://contrib:9unsupp8@hprc.external.hp.com/sysadmin/programs/tapeinfo/

NOTE: there are two tapeinfo programs within HP. The standard program is part of the online diagnostics, but this one is a contributed program from the tape drive labs. Be sure you are executing the correct version.

Re: DDS4 DAT drive capacity ?

Leif Halvarsson_2 — Mon, 16 May 2005 02:09:28 GMT

Hi,

Was there a DDS 4 media in the drive ?

A DDS 4 media has 20 GB of native (uncompressed) capacity. If everything is OK, 20 GB should fit on a media as a minimum.

40GB compressed data is an average, compression ratio is very data dependant.

Re: DDS4 DAT drive capacity ?

Naveej.K.A — Mon, 16 May 2005 02:16:24 GMT

Jerry,

Put the tape you were trying back in and issue the

#mt -t /dev/rmt/0mn status

to confirm that you were using a DDS4 catridge.

All the best,
Naveej

Re: DDS4 DAT drive capacity ?

Jerry_109 — Tue, 17 May 2005 09:21:55 GMT

Hello
# lssf /dev/rmt/*
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t best density available at address 0/0/1/0.1.0 /dev/rmt/0m
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley best density available at address 0/0/1/0.1.0 /dev/rmt/0mb
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t no rewind best density available at address 0/0/1/0.1.0 /dev/rmt/0mn
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley no rewind best density available at address 0/0/1/0.1.0 /dev/rmt/0mnb
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t best density available at address 0/0/1/0.1.0 /dev/rmt/c0t1d0BEST
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley best density available at address 0/0/1/0.1.0 /dev/rmt/c0t1d0BESTb
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t no rewind best density available at address 0/0/1/0.1.0 /dev/rmt/c0t1d0BESTn
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley no rewind best density available at address 0/0/1/0.1.0 /dev/rmt/c0t1d0BESTnb
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t DDS1 DAT tape cartridge at address 0/0/1/0.1.0 /dev/rmt/c0t1d0DDS
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley DDS1 DAT tape cartridge at address 0/0/1/0.1.0 /dev/rmt/c0t1d0DDSb
stape card instance 0 SCSI target 1 SCSI LUN 0 at&t no rewind DDS1 DAT tape cartridge at address 0/0/1/0.1.0 /dev/rmt/c0t1d0DDSn
stape card instance 0 SCSI target 1 SCSI LUN 0 berkeley no rewind DDS1 DAT tape cartridge at address 0/0/1/0.1.0 /dev/rmt/c0t1d0DDSnb
stape property table configuration device /dev/rmt/stape_config

Re: DDS4 DAT drive capacity ?

Victor BERRIDGE — Tue, 17 May 2005 09:33:47 GMT

Here at the bottom of you log we read DDS1...

What is the output of mt -t /dev/rmt/0mn status?

All the best
Victor

Re: DDS4 DAT drive capacity ?

Jerry_109 — Tue, 17 May 2005 10:11:10 GMT

# mt -t /dev/rmt/0mn status
Drive: HP C5683A
Format:
Status: [41112600] BOT online compression immediate-report-mode
File: 0
Block: 0
[/root]

Re: DDS4 DAT drive capacity ?

Michael Schulte zur Sur — Tue, 17 May 2005 10:23:05 GMT

Jerry,
it should say something on the tape.
Have you tried my version to see how much is left to save after the gzip?

greetings,

Michael

Re: DDS4 DAT drive capacity ?

Bill Hassell — Tue, 17 May 2005 10:31:28 GMT

mt reports:

BOT online compression immediate-report-mode

The word "compression" means that the tape drive will turn on it's hardware compression when recording. Did you get a copy of tapeinfo?

Re: DDS4 DAT drive capacity ?

Jerry_109 — Tue, 17 May 2005 11:04:50 GMT

Hello Michael,

Thanks for the "tar" verification command. My formula to determine how much data went to tape was not correct. This is the formula I was using:
example mount point /opt/ecc:

# bdf /opt/ecc
Filesystem kbytes used avail %used Mounted on
/dev/vg00/optecc 1048576 785203 258532 75% /opt/ecc
root[/opt/ecc]
# bc
.58*785203
455417.74 = 455M

Using your method
tar -cvf - . | gzip | wc -c
280056296 = 280M

My numbers were way off. I figured using tar/gzip I would get approx. 58% savings, and tried to use this formula on the filesystems I was sending to tape, but I am so wrong, because of different types of files being converted via tar/gzip.

The tape located in the DDS4 drive is labeled as follows:

hp dds-4 C5718A
DDS4 Digital Data Storage
compressed capacity: 40GB

I guess I'll just have to settle for the approx. 20G rule, and thank you all for your help. At least I have some good commands to put in my notebook.

Re: DDS4 DAT drive capacity ?

Bill Hassell — Tue, 17 May 2005 11:56:58 GMT

AS mentioned, there is no formula to compute the compression ratio. The actual compression is done byte-by-byte using the algorithm of the selected program (gzip uses one method, the tape drive uses another, fax machines use another, MPEG-3 encoders use another, and so on). In other words, there is no way to predict the compressibility of any collection of files until you actually run it through the program (or hardware).

Re: DDS4 DAT drive capacity ?

Jerry_109 — Tue, 17 May 2005 12:36:22 GMT

Thanks Bill