topic Re: Ultrium, compression, which driver? in StoreEver Tape Storage

Ultrium, compression, which driver?

Laszlo Csizmadia — Tue, 19 Aug 2003 08:10:13 GMT

I'm a little confused now that read about Ultrium tape drives "smart data compression" capability. The firmware's compression engine can decide compress the data flow or switch to pass thru mode. But does it matter which device file is used to write to the tape? I have these:
/dev/rmt/9m /dev/rmt/c17t3d6BEST
/dev/rmt/9mb /dev/rmt/c17t3d6BESTb
/dev/rmt/9mn /dev/rmt/c17t3d6BESTn
/dev/rmt/9mnb /dev/rmt/c17t3d6BESTnb
Are /dev/rmt/c2t1d0BEST and /dev/rmt/9m the same?
I thought that in case of e.g. DLTs the data compression is a question of device files used to write to the tape. Is it right or not?
With OmniBack /dev/rmt/9m is used. Is it ok?

Re: Ultrium, compression, which driver?

Leif Halvarsson_2 — Tue, 19 Aug 2003 08:34:50 GMT

Hi,

Hi,
You are correct, hardware compression or not depends on the device file used. If created as default device files, /dev/rmt/c2t1d0BEST and /dev/rmt/9m is the equivalent (check the minor number).

The /dev/rmt/9m is OK for OmniBack.

Re: Ultrium, compression, which driver?

Massimo Bianchi — Tue, 19 Aug 2003 08:36:15 GMT

Hi,
to see which device are the same, use

cd /dev/rmt
lssf *

You will see a detailed description of each special file.
/dev/rmt/c2t1d0BEST and /dev/rmt/9m are the same if their description and HW patch are the same.

In my case:

stape card instance 3 SCSI target 2 SCSI LUN 0 at&t best density available at address 5/0/6/0/0.8.0.255.1.2.0 0m
stape card instance 3 SCSI target 2 SCSI LUN 0 berkeley best density available at address 5/0/6/0/0.8.0.255.1.2.0 0mb
stape card instance 3 SCSI target 2 SCSI LUN 0 at&t no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.0 0mn
stape card instance 3 SCSI target 2 SCSI LUN 0 berkeley no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.0 0mnb
stape card instance 3 SCSI target 2 SCSI LUN 1 at&t best density available at address 5/0/6/0/0.8.0.255.1.2.1 1m
stape card instance 3 SCSI target 2 SCSI LUN 1 berkeley best density available at address 5/0/6/0/0.8.0.255.1.2.1 1mb
stape card instance 3 SCSI target 2 SCSI LUN 1 at&t no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.1 1mn
stape card instance 3 SCSI target 2 SCSI LUN 1 berkeley no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.1 1mnb
stape card instance 3 SCSI target 2 SCSI LUN 0 at&t best density available at address 5/0/6/0/0.8.0.255.1.2.0 c3t2d0BEST
stape card instance 3 SCSI target 2 SCSI LUN 0 berkeley best density available at address 5/0/6/0/0.8.0.255.1.2.0 c3t2d0BESTb
stape card instance 3 SCSI target 2 SCSI LUN 0 at&t no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.0 c3t2d0BESTn
stape card instance 3 SCSI target 2 SCSI LUN 0 berkeley no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.0 c3t2d0BESTnb
stape card instance 3 SCSI target 2 SCSI LUN 1 at&t best density available at address 5/0/6/0/0.8.0.255.1.2.1 c3t2d1BEST
stape card instance 3 SCSI target 2 SCSI LUN 1 berkeley best density available at address 5/0/6/0/0.8.0.255.1.2.1 c3t2d1BESTb
stape card instance 3 SCSI target 2 SCSI LUN 1 at&t no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.1 c3t2d1BESTn
stape card instance 3 SCSI target 2 SCSI LUN 1 berkeley no rewind best density available at address 5/0/6/0/0.8.0.255.1.2.1 c3t2d1BESTnb

0m is c3t2d0BEST
1m is c3t2d1BEST

Massimo

Re: Ultrium, compression, which driver?

Laszlo Csizmadia — Tue, 19 Aug 2003 11:27:14 GMT

Right. /dev/rmt/c2t1d0BEST and /dev/rmt/9m are the same. The BEST devices are the same.
But I still don't understand something. I have Ultrium-2 drives with these devices(forget BEST):
# lssf /dev/rmt/9*
stape card instance 13 SCSI target 3 SCSI LUN 6 at&t best density available at address 1/0/2/0/0.97.35.255.1.3.6 /dev/rmt/9m
stape card instance 13 SCSI target 3 SCSI LUN 6 berkeley best density available at address 1/0/2/0/0.97.35.255.1.3.6 /dev/rmt/9mb
stape card instance 13 SCSI target 3 SCSI LUN 6 at&t no rewind best density available at address 1/0/2/0/0.97.35.255.1.3.6 /dev/rmt/9mn
stape card instance 13 SCSI target 3 SCSI LUN 6 berkeley no rewind best density available at address 1/0/2/0/0.97.35.255.1.3.6 /dev/rmt/9mnb

So there are at&t and berkely best density with rew or not rew. What is the differences between these two?
Does it mean that the drive always makes HW compression? How to avoid this? Does it make any sense to avoid this?
Thanks.

Re: Ultrium, compression, which driver?

Leif Halvarsson_2 — Tue, 19 Aug 2003 11:59:30 GMT

Hi,
This drives is all compressed. To avoid hardware compression you have to create an uncompressed device file manually (with mksf).
It depends on the application if berkely or at&t style file should be used. For example, OmniBack uses at&t files on HP-UX but berkely files on Solaris.

Re: Ultrium, compression, which driver?

Leif Halvarsson_2 — Tue, 19 Aug 2003 13:58:16 GMT

Hi,
You can find instructions how to create devicefiles in this document:
http://h200005.www2.hp.com/bc/docs/support/SupportManual/lpg28833/lpg28833.pdf
To create a uncompressed file:
mksf -H -a -b U_18

Re: Ultrium, compression, which driver?

Curtis Ballard — Wed, 20 Aug 2003 13:55:06 GMT

The Ultrium drive "Smart Data Compression" capability only is used when a "best" density device file is used. All the device files you list will use a "best" density which means "hardware compression". Since hardware compression is on the Ultrium drive will compress the data when it can.

Where "Smart Data Compression" comes into play is when the data can not be compressed. Most drives will always attempt to compress data when hardware compression is on so with data that can't be compressed further you actually get larger files. It is up to the user to pick the compromise device file.

With most drives if you have lots of compressed files you are backing up you want to use a non-compressing device file. You will lose some capacity on the files that could be compressed though. If you only have a few compressed files and mostly files that can be compressed you choose a device file that enables hardware compression. You will lose some capacity when you back up already compressed files though.

The Ultrium drive "Smart Data Compression" takes care of all the guess work for you. Alway use a compressing device file and the drive will automatically pick the method that will take the least space on the tape.

No more compromises.

No more capacity loss.

Re: Ultrium, compression, which driver?

Laszlo Csizmadia — Thu, 21 Aug 2003 07:18:17 GMT

Thanks to all the information.
Now I understand almost everything. I have just one more quiestion:
Let's say I have Ultrium 420. The docs claims 30MB/s native write speed. How to measure it?
#time dd if=/dev/zero of=/dev/rmt/xxx bs=1024k count=10000
It's a nice way but what device to use in /dev/rmt/xxx ? Comressed one ore not?
30MB/s native(!) write speed suggests me a value without any HW or smart data compression.
Like e.g. in DLT8000 6MB/s native, ~12MB/s compressed.

Re: Ultrium, compression, which driver?

Massimo Bianchi — Thu, 21 Aug 2003 07:27:32 GMT

I would not measure that way the compression, but rather use an oracle used datafile.

/dev/zero gives you a lrge bunch of zeros. Very very compressable :)

for the test use /dev/rmt/Xm
but as a source .... vmunix, datafile from a system tablespace.

Take care in the measure, there may be issues with:
scsi, fc, disk capacity, cpu speed, bus architecture.

We were not able to achieve the full speed on a K due to bus architecture with many cpus...

Massimo

Re: Ultrium, compression, which driver?

Leif Halvarsson_2 — Thu, 21 Aug 2003 07:53:11 GMT

Hi,
It can be hard to reach the full performance of a Ultrium 460 drive due to other limitations in the system. For filesystem backup tests, ftio or fbackup is faster then cpio or tar. For raw performance, DataProtector disk image backup is a good tool (you can download an evolution copy of DP from HP).
To test native performance you must use an uncompressed device file else the drive hardware compression is on.

Re: Ultrium, compression, which driver?

Curtis Ballard — Thu, 21 Aug 2003 14:27:17 GMT

If you use dd for testing and use /dev/zero for the input file then you must use a non-compressing device file. If you use a compressing device file with /dev/zero then you are testing your system bus speed to the drive, not the driver performance, as almost nothing will get written to the drive.

Testing with a compressing device file and /dev/zero is a good technique to test your I/O bandwidth and see what the maximum transfer rate your system is capable of. Note that the bandwidth will be the slower of the system I/O performance and the device I/O performance. If the drive can sink zero's faster than the host can send them the measure is your host I/O bandwidth. If the host can send them faster than the device can sink them then the measure is your device bandwidth. With a LTO gen 2 there is a really good chance you will hit your system limit. That drive can chew up a lot of zeros!

Check the following URL for some performance testing tools and tips. http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=lpg50460#P5_340