LTO hardware compression (or not)

[Antoon Frehe]

Dear all,

I have an Ultium drive hooked up to my C3000 HPUX box. I used /sbin/insf to create the raw devices and got 4. Doing some tests i can't get the speed anywhere near the specified 15Mb /sec. It stays at around 1Mb/sec. I try this by doning a "dd if= of=" with different block sizes. It's connected to a HVD-scsi which is i guess fast enough.

My thoughts are that none of the devices that is created by insf is a non-compressing device. Anything i dump will be compressed by the Ultrium drive itself which tapes (too much) time. At least that's my theory. I can't find any referenced on the net that confirm this theory of mine.

Does anyone have any experience with this compress-noncompress raw device under HPUX and if so, is there a bit in the minor number of the device that specifies whether the compress option of the drive whould be used or not? Cab i use mknod to create a non compressing tape device?

Thx in advance.
Antoon

[Duncan Edmonstone]

Can you supply some more info about your SCSI interface... just cos its HVD doesn't mean it operates at > 15MB/s... see the attached link:

http://www.scsita.org/terms/scsiterms.html

Also have you made sure the the LTO is the *only* thing on the SCSI bus (not being shared with your fast disk for example)

HTH

Duncan

[Antoon Frehe]

According to ioscan -f it's a "Ultra Wide SCSI". I must admit, i'm not on top of scsi specs and definitions these days any more. And yes, i do have other disks connected to this bus. However, they are not accessed regularly and a small test shows that i can easily move date from 1 disk on de the bus (which my Ultrium is also connected to) to another at 8Mb/sec. That's done with "dd if=bigfile of=anotherdisk/bigfile".

I'm not expecting max specified performance ofcourse but at the current rate i won't even be able to write one tape a day.

thx Antoon

[Martin Johnson]

Where does the data that you are coping to tape reside? Hopefully, on a disk that is *NOT* on the same scsi bus.

When copying the data, does the drive stream, or does it start and stop. If it starts and stops, the drive is not being used to its fullest capacity. The problem lies elsewhere, disk, buss, caching.

[Aaron Tatone]

Antoon,

We had similar numbers using a dd to the tape drive. The problem with dd is that it is a single stream. In order to get a decent throughput to the drive, use backup software that allows for streaming of files to the tape (i.e. Omniback, TSM, veritas).

If your disk back-end system is high-end, capable of 15-30 MB/s transfer rates, then do a dd with block size of around 8192k (dd -if -of -bs 8192k). This should give you better numbers.

Aaron

[Leif Halvarsson_2]

Hi
You should not need any "tricks" to get performance with the ULTRIUM drive. Create the devicefiles with SAM (ore use the files witch is created when you boot the computer whith the drive connected). The one called (for example) /dev/rmt/0m or /dev/rmt/xxxxxBEST will work fine. There is a utility in HP-UX called ftio witch is better than dd for performance tests, try this (if you dont have a hight performance backup program). You should always use hardware compression to get best performance.
I use a ULTRIUM 230 drive connected to a A400 computer and there is no problem to stream it with 30MB/s.

[Antoon Frehe]

Thanks guys for all the tips and suggestions. I think i've accepted that i'm not going to reach the ideal situation (for the moment). I do have a good machine and a fast bus but there are other things going on on this machine which may interfere with optimal performance. In my latest tests i still have not been able to reach a streaming situation and it seems there is a big performance penalty for that. Next tests i will perform during maintenance when i can swith everything off that's not related to the test.

Still, one day, i'd like to know whether the 4 devices created for this Ultrium unit activate the hardware compression of the device or not. After all, my backup software allows me to specify to use gzip on filesystems before they are sent to tape. This is ofcourse rather silly if the unit does hardware compression and i cannot swith it off.

Antoon

[Jan Klier]

To answer your first question: yes several of these device special files that are created with insf will enable compression. The one ending in *BEST will use it if support by the hardware (which is the case for LTO).

See 'man 7 mt' for more detail on any HP-UX system.

Now, using data compression itself will not slow down the drive and your logic doesn't entirely hold. However, if hardware compression is enabled and the data being backup are compressible, you need to move the data even faster to the drive in order to keep it streaming.

The native maximum sustained speed of a 230 LTO drive is 15MB/s, however assuming 2:1 compression (std. avg.) it goes up to 30MB/s.

LTO drives have a variable speed function and they can slow themselves down in order to adjust to a slower data stream and still keep streaming. But this algorithm stops working when the data rate drops below 6MB/s (uncompressed).

Other factors to consider is the source of your data, the block size used, etc. and your bus configuration.

Actually, a LTO drive wants to transfer data over the bus at a burst rate of 80MB/s. However, HVD SCSI cards top out at 40MB/s - so using a HVD card puts an additional burden on your configuration.

If you want to test your systems ability to provide data fast enough, there is a performance assessment tool from HP. We just finalized an HP-UX version but I'm not sure if it's on the website already. If it is, you should be able to find it at http://www.hp.com/support/pat