Ultrium 230 Tape Drive - Too fast for my PC ?

[SebastianMWS]

Greetings all,
 
First time on HP forum pages, and chances are I won't be all that popular after posting this writing. First off all, I sincerely apologize in advance, for the sheer length of this post. Second, the topic is one of the least popular, be it here or any other forums specializing in computers. Thank you all for understanding !

Background:

For quite some time now, I've been contemplating the purchase of a mid-range standalone tape drive for personal use, mainly for long-term archiving of my photo slide scans, and additionally for safe-keeping most valuable graphic design work for future use. First of all, I should mention that I already use a backup setup – a small external enclosure with 10k SAS drives, teamed in several RAID 1 mirrored pairs. Additionally I make copies of most valuable work and photo scans on DVD drives. Tape drive and tapes are meant to be an extra security, with long-term archival properties in mind.

Goals:

Since this is a »fresh-start«, I am not chained to any specific type of tape storage technology, and thus I can choose more freely the most suitable one, matching my preferences and budget. Here's the brief outline of my prerequisites, in no particular order of preference:

- Tape capacity: 20GB min - 100GB max (native/uncompressed)
- LVD/SE SCSI interface
- Direct attachment, no server or network of any kind involved
- Industry proven long-term reliability of mechanics, electronics and read/write heads
- Easy basic maintenance (cleaning innards, manual unloading stuck tapes, lost tape leader simple repairs, etc.)
- Low to moderate power consumption
- Tape drive native and HW compression transfer rates, best suited for real-life capabilities of my computer (subsystem throughput, OS, choisen backup application, etc.) , to allow for 100% consistent streaming mode.

Arguments:

This is a long-term investment, so my main concern is tape drive longevity. Avoiding bottlenecking and frequent re-positioning during writes is not an option, it’s a must !

Meet the HP Ultrium 230 External Tape Drive...

I recently came across a brilliant opportunity, to purchase a brand new HP SureStore Ultrium 230e tape drive, at the price that is almost too good to be true. But before I jump onto this wagon, I really need to identify most critical areas of potential issues. I'm hoping to receive some sound advices and recommendations.

Points of focus, after doing an extensive research, are the following:

1.    Throughput & streaming:

A major source of my doubts and concerns, regarding the type of tape drive to purchase, is for being worried that my system will not be able keep up with the speed of “data conveyor belt” – feeding the tape drive fast enough to allow for consistent streaming mode.

If anyone cares to take a quick glance at my PC specs, and help me with a rough assessment of my system throughput projection (worst case scenario !), I may be a few steps closer to my final decision. I will merely focus on those system details, and intended tape drive usage, that I believe matter the most for the topic at hand (in my humble opinion);

-    Workstation/Server board Tyan S2925 (ATX Form Factor)
-    AMD Phenom X4 2.5MHz
-    8GB ECC RAM (I could designate 1GB just for extra tape drive cache)
-    3x PCI 32-bit/33MHz slots*
-    One PCI slot occupied by Adaptec AHA-2940U2W Ultra-2 Wide SCSI card (PCI 32-bit, 80MB/s max)
-    Ultra2 Wide segment** of Adaptec AHA-2940U2W reserved exclusively for tape drive
-    Pair of 10K SAS hard drives in RAID_1, reserved exclusively for temporary holding storage of files to be archived. Attached to SAS HBA x8 PCIe  
-    Tape drive mainly designated for archiving large files (TIFF photo scans), but not exclusively so.
-    Linux Fedora 15 distro


*Unfortunately I cannot make use of much faster, bandwidth generous PCI Express technology for tape drive SCSI controller, as those slots are already occupied by hard drive SAS controller and video card;

**Link to AHA-2940U2W Product Brief: http://download.adaptec.com/pdfs/aha2940u2w.pdf
In case it is rather obvious (to seasoned IT experts), based on these specs and »on paper« system performance, that my PC is a no match for Ultrium 230 LTO-1 streaming requirements, not even with HP Data Rate Matching feature at work (min 6MB/s at native/uncompressed rate), I would immensely appreciate if anyone could advise me on »down-scaling« scheme of my ambitions, in this particular order:    

-    HP SureStore Ultrium 215 HH
-    DLT 7000/8000 (HP, Quantum or Sun flavour)
-    Quantum/Benchmark DLT1 FH
-    DLT 40/4000 (HP, Quantum or Sun flavour)


2.    Performance Assessment & Diagnostic Tools:

I looked long and hard, initially browsing through HP Ultrium 230 documentation and list of available tools/utilities. Sadly, not one of them is supported in Linux Fedora, or any other desktop Linux distros for that matter.

-    HP L&TT Tools
-    HPTapePerf
-    HPReadData
-    HPCreateData

What are my other options ? Quantum’s xTalk is also available only for enterprise Linux distros. Can somebody recommend any well regarded open-source tools, with features and performance roughly comparable to HP's own tools ?

3. Ultrium 230 Firmware updates under Linux : How on earth … ?!?

Once again here, I bump into this mind boggling wall of Linux support .. or a lack thereof ! HP L&TT Tools, no doubt magnificent and dandy, are not available for Fedora (or Ubuntu, or Debian, or OpenSuse, or …. ).  This may sound as a desparate attempt, an imulse thought spawn by frustration, but … will the Red Hat version of L&TT install and work in Fedora ? Do I feel lucky ?

It’s when I had begun asking myself these questions, that the first serious doubts about HP’s Ultrium surfaced up. To go for it anyway, and pray I won’t ever need a firmware update ? Sounds pretty unnerving, and not exactly a future-proof decision.

4. Switching HW compression ON/OFF: The manual way

HP Ultrium 230, being my No.1 choice, uses a firmware based automatic HW compression ON/OFF switching By default compression setting is ON, and it’s turned off when the drive chooses to do so  (uncompressible data, or for data already compressed). While this may look good on paper, and is perhaps useful for most backup/archiving scenarios, I am never too happy about any “auto-this-and-that” ingenuity, unless it can be  switched on and off manually, at my free will and based on my own judgement (good or bad).

There seem to be further conlicts of interests, as HP warns users to - quote…”Make sure any software data compression is turned off”. On the other hand, some manuals/guides for Linux backup utilities caution in the opposite direction – to turn HW compression off, with a note that this issue is less critical with LTO drives (source: Amanda Wiki).

Come to think again, I’d actually prefer to turn off HW compression on the drive permanently, and utilize either the uncompressed tape writes by default, or use the Linux compression tools whenever I see it fit.

Question:

Can I switch HW compression ON and OFF on Ultrium 230, using MTCOMPRESSION ioctl ? If not, are there any advanced Linux backup utilities, that support switching off HW compression on Ultrium/LTO drives  ?

5.  Responsiveness/speed of backup utilities seems to be very common culprit for mediocre tape backup performance (inconsistent streaming). While I’d like to believe I should be just fine, using time proven Linux tape backup utilities such as tar and kDat, I’m also concerned due to comments and recommendations I read, especially HP’s own not to rely on tar and similar native Linux apps, supposedly because they are not fast enough for Ultrium 230/LTO-1.

If that’s the case, and assuming that HW components of my sistem are up to the task, I’ll need to carefully select a backup utility, with performance to match the available throughput scheme for streaming mode. Once again I’m facing the fact, that most advanced free open-source backup tools, sush as Amanda or Bacula for instance, seem to be designed for network/server/client backup schemes.

So what are my options, regarding a speedy backup tool for desktop Fedora box with direct attached tape drive ?

6.    Essential things about tape drive backup/archiving, which I have acknowledged:

-    During tape writing or tape restore tasks, no other application/process, being BUS/CPU/HD subsystem  intensive, should be running in parallel on computer, in order to provide maximum transfer rates and improve chances for consistent streaming mode

- Block size 64KB, the larger the better.

This should be it. I am aware it is lengthy, but I can also assure you, first, that I worked hard to make it as brief as possible, and second, that I've done my homework as thoroughly as I thought I should have, before posting on this forum.

Thank you all for your time and effort, and your patience in particular !

Kind regards,

Bostjan


P.S.: Have mercy...

[Rajat Bhargava]

A little late but just saw this post.

I will try to be as precise as possible with my answers:

1.    Throughput & streaming:

The specs on the PC will not be able to handle the streaming data the drive requires to perform optimally but the LTO drive is adaptive and will slow down. Infact all tape drives across technologies will be adaptive in nature, but there is an ultimate speed limit.

Recommendation: DAT drives available in USB and SCSI

2.    Performance Assessment & Diagnostic Tools:

LTT is the best one available. You can try installing it on Fedora or just go for Red Hat

3. Ultrium 230 Firmware updates under Linux : How on earth … ?!?

LTT again is the preferred way. Red Hat is the preferred OS as it is tried and tested

Following OS's listed in Quick Specs:

• HP-UX 11i (PA-RISC)
• HP-UX 11i v2 (PA-RISC & IPF)
• HP-UX 11i v3 (PA-RISC & IPF)
• Linux Red Hat 9.0 (x86)
• Linux Red Hat EL 3 (x86, x86_64 & ia64)
• Linux Red Hat EL 4 (x86, x86_64 & ia64)
• Linux Red Hat EL 5 (x86, x86_64 & ia64)
• Linux Red Hat EL 6 (x86, x86_64 & ia64)
• Linux SUSE SLES 9 (x86 & x86_64)
• Linux SUSE SLES 10 (x86, x86_64 & ia64)
• Linux SUSE SLES 11 SP2 (x86, x86_64 & ia64)

The LTO2 drive has no firmware upgrades available for the past couple of years I would say. I dont think any furture upgrades will be available from HP.

4. Switching HW compression ON/OFF: The manual way

Not possible to turn off HW compression at all. Would have been possible in very old drives but not in LTO's/Ultriums.

5.  Responsiveness/speed of backup utilities:

Since the throughput is going to be slow from the PC, speed should not be an issue. You could use "tar".

Let me know if I have answered your questions.

Regards

Rajat Bhargava

[SebastianMWS]

Hello Rajat,

Many thanks for your informative replies ! To be honest, I have practically given up on receiving any kind of response to my questions, be it here on HP Forum, or anywhere else I posted the same questions.  A reason more why I was so surprised but also delighted when I read your post. Late or not, it will be a valuable and very useful for my decisions and actual installation process.

Since you invited me, so to speak, to ask any further questions, I'd like to offer a few thoughts as a result of my extensive research. Albeit my knowledge and experiences are nowhere near to yous, I hope that I got at least some facts and figures right;

1.    Throughput & streaming:

»The specs on the PC will not be able to handle the streaming data the drive requires to perform optimally but the LTO drive is adaptive and will slow down. Infact all tape drives across technologies will be adaptive in nature, but there is an ultimate speed limit. Recommendation: DAT drives available in USB and SCSI«

I actually managed to find the following pieces of information in the »hp StorageWorks Ultrium tape drive – getting started« document ...

• »Your disk sub-system. A single hard disk system will not be able to transfer data at the optimum transfer speed. Raided disk solutions are preferable.«

• »Note: For Ultrium 460 and 230 drives, a RAID disk subsystem is also good practice, but a single disk may be sufficient, if it is a fast disk, such as a 15K RPM drive, and depending on your data’s compressibility. You can use our free performance assessment tools, which are available standalone or integrated into HP Library & Tape Tools, to check tape performanceand test whether your disk subsystem can supply data at the optimum transfer rate. You can also use these tools to calibrate your restore performance and maximal tape performance with more compressible data.«

Based on these two paragraphs I decided not to take my chances, and rather use a pair of fast 15K SAS drives in a stripped RAID 0 setup, dedicated just for feeding the tape drive during backups.

As for your recommendation to use the DAT tape drive instead of Ultrium;

I believe that DAT 40 should be perfectly fine for weekly / monthly backups, but for my long term photo archiving, I'd like to make use of something more robust and realible, a well proven industry standard such as LTO-1 or DLT8000 (perhaps a slower and less power hungry DLT1, but only if based on personal recommendations).

2.    Performance Assessment & Diagnostic Tools:

»LTT is the best one available. You can try installing it on Fedora or just go for Red Hat.«

Based on post I read on this forum (could be yours), I'll try my luck and go for Fedora install.

3. Ultrium 230 Firmware updates under Linux : How on earth … ?!?

»LTT again is the preferred way. Red Hat is the preferred OS as it is tried and tested«

I'm actually not inclined to perform Firmware upgrades just for the sake of it. I don't plan to do it unless I have to. Again, I'm hoping that L&TT installation on Fedora will be a success, and that all it's features and tools, including FW upgrade, will work as they should.

4. Switching HW compression ON/OFF: The manual way

»Not possible to turn off HW compression at all. Would have been possible in very old drives but not in LTO's/Ultriums.«

While I couldn't find many user feedbacks about this topic, those that I did were very useful. According to sources listed below, turning HW compression on Ultrium 230/215 is possible, so the only question that remains is whether any of those solutions can or cannot turn the HW compression off on a permanent basis.  

-          http://wiki.zmanda.com/index.php/Hardware_compression :

Based on this article it all seems that a permanent »OFF« setting is not possible, like it was the case with older DLT tape drives (DIP switches, ON/OFF button on front panel).  Power cycle (or system reboot) and using a tape that was previously written with HW compression ON, are just two examples when Firmwware in LTO-1 tape drive decides to reset to default »ON«, or to override the SCSI command 'mt compression 0' in Linux.

That's not a problem for me – I can manually set the HW compression OFF before each backup with 'mt compression 0' command (perhaps a script would be handy), and/or properly overwrite those LTO tapes that were written with HW compression ON (as Amanda article is suggesting).                       

-          mt-st package for Fedora, stinit command and stinit.def config file :

http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/amanda-20/disabling-lto-2-hardware-compression-68240/

http://linux.die.net/man/8/stinit

Now this looks promising, though like 'mt' command, I don't belive this method can turn off HW compression permanently. I need to study it thoroughly.

5.  Responsiveness/speed of backup utilities:

»Since the throughput is going to be slow from the PC, speed should not be an issue. You could use "tar".«

I definitely plan to master and use a 'tar' utility, and it seems that a lot of people prefer it over other alternatives. Still, using 'tar' exclusively may not work for me.  After a along search I finally came across invaluable piece of information, regarding open source Amanda backup application – it is possible to install Amanda server and client on a single machine.

A great find for me, but I still have to figure out whether it's possible to back up such host/client single system with a direct attched tape drive, without any utilization of onboard LAN. A post on amanda-user forum is suggestion that this is not possible (entirely so) ...

http://www.mail-archive.com/amanda-users@amanda.org/msg04736.html

Any additional wisdom, a suggestion or a tip for the reading material, would be most appreciated.

Sincerely,

Bostjan Kravcar