DPM 2010 Compression

[Ron Opp]

I am using a single LTO5 drive with DPM 2010.  I have installed the latest firmware and HP driver.  I am only able to write 1.4TB of data to a tape.  Compression is on.  Should it be off?  Or is there something else going on?

Thanks.......

[Eder]

We are also havinf the same problem. Not able to find a solution

[thomasr]

There are several possibilities.

1) Soft errors, caused by either the tape drive head being dirty or the tape media itself being of less than stellar quality, will cause the data to be re-written (but the original write area is not reclaimed), so you'll be using at least twice as much space for each bad block encountered.   Some brands of tape have less 'extra space' to make up for this than others.    I don't know if DPM provides the ability to look in to see soft errors on a tape or not; it would be worthwhile ot check this out, though.

2) Pre-compressed or pre-encrypted data can not be compressed further.   If you're performing encryption or compression before the data gets to the tape drive, expect to see no compression there.   LTO-4 and LTO-5 can both compress and then encrypt data with no loss of performance, so use the tape drive's hardware before you try using the backup application for these functions.  Then there's the issue of files such as mpeg or jpeg or zip files -- these are compressed just sitting on your hard drive; there's nothing you can do about them.

3) Small files, deeply nested directories, long file names: Each file that's saved as part of a backup has metadata associated with it.  This metadata includes the full file name, the full directory path, the file attributes, permissions, ...    If your files are tiny (and the last information I saw showed that the median file size on a typical server was somewhere in the 1K - 2K range, meaning that many, many files would have been 512 bytes or less), you're storing all that metadata, which may be larger than the file itself.  This overhead quite obviously comes out of the space on the tape, and will decrease how much data fits on the tape (it's normally hidden when your data is compressible, but if you're starting with minimally compressible data, and have a larger-than normal percentage of tiny files, you'll be stuck.

So my recomendations:

- Check for soft write errors either in DPM, or by running tests in HP's free Library and Tape Tools (download from http://www.hp.com/support/tapetools ).

- Don't compress or encrypt data except in the tape drive HW, if at all possible.

- Understand the distribution of files on your disk, to see if it's a metadata issue.

- Use Library and Tape Tools to check for tape or drive issues (diagnostic tests are available)

[RONO]

I am also using DPM 2010 with a single LTO-5 drive and can only put 1.4TB of data on a single tape.

I spent two hours on the phone today with HP tech support testing the function of the tape drive.  The drive is performing as it should but still only getting 1.4TB on a tape.  I'm not convinced that DPM is turning compression on.

Has anyone found a solution?

[SkyTik]

Just a note: 3TB capacity is with compression 2:1 enabled. But this ratio AFAIK is reached spontaneously. Depending on data written on tape. So if you are able write only 1,4TB it could be because of data type you writing on that tape

[ssobarry]

I use a HP StrogageWorks 3000 SAS, every thing worked fine, data compression was ok,.... the tape drive broke a few week ago, I  got a new 1 , same model.

But now the  Data compression isn't working any more, before I could write more then 2TB on a drive, now not more then the native capacity (1,5TB).

-Data compression is enabled in the  DPM2010 software

-Encryption is Disabled

-Firmware up to date

-Driver up to date

-Strange thing, data compression test successful (HPT&LL)

-after dpm2010 wrote to tape, HPT&LL states, data compression  "not enabled", Data compression rate  1.0:1

It looks like the tape drive doesn't honor the ioctl  that is send by DPM2010 to the tape drive to tell it to enable hardware data compression for the job.

Hope some one can help me to solve the problem

[JAMES THORNTON]

Having the same issue with a StorageWorks G2 Autoloader, LTO-5 cartridges and SCDPM 2012. This is the third forum I’ve ran across talking about this issue. Posting here in hopes of someone finds a solution. 

[Dennis Handly]

>Having the same issue...

As mentioned above, how compressible is your data?

[JAMES THORNTON]

Same data was compressing fine on our old SDLT 600. This is a new server, new tape system and a new version of SCDPM, 2012. Not sure which piece is the problem, still testing. Going to try a copy of Yosemite we have laying around to see if it's software or hardware related, will report back.