DAT72 "real" native capacity (uncompressed)

[shorton]

I am using a HP DAT72 internal USB tape drive.  I am using HP brand DAT72 media.  I use Novastore Professional backup software.  Software compression is turned off so as not to get any software compression encapsulation overhead.

I have some backup sets of 100% jpeg files.  I broke my sets into what I thought would be conservative 32GB blocks.  My understanding is I can get 36GB of uncompressed data on one tape.  However my backups are getting right to the end with jsut a little bit left to go and the tape is filling.  So it's not holding 36GB as compared to what is reported on disk or by the backup software.

I thought maybe it was a difference in how "GB" is being calculated for the tape, vs. Microsoft or the tape software.

What is the native size.  in BYTES? 

Or, what should I expect to get when writing compressed data?

[shorton]

Along these lines, when trying to maximize a backup set, does anyone know how much overhead there is for the data to go to tape (headers, indexes, etc.?)

Could any of this be because the tape folks count DAT 72's 36GB as

36,000,000,000 bytes

But Microsoft and the backup software considers 36GB as

36 * 1024^3 = 38,654,705,664 bytes ??

If that was the case, then a 36"GB" tape would hold 33.52.

But I'm still not quite getting that much compressed data onto the tape.  My backup stopped and prompted for the next tape at 29.52 GB (31.7 GB @ 1,000^3 GB format).  Still way short of 36GB

[thomasr]

I think you're right that the difference between binary GB and decimal GB.

In addition to that, remember that the backup job has to store the complete file name, path, attributes (R/W. Archive, System, Hidden, as well as last access time, creation time, etc.), permissions, ...   For tiny files with long file names in deeply nested directories, this can add a substantial amount of data to the backup set.

Additionally, write errors generate re-writes, skipping the space previously used for that block, so if you have a substantial number of errors (dirty head, defective media, media or head nearing end-of-life), you'll chew up a lot of extra space as well.   You can usually check this in the tape properties in the backup application, or in Library and Tape Tools through a support ticket.

Library and Tape Tools will also run diagnostics on your media and drive to help you understand if questions like this are caused by errors.

[thomasr]

One other thing -- noticing you are backing up jpeg files, which are already compressed data. You might get slightly more on a tape if you turn hardware compression off also. I don't remember if DAT has a split-path for compression like LTO does, but if not (and it's probably not), then compressing previously-compressed data may cause the space used to expand slightly.

[shorton]

Thomas:

Thanks.  My software doesn't have a way to turn off the hardware compression.  My understanding from HP's docs (read a whiel back) was that the hardware compression on this drive was adaptive in that it woudl self-administer its state when it encountered what it thought was compressed data.  Anyway, I don't have a way to control it that I am aware of with my software.

My software does have a feature called "chunk or chunking" where it apparently divides the data into certain size chunks. It's not for tape and shoudl have been disabled, but I found it had been left on accidentally for this particular backup  I turned it off and restarted the backup.  BUt it still went to 29.5 GB and stopped (filled).

The media in new, freshly opened, the drive should be OK.   But I haven't run the diagnostics on it.  Errors woudl account for issues, I had not considered that.  These are 100% photp jpeg backups.  Most in the 2M per file size.  Directories are probably nested about 3 levels deep from the backup root.  About 17000 files.  Even at 200 bytes per path (surely that's enough), that's only an extra 0.003GB.

Do I need a support case open in order to run the diagnostics?  I didn't understand that part.

Thanks,

Scott

[shorton]

Thomas:

I updated my HP Tape tools (LTT) and ran the drive test.  It tested fine.  Said I wasnt' using the best media to see the compatible media tab.  Which said to use teh media I'm usng (HP C810A).  Not sure what's up with that.  Otherwise no errors reported.

Ran the backup with NTBackup on the same dataset to take the Novastor backup software out of the picture.   NTbackup reports bytes and it stopped pretty much at the same place as the other program.  The actual bytes backed up when the tape ran out was 31,694,496,269.    That divided by 1024^3 is 29.5GB, just wat the other program reported.

So it would appear it will not do 36GB of already compressed data.  I don't want to disable hardware compression becasue I use it for other (non jpeg) backups.  I guess I'll just reduce my (non compressable) backup sets to under 31.6 billion bytes.