Explanation of records and blocks

[Bruce Baillie]

I am doing backups using find, cpio | dd to a DDS-3 tape which holds 24GB compressed. When the backup is finished, dd reports 18777+1 records out and the backup file reports 38456242 blocks. How do I convert records and/or blocks to bytes?

[A. Clay Stephenson]

Without knowing more you can't. What did you specify as the output block size and input block size for dd? Judging from your statistic i would guess that you specified an obs=10240k. The 18777+1 records indicates that you output 18777 full blocks (of obs size) and 1 partial block.

[Bruce Baillie]

I use obs=1024k

[A. Clay Stephenson · Accepted solution]

Okay, now we know enough to answer:

The backup file is 38456242 blocks X 512 bytes/block = 19689595904 bytes

dd output 18777 blocks X 1024 * 1024 bytes/block = 19689111552 byes + a portion of one 1024KiB block because the input file size was not an exact multiple of 1024KiB.

[Bruce Baillie]

Thanks, just what I needed.

[Bill Hassell]

And to clarify: The DDS-3 only holds 12Gb, period. If you backup data that has 90% zeros in every file, then the DDS-3 can store more than 100 Gb, but random data is simply not compressible so there will be no compression. The 2:1 compression value is an old marketing number from a decade ago and assumes that the data patterns are compressible to about 2:1. dd is just taking raw data from the disk so there may be both random and highly compressible data. In general, vg00 from a standard HP-UX system compresses to about 1.3:1.

[Bruce Baillie]

Bill, thanks for the information. The backups I am doing are file systems that contain CAD data only. If the command, compress, is any indication of how much these files can be compressed when written to a DDS-3 tape, I found it to be over 2:1. That is the reason I am getting almost 20GB on a single tape.

[Tim Nelson]

Bruce just out of curiosity why are you using dd ? raw disk ?

fbackup, cpio, tar are all much better for tape control, standard blocking, header info and volume sets?

A restore from a dd will require that the person knows all the switches and blocking factors used.

[Bruce Baillie]

Tim, The scripts were written long before I got here. I have been here 10 years. They run from cron hourly and daily. The monthly is done from a menu as are recoveries, if we need to recover data. If I make changes, and I am not here if someone needs to recover data, then confusion starts. Everything is well documented so even a cave man can do it. I just decided to leave things as-is, it works.

[Bruce Baillie]

Tim, oh, by the way,the script does a find, pipe to cpio then pipe to dd.