moving many raw disk to a remote LTO tape by dd command ??

[Donghee Lee]

hi..
i need to move rlvol used by oracle on local machine(HP-UX 10.00) to remote server(HP-UX 11.00) which also have single LTO drive on remote site.
BUT i got a some limitation..
there are many small rlvol have to move on local machine , so i would like to backup many rlvol to only one LTO tape if possible to save lots of time to do if possible.

thanks in advance.

[Stefan Farrelly · Accepted solution]

What on earth is an LTO Tape ?

Anyway, if you tape drive is big enough you can copy multiple raw lvols to it using dd and the no rewind option on the tape, eg;

dd if=/dev/vgXX/rlvol1 of=/dev/rmt/0mn bs=64k
dd if=/dev/vgXX/rlvol2 of=/dev/rmt/0mn bs=64k
etc.

And keep going until your tape fills up! When you restore each lvol is in sqeuential order on the tape so again use the norewind option for the tape drive to restore;

dd if=/dev/rmt/0mn of=/dev/vgXX/rlvol1 bs=64k
dd if=/dev/rmt/0mn of=/dev/vgXX/rlvol2 bs=64k
etc until done.

Unless youre using a large capacity DLT you may need to use multiple tapes, just make a note which lvols you got onto each tape and in which order!

[Pete Randall]

Stefan,

Just for background info (from http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xe9878cc5e03fd6118fff0090279cd0f9,00.html), in particular, Jan Klier's post:

LTO drive (230):
* 15MB/s native sustained transfer rate
* 30MB/s sustained transfer rate assuming 2:1 compression.

DLT 8K:
* 6MB/s native sustained
* 12MB/s sustained transfer rate assuming 2:1 compression

(data taken from specification files of external HP site).

LTO drive is definitely superior in transfer performance, so much though that
on many systems the system can't keep up with the drive to keep it streaming.

[Pete Randall]

Sorry, didn't get all of that - 100 GB native capacity, great transfer rate, etc., etc. LTO stands for linear tape open - sometimes referred to as Ultrium, I think.

For what it's worth,

Pete

[Carlos Fernandez Riera]

Search:

dd +gzip +bs +remsh ( boolean)


http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x59d699f08a51d61190040090279cd0f9,00.html

[Volker Borowski]

Watch out !

I am sure you'll need postactivities after this copy, when you switch from UX 10 to UX 11.

At least oracles catalog+catproc scripts should run, to re-compile the stored procedures inside the database.

Stefans suggestion should work in general, but you should check with some native tar-commands, that the device-file names have not been mixed up between 10.0 and 11.0. I think LTO required some patches under UX10, and it might turn out that the media ist not read-compatible between the two OS versions.
Play it safe and check the device files before your journey.

Good luck
Volker

[Donghee Lee]

first, i have to say very thank you guys for kind answers..

Assume there are hostA (HPUX 10.20 32bit) and hostB (HPUX 11.00 64bit) ..
hostB has a single LTO Drive identified by 1m.

in hostA, i tried followings with good result.
-------------
hostA> dd if=/dev/vg00/rlvol6 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k
hostA> dd if=/dev/vg00/rlvol5 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k
--------------------------------

in hostB, when i restore rlvol6 and rlvol5 to image file from tape media,2m, it works..
but, rlvol6 and rlvol5 restore from tape media to rlvols which were already created properly
on hostB was not work.


Here is results of mine.

---------------------------------
hostB> mt -t /dev/rmt/2mn rew
hostB> dd if=/dev/rmt/2mn of=/omdata5/test.1 bs=1024k
0+12827 records in
0+12827 records out

it worked.

hostB> mt -t /dev/rmt/2mn rew
hostB> dd if=/dev/rmt/2mn of=/dev/vg00/rlvol4 bs=1024k

I/O error
0+1 records in
0+1 records out
--------------------------------

Have you ever experienced this kind of problem..??

plz.. help me..

[Donghee Lee]

sorry.. 2m is correct for LTO tape device..

[harry d brown jr]

Donghee,

Looks like your second tape file is corrupt or the LTO didn't position correctly. I'd suggest you using TWO separate tapes.

live free or die
harry

[Stefan Farrelly]

Hmm, very interesting. So you can dd off the tape to an image file aok, but not to a new raw lvol. Things to check;

1. Is the raw lvol (new one) the exact same size as the old one on hostA ?
2. Is the new lvol unmounted on hostB unmounted at the time of the dd ? it needs to be.
3. When you did your dd to an image file did you remember to rewind the tape before you tried now dd'ing it to the new raw lvols ? ie;
mt -t /dev/rmt/2m rew
dd if=/dev/rmt/2mn of=/dev/vg00/rlvolX ...
You need to keep a careful track of where your tape is when restoring, is it rewound ? is it at position X (Y lvols already done, Z to go..)

[Carlos Fernandez Riera]

Try to read tape with dd to /dev/null

dd if=/dev/rmt/2mn of=/dev/null bs=1024k..


Try to write directly from lvol to lvol

HOSTB:

remsh hosta dd if=/dev/vgxx/rlvolxx bs=1024k | dd of=/dev/vgyyy/rlvolyyy bs=1024k


Check if the tape is full... or retry whole process again. In the link i pointed i am writing from lvol to lvol and gziped... it is faster. Then you can write the tape from LTO host.

[Bill Hassell]

The I/O error was reported by dd and is most likely being reported by the remote tape device. I would not trust dd and a pipe to remsh as a reliable backup strategy at all. The I/O error cannot not be properly reported or handled and there is absolutely no error checking or verification of the data using this approach.

If this data is important (and you've certainly spent a lot on hardware already), I would purchase a copy of Omniback and use that to transfer the data across the network to tape. Using dd and remsh is the famous: Send-and-Pray Protocol, especially with raw volumes.

[Donghee Lee]

i checked and tried again.. but still couldn't solved problem..
new rlvol is the same size as the old one and surely unmounted..
also tape is rewinded any time i tried dd on hostB.
i think LTO tape is O.K
as listed below, dd from tape to image file worked ..
from image file to rlvol , new one on hostB , also worked..


test.sh
-------
dd if=/dev/vg00/rlvol6 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k
dd if=/dev/vg00/rlvol5 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k

hostA
-------
hostA:/tmp> sh test.sh
600+0 records in
600+0 records out
0+10636 records in
0+10636 records out
800+0 records in
800+0 records out
0+14254 records in
0+14254 records out

hostB
-------
hostB:/tmp/donghee> mt -t /dev/rmt/2m rew
hostB:/tmp/donghee> dd if=/dev/rmt/2mn of=/dev/vg00/rlvol4 bs=1024k
I/O error
0+1 records in
0+1 records out

hostB:/tmp/donghee> mt -t /dev/rmt/2m rew
hostB:/tmp/donghee> dd if=/dev/rmt/2mn of=/omdata5/test.1 bs=1024k
0+10636 records in
0+10636 records out
hostB:/tmp/donghee> dd if=/omdata5/test.1 of=/dev/vg00/rlvol4 bs=1024k
600+0 records in
600+0 records out


hostB:/omdata5>mount /dev/vg00/lvol4 /usr_test
vxfs mount: /dev/vg00/lvol4 is corrupted. needs checking
hostB:/omdata5>fsck -F vxfs -y /dev/vg00/lvol4
log replay in progress
vxfs fsck: file system does not contain a valid log
vxfs fsck: cannot perform log replay
pass0 - checking structural files
pass1 - checking inode sanity and blocks
pass2 - checking directory linkage
pass3 - checking reference counts
pass4 - checking resource maps
OK to clear log? (ynq)y
set state to CLEAN? (ynq)y
hostB:/omdata5>mount /dev/vg00/lvol4 /test_dir


here is another test ( rlvol to rlvol of remote machine )
hostA:/tmp/donghee> dd if=/dev/vg00/rlvol6 bs=1024k | remsh hostB dd of=/dev/vg00/rlvol4 bs=1024k
I/O error
0+1 records in
0+1 records out

when i tried with block size 512k,64k,16k it failed..
but block size 8k is worked ..

any limitation on it..??

[Carlos Fernandez Riera]

yes... It could be something....


The size of the buffer for pipes is 8k, so

( i prefer to begin with remsh)

remsh hostb dd if=.... ibs=1024k | dd of=xxxx obs=1024k

or maybe better:
set bs=8k for all dd. When you read a rlvol the minimum size for bs is 1024b.

I remember that the use of ibs and obs ( even with the same value ibs=8k obs=8k) is a lost of performance.

[Donghee Lee]

thanks Bill ..
i've done tested more with ibs , obs option on hostB
it worked .. no error when restore from tape to raw disks. But very poor perfomance..

here is elapsed time for each step..

hostA ( k460 with EMC disk )
------
dd if=/dev/vg00/rlvol6 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k
dd if=/dev/vg00/rlvol5 bs=1024k | remsh hostB dd of=/dev/rmt/2mn bs=1024k

Elapsed time : 9 min

hostB ( n4000 with XP512 on SAN )
------
mt -t /dev/rmt/2m rew
dd if=/dev/rmt/2mn ibs=1024k of=/dev/vg00/rlvol4 obs=1024k
dd if=/dev/rmt/2mn ibs=1024k of=/dev/vg00/rlvol5 obs=1024k

Elapsed time : 90 min <-- Can you believe this..??

As Bill said.. it's terrible..
is there any option with dd for better perfomance.. ??

on more question...
why the perfmance is like that which ibs , obs option... ??

[Carlos Fernandez Riera]

AM i BIll? :-)

The question is too easy... dd must copy between buffers too, .. not resuse the same buffer. dd have to read several times ( bs=512b by default) to fill 1024k buffer.


I suggest you once again to use gzip. Try it with bs=8k and others ...

[Donghee Lee]

thanks everyone..
i've solved this with obs option on local machine.

machine without tape drive.
------
dd if=/dev/vg00/rlvol6 bs=1024k | remsh hostB dd of=/dev/rmt/2mn obs=1024k

and then restore from tape drive

remote machine with tape drive
------
mt -t /dev/rmt/2m rew
dd if=/dev/rmt/2mn of=/dev/vg00/rlvol4 bs=1024k

it took just 3 minutes handling 1 GB.