DD performance - copying volumes...

[Filipe_1]

Hi folks,

In order to copy volumes as fast as we can between servers, we are doing the following:

- Importing the BCVs from the original server on the target server.

- We created volumes on the target that are similar in size to the volumes in the BCV

- We import the original's VGs on the target.

- To make things as fast as possible we issued a dd from volumes of different VGs, as the following:

if=/dev/vgbcv/lv_u05 of=/dev/vgtarget/lv5_u05 bs=8192k
if=/dev/vgbcv/lv_u04 of=/dev/vgtarget/lv5_u04 bs=8192k

(...)

The fact is that it is taking forever.

Despite disk bottleneck issues, anyone have a clue if this is realyy the fastest way to accomplish the copy or if there are some dd's configuration that could speed up things?

Thanks in advance,

Filipe.

[RAC_1]

What are the sizes of the lvols being copied?
Also try to match dd bs size to that of fs size. If all lvols are vxfs, you should look at vxdump

Anil

[Steven E. Protter]

How fast the copy goes can be effected by external factors such as other activity on the system.

We've had reasonably fast results by using fbackup to back up to a Ultrium tape and then restore after the logical volume and filesystem have been set up again.

The write seems a bit slow, back to disk, but thats an issue of how the disk is presented to the HP-UX server and I can't control that.

SEP

[Filipe_1]

Hi folks,

The total amount of information that we are trying to copy is 250gb.

We are trying to copy unmounted devices because we assumed that it would be faster than copying mounted filesystems. The copy is still running and we started it four hours ago.

Would vxdump help us? Can it be quickier than copyng volumes via DD?

[Steven E. Protter]

You need to run a specific test, but vxdump may be able to go faster.

The reason why tape is so fast, is that the new tape format is fast. Plus the write to tape is really a big, gigantic write that completes with nothing else for bandwidth on the tape.

Ultirum 430 tape drives could handle the copy.

I'd futher suggest setting up the two mount filesystems and using a simple cp -r or mv command or scp -rp between systems. The latter would max out your network bandwidth and should be done off peak.

SEP

[Duncan Edmonstone]

In similar migrations I have done, I've always found dd to be the slowest way of doing this - of the standard OS tools I've always found that cpio was the fastest - never tried it with vxdump though.

HTH

Duncan

[A. Clay Stephenson]

You need to be using the raw LVOL's

dd if=/dev/vgbcv/rlv_u05 of=/dev/vgtarget/rlv5_u05 bs=1024k &
dd if=/dev/vgbcv/rlv_u04 of=/dev/vgtarget/rlv5_u04 bs=1024k &

I would drop about 4-6 of these at a time into background followed by a wait and then another 4-6.

There is no point in increasing bs above 1024k because the OS will break larger i/o transfers into a series of smaller physical i/o's.

[Filipe_1]

Folks,

1. There is no tapes involved. This is just a volume to volume copy from different VGs, one of them are BCVs.

2. What the gain I could have using raw devices istead of cooked ones? Since I am using a 8m buffer, would I gain something using the buffer cache (rdevs) AND dd buffer or what?

I undestand that over some buffer size I wouldn't have any significant increase of the copy performance, AFAIK.

Thanks a bunch,

Filipe.

[RolandH]

Clay give a good advice and to increase the performance still a bit more you can switch of
the immediate reporting (*) of the target disk before you start the dd.

use the scsictl command example
# scsictl -a -m ir=1 -m ir /dev/rdsk/c7t1d0
immediate_report = 0; queue_depth = 8; immediate_report = 1
# dd ...
# scsictl -a -m ir=0 -m ir /dev/rdsk/c7t1d0

You should NOT forget to switch it off after the dd is completed. This is a dangerous option. You can do it for this kind of copy jobs. But you should never switch it on for disk that hold critical data.

Roland


(*)
For devices that support immediate reporting, this mode controls how the device responds to write requests. If immediate report is enabled (1), write requests can be acknowledged before the data is physically transferred to the media. If immediate report is disabled (0), the device is forced to await the completion of any write request before reporting its status.

[A. Clay Stephenson]

No, you want to be using the raw device nodes because you don't want the buffer cache. Remember, in your case, EVERY disk block is going to be read and EVERY disk block is going to be written. There is no point in adding the extra overhead of writing the data to the buffer cache or searching the buffer cache for those blocks. In your case, the cache is the dd buffer. For the same reason you do not want to specify ibs=xxx and obs=xxx rather than simply bs=xxx because specifying both involves the overhead of an input to output buffer (internal to dd) copy. It won't do any harm to go larger than 1MB; it simply won't do any good. If you have multiple dd's going simultaneously (as you should) then the less memory pressure the box is under the better although 8MB per dd is tiny.

You are going to get your best performance if you are able to split your i/o's as much as possible over separate SCSI channels. If you have primary and alternate paths to the same physical devices then split across those.

[Filipe_1]

There is something really weird that I want to share with you.

The copy of a 5Gb volume using raw devices:

dd if=/dev/vg01/rlv_oracle of=/dev/vgora01/rlv5_oracle bs=16384k

312+1 records in
312+1 records out

real 8:17.39
user 0.01
sys 6.87


... And using cooked:

dd if=/dev/vg01/lv_oracle of=/dev/vgora01/lv5_oracle bs=8192k

625+0 records in
625+0 records out

real 23:19.48
user 0.02
sys 2:19.74

Haw on earth could this happen, since these are exactly the same devices???

Regards,

Filipe.

[A. Clay Stephenson · Accepted solution]

See my comments above; it explains it perfectly although your test would have at least seemed more fair had you used identical blocksize's --- but again anything above 1MB is essentially wasted so you are actually fair enough.

When literally every block has to read and written one time, the buffer cache overhead is a waste.

[RolandH]

@ Clay

you can prevent data to go through the data buffer cache for block devices, too.

Since HP-UX 11.11 there is the possibility to use vxtunefs parameter "discovered_direct_iosz".

See manpage of vxtunefs(1m)

vxtunefs use just a configuration file /etc/vx/tunefstab with some parameters for each single mount point.

Roland