Operating System - HP-UX
1821801 Members
3049 Online
109637 Solutions
New Discussion юеВ

Best Performance using dd command

 
AwadheshPandey
Honored Contributor

Best Performance using dd command

Dear Experts,

I want to clone few disks using dd command. Pls let me know the best options to get better performance. Also let me know if I can use multipe dd on single system with optimal performance.

Thanks in advance.

Regards,

Awadhesh
It's kind of fun to do the impossible
7 REPLIES 7
James R. Ferguson
Acclaimed Contributor

Re: Best Performance using dd command

Hi:

Use a large block size and use the raw disk devices so that the buffer cache isn't brought into play:

# dd if=/dev/rdsk/c1t2d3 of=/dev/rdsk/c4/t5/d6 bs=1024k

You can use as many 'dd' processes at once as your I/O subsystem can tolerate and your end-users aren't impacted.

Regards!

...JRF...
Pete Randall
Outstanding Contributor

Re: Best Performance using dd command

There is little I can add to Jim's comments. The two keys are using the raw device and a large block size. Note the "k" in "bs=1024k"! Not specifying a block size at all defaults to 512 bytes and takes FOREVER!!


Pete

Pete
Jaime Bolanos Rojas.
Honored Contributor

Re: Best Performance using dd command

Awadhesh,

Sure this thread will help you in your task:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1225976906699+28353475&threadId=833830

But bottom line, dd will not get you good performance, it just wasn├В┬┤t made for that.

Regards,

Jaime.
Work hard when the need comes out.
Jaime Bolanos Rojas.
Honored Contributor

Re: Best Performance using dd command

Also on 11.11 systems, the vps_pagesize kernel parameter can be tuned to affect
performance. For example, a typical value to set the parameter to is 8k

Regards,

Jaime.
Work hard when the need comes out.
Bill Hassell
Honored Contributor

Re: Best Performance using dd command

dd is the fastest method there is in HP-UX to copy disk tracks. Be sure you use the raw device file file (either /dev/rdsk/... or /dev/vgxyz/rlvol...) to bypass the kernel's buffer cache. You can start multiple copies but you do need to be aware of the possible impact on the system. The effect of dd varies widely depending on whether you have simple disks (JBOD) or a multi-pathed disk array.


Bill Hassell, sysadmin
Ninad_1
Honored Contributor

Re: Best Performance using dd command

Hi Awadhesh,

I would suggest you do a test with different block sizes - starting from 1024k, 2048k, 4096k ..
I vaguely remember I had done some testing to understand exactly what you have asked and if I remember correctly I had observed that dd was faster upto a particular block size (I think it was 16k) beyond which I could not see any performance improvements.
I had thought that this could be something to do with the chunk size of the Storage array configured LUNs presented to the server - but not sure as later I understood that the storage arrays would use their cache to get the data writeen which in turn is actually written to disks.

One more important thing I would suggest to you is how you will track the log of the dd if you are running multiple dd in parallel and redirecting the o/p & error to a common file - then it would be difficult to know which dd completed successfully and which did not.
So while logging the output/errors - you may wish to use different log files for each dd that will be done.

Regards,
Ninad
AwadheshPandey
Honored Contributor

Re: Best Performance using dd command

I have done various tests and found dd with 512MB block size gave best results. Also I did multiple instances of dd using various sessions, Now I am confident using dd.
thanks Gurus for your support.
It's kind of fun to do the impossible