topic Best Performance using dd command in Operating System - HP-UX

Best Performance using dd command

AwadheshPandey — Thu, 06 Nov 2008 12:58:50 GMT

Dear Experts,

I want to clone few disks using dd command. Pls let me know the best options to get better performance. Also let me know if I can use multipe dd on single system with optimal performance.

Thanks in advance.

Regards,

Awadhesh

Re: Best Performance using dd command

James R. Ferguson — Thu, 06 Nov 2008 13:06:31 GMT

Hi:

Use a large block size and use the raw disk devices so that the buffer cache isn't brought into play:

# dd if=/dev/rdsk/c1t2d3 of=/dev/rdsk/c4/t5/d6 bs=1024k

You can use as many 'dd' processes at once as your I/O subsystem can tolerate and your end-users aren't impacted.

Regards!

...JRF...

Re: Best Performance using dd command

Pete Randall — Thu, 06 Nov 2008 13:09:33 GMT

There is little I can add to Jim's comments. The two keys are using the raw device and a large block size. Note the "k" in "bs=1024k"! Not specifying a block size at all defaults to 512 bytes and takes FOREVER!!

Pete

Re: Best Performance using dd command

Jaime Bolanos Rojas. — Thu, 06 Nov 2008 13:12:04 GMT

Awadhesh,

Sure this thread will help you in your task:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1225976906699+28353475&threadId=833830

But bottom line, dd will not get you good performance, it just wasnÂ´t made for that.

Regards,

Jaime.

Re: Best Performance using dd command

Jaime Bolanos Rojas. — Thu, 06 Nov 2008 13:17:25 GMT

Also on 11.11 systems, the vps_pagesize kernel parameter can be tuned to affect
performance. For example, a typical value to set the parameter to is 8k

Regards,

Jaime.

Re: Best Performance using dd command

Bill Hassell — Fri, 07 Nov 2008 02:28:30 GMT

dd is the fastest method there is in HP-UX to copy disk tracks. Be sure you use the raw device file file (either /dev/rdsk/... or /dev/vgxyz/rlvol...) to bypass the kernel's buffer cache. You can start multiple copies but you do need to be aware of the possible impact on the system. The effect of dd varies widely depending on whether you have simple disks (JBOD) or a multi-pathed disk array.

Re: Best Performance using dd command

Ninad_1 — Fri, 07 Nov 2008 03:00:07 GMT

Hi Awadhesh,

I would suggest you do a test with different block sizes - starting from 1024k, 2048k, 4096k ..
I vaguely remember I had done some testing to understand exactly what you have asked and if I remember correctly I had observed that dd was faster upto a particular block size (I think it was 16k) beyond which I could not see any performance improvements.
I had thought that this could be something to do with the chunk size of the Storage array configured LUNs presented to the server - but not sure as later I understood that the storage arrays would use their cache to get the data writeen which in turn is actually written to disks.

One more important thing I would suggest to you is how you will track the log of the dd if you are running multiple dd in parallel and redirecting the o/p & error to a common file - then it would be difficult to know which dd completed successfully and which did not.
So while logging the output/errors - you may wish to use different log files for each dd that will be done.

Regards,
Ninad

Re: Best Performance using dd command

AwadheshPandey — Fri, 21 Nov 2008 10:30:45 GMT

I have done various tests and found dd with 512MB block size gave best results. Also I did multiple instances of dd using various sessions, Now I am confident using dd.
thanks Gurus for your support.