dd and its block size

[Nguyen Anh Tien]

Hi Gurus,
I have 02 vg as below
1, VG Name /dev/vgtaffeva
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 16
Cur PV 2
Act PV 2
Max PE per PV 65535
VGDA 4
PE Size (Mbytes) 32
Total PE 28796
Alloc PE 28175
Free PE 621
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 33553920m
VG Max Extents 1048560
--- Logical volumes ---
LV Name /dev/vgtaffeva/lvtaff
LV Status available/syncd
LV Size (Mbytes) 901600
Current LE 28175
Allocated PE 28175
Used PV 2
2, --- Volume groups ---
VG Name /dev/vgpkg2
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 200
Cur PV 40
Act PV 40
Max PE per PV 65535
VGDA 80
PE Size (Mbytes) 128
Total PE 15960
Alloc PE 15960
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 1638375g
VG Max Extents 13107000
--- Logical volumes ---
LV Name /dev/vgpkg2/lvol1
LV Status available/syncd
LV Size (Mbytes) 2042880
Current LE 15960
Allocated PE 15960
Used PV 40

NOW.I WANT TO USE DD TO COPY DATA BETWEEN LVS
#dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=1024k
My question are:
1, How to monitor how many percent copied???
2, what bs is best in this case???
Does any one has experienced with these??
thanks for your help
Regards
Tien

[Dennis Handly]

>1. How to monitor how many percent copied?

I don't think you can. Unless you use gpm to look at the file offsets.

>2. what bs is best in this case?

As large as possible. With nearly 1 Tb to copy, you probably need more than 1 Mb for your blocksize.

[Ninad_1]

Hi,

I have not tested this myself nad may need some corrections but I feel what you need can be done in following way (Only it will add its processing time to total copying time)

STEP=0
while [[ $STEP -lt 5635 ]] ; do
SKIP=$(echo $STEP \* 5 | bc)
dd if==/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=32768k skip=$SKIP count=5
if [[ $? != 0 ]] ; then
echo "Encountered error while copying at step $STEP"
else
echo "$(echo "($STEP \* 100)/5635" | bc) % completed"
fi
let STEP=$STEP+1
done


NOTE - Pls test this before using as I have not tested a bit of it.

Regards,
Ninad

[Ninad_1]

Hi,

Sorry - forgot to add exit 1 if the dd failed.

Thus the if will now be as

if [[ $? != 0 ]] ; then
echo "Encountered error while copying at step $STEP"
exit 1
else
echo "$(echo "($STEP \* 100)/5635" | bc) % completed"
fi

Regards,
Ninad

[Robert-Jan Goossens]

Hi Tien,

One little remark on your command.

# dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=1024k

should be dd raw logical device to raw device.

# dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/rlvol1 bs=1024k

Regards,
Robert-Jan

[Dennis Handly]

>Ninad: what you need can be done in following way (Only it will add its processing time to total copying time)

Other than increasing the blocksize, why would copying only 5 blocks at a time be any better?
Also the skip= only works on the input file, so you would be overwriting the first 5 blocks.

And for the numbers you have, there is no need to use bc(1):
(( SKIP = STEP * 5 ))

For a numeric compare, the status check should be:
if [[ $? -eq 0 ]]; then

[Ninad_1]

Hi Dennis,

You are right in pointing out the mistake.
I think using seek=5 along with the earlier dd command, would skip the corresponding blocks from the outfile as well, thus there would be no overwriting of the outfile

Why 5 - is because of the number of PEs for the lvol to be divided into some integer divisible figure and to give update at a better frequency. even a count of 25 can be used which will give updates every 800 MB of data copied.

Regards,
Ninad

[Dennis Handly]

>Ninad: I think using seek=5 along with the earlier dd command

Yes.

>to give update at a better frequency.

Ah, you were trying to solve 1).

[chris huys_4]

Hi Tien,

I wouldnt use dd to copy data between the logical volumes.

You are working with a volumemanager, use volumemanager commands, to copy the contents of the data, across.

Or if the data is some raw database file, use the database utilities, who were made for it.

dd is from a time when volumemanagers didnt yet exist..

Greetz,
Chris

[Nguyen Anh Tien]

Thank you all for your replies,
I have a test for choosing bs yesterday.
BS seem to bring best perf is 32M (same as PE size of source lv) Please see attached file for the result.
# n of=/dev/vgpackage2/rlvol1 bs=32768k count=1000 ;date <
Mon Dec 29 14:25:18 TST 2008
1000+0 records in
1000+0 records out
Mon Dec 29 14:28:07 TST 2008
DD MOVES 32GB/3MINS=11GB/MIN

Thank Ninad for your skipp option, I will duplicated at my lab to confirm it able to skip out put file as well.
Thank Denis for your question, It is much help me,
Thank you all
Regards
Tien