- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: dd and its block size
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-28-2008 10:57 PM
тАО12-28-2008 10:57 PM
dd and its block size
I have 02 vg as below
1, VG Name /dev/vgtaffeva
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 16
Cur PV 2
Act PV 2
Max PE per PV 65535
VGDA 4
PE Size (Mbytes) 32
Total PE 28796
Alloc PE 28175
Free PE 621
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 33553920m
VG Max Extents 1048560
--- Logical volumes ---
LV Name /dev/vgtaffeva/lvtaff
LV Status available/syncd
LV Size (Mbytes) 901600
Current LE 28175
Allocated PE 28175
Used PV 2
2, --- Volume groups ---
VG Name /dev/vgpkg2
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 200
Cur PV 40
Act PV 40
Max PE per PV 65535
VGDA 80
PE Size (Mbytes) 128
Total PE 15960
Alloc PE 15960
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 1638375g
VG Max Extents 13107000
--- Logical volumes ---
LV Name /dev/vgpkg2/lvol1
LV Status available/syncd
LV Size (Mbytes) 2042880
Current LE 15960
Allocated PE 15960
Used PV 40
NOW.I WANT TO USE DD TO COPY DATA BETWEEN LVS
#dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=1024k
My question are:
1, How to monitor how many percent copied???
2, what bs is best in this case???
Does any one has experienced with these??
thanks for your help
Regards
Tien
- Tags:
- dd
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-28-2008 11:47 PM
тАО12-28-2008 11:47 PM
Re: dd and its block size
I don't think you can. Unless you use gpm to look at the file offsets.
>2. what bs is best in this case?
As large as possible. With nearly 1 Tb to copy, you probably need more than 1 Mb for your blocksize.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 01:12 AM
тАО12-29-2008 01:12 AM
Re: dd and its block size
I have not tested this myself nad may need some corrections but I feel what you need can be done in following way (Only it will add its processing time to total copying time)
STEP=0
while [[ $STEP -lt 5635 ]] ; do
SKIP=$(echo $STEP \* 5 | bc)
dd if==/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=32768k skip=$SKIP count=5
if [[ $? != 0 ]] ; then
echo "Encountered error while copying at step $STEP"
else
echo "$(echo "($STEP \* 100)/5635" | bc) % completed"
fi
let STEP=$STEP+1
done
NOTE - Pls test this before using as I have not tested a bit of it.
Regards,
Ninad
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 01:14 AM
тАО12-29-2008 01:14 AM
Re: dd and its block size
Sorry - forgot to add exit 1 if the dd failed.
Thus the if will now be as
if [[ $? != 0 ]] ; then
echo "Encountered error while copying at step $STEP"
exit 1
else
echo "$(echo "($STEP \* 100)/5635" | bc) % completed"
fi
Regards,
Ninad
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 01:23 AM
тАО12-29-2008 01:23 AM
Re: dd and its block size
One little remark on your command.
# dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/lvol1 bs=1024k
should be dd raw logical device to raw device.
# dd if=/dev/vgtaffeva/rlvtaff of=/dev/vgpkg2/rlvol1 bs=1024k
Regards,
Robert-Jan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 01:36 AM
тАО12-29-2008 01:36 AM
Re: dd and its block size
Other than increasing the blocksize, why would copying only 5 blocks at a time be any better?
Also the skip= only works on the input file, so you would be overwriting the first 5 blocks.
And for the numbers you have, there is no need to use bc(1):
(( SKIP = STEP * 5 ))
For a numeric compare, the status check should be:
if [[ $? -eq 0 ]]; then
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 02:18 AM
тАО12-29-2008 02:18 AM
Re: dd and its block size
You are right in pointing out the mistake.
I think using seek=5 along with the earlier dd command, would skip the corresponding blocks from the outfile as well, thus there would be no overwriting of the outfile
Why 5 - is because of the number of PEs for the lvol to be divided into some integer divisible figure and to give update at a better frequency. even a count of 25 can be used which will give updates every 800 MB of data copied.
Regards,
Ninad
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 03:09 AM
тАО12-29-2008 03:09 AM
Re: dd and its block size
Yes.
>to give update at a better frequency.
Ah, you were trying to solve 1).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 08:07 AM
тАО12-29-2008 08:07 AM
Re: dd and its block size
I wouldnt use dd to copy data between the logical volumes.
You are working with a volumemanager, use volumemanager commands, to copy the contents of the data, across.
Or if the data is some raw database file, use the database utilities, who were made for it.
dd is from a time when volumemanagers didnt yet exist..
Greetz,
Chris
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО12-29-2008 07:02 PM
тАО12-29-2008 07:02 PM
Re: dd and its block size
I have a test for choosing bs yesterday.
BS seem to bring best perf is 32M (same as PE size of source lv) Please see attached file for the result.
# n of=/dev/vgpackage2/rlvol1 bs=32768k count=1000 ;date <
Mon Dec 29 14:25:18 TST 2008
1000+0 records in
1000+0 records out
Mon Dec 29 14:28:07 TST 2008
DD MOVES 32GB/3MINS=11GB/MIN
Thank Ninad for your skipp option, I will duplicated at my lab to confirm it able to skip out put file as well.
Thank Denis for your question, It is much help me,
Thank you all
Regards
Tien