cp vs cpio

[Lai Nee Shyang_1]

Hi there,

I need to copy about 2.4 Tb of sapdata from 1 storage system to another storage system seen on the same host (with 10 CPU). I've been using "cp -pr", but someone suggest that I use cpio. May I know what are the benefits of using cpio over cp. Some say that cpio can be multi-threaded, meaning it will fully utilize available CPU during copy. Is that true?

Comments please.

[Peter Brimacombe · Accepted solution]

2 Tbs that's a lot
I use
find /directory -depth -print | cpio -pd /newdir
to copy anything bigger than 10 million bytes

I think that find | cpio would be faster than cp

[Printaporn_1]

Hi,

This is the fastest way
- goto the source directory
#find . | cpio -pdumv /newdir

[Tom Rosenfeld]

If you are copying to another device of the exact same size e.g. using Logical Volumes of the same size, you can use dd(1) which is orders of magnitude faster (i.e. gigabytes per hour)

For example:
dd if=/dev/vg05/rlvol1 of=/dev/vginf1/rlvol1 bs=2048k

-tom

[Paula J Frazer-Campbell]

hi
If you are going to use the find option then drop the "v" option as it will slow down the copy using verbose -

best is find .|cpio -pudvm /


HTH


Paula

[Paul Williams_1]

dd is definitely the quickest method of copying sap data

[Bernie Vande Griend]

One thing to keep in mind is that cpio and tar do not support files over 2GB. I would recommend dd for this. I've also used fbackup|frecover for large copies.

[Lai Nee Shyang_1]

Thanks guys,

I guess I've to test it out to see which one suits best.

Lai

[Steven Sim Kok Leong]

Hi,

What kind of storage systems are you using? If you are migrating eg. from a Hitachi box to an XP512, you can make use of Remote Copy to copy from one RAID array to another over ESCON links. In this case, cut-over downtime is minimized because data on both RAID arrays are synchronized at all times until the cut-over when the ESCON links are disconnected.

Are there differences between the filesystems residing on both storage systems? If you are copying at the OS level, do note that a difference in block size between the source filesystem and destination filesystem does matter, especially so when you have multiple small files. The default block size for vxfs is 1 kb while the default block size for hfs is 8 kb.

As for choice of OS-level copying tools, I would use one that allows me to continue an interrupted copying process (eg. network got disconnected) where I left off, rather than recopying everything again, especially so if I am going to copy 2 TB at one go.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com

[Lai Nee Shyang_1]

Hi Steven,

I'm migrating data from a XP512 box to a XP256 box frequently (about once per fortnight), both seen on the same hosts powered by 10 CPUs. We did ask HP about Remote copy via Escon link but it will take roughly 2 full days to do a complete resync and they cannot promise a delta resync after spliting. So I've to resort to copying. Both source and destination are using same block size (both for sapdata).

I've tried copying a 2Gig sapdata via cp, cpio and dd with "time" cmd to get the process timing. The cp took 51.3s and cpio took 58.5. I've problem with the dd command. Here's the syntax "time dd if=/dev/vgd02/testing of=/dev/vgd01/testing2 bs=1024k"
Anything wrong with that?

Thanks

Lai

[Steven Sim Kok Leong]

Hi,

The character special (raw) device file should always be used for devices. Try using the raw device file /dev/vgp01/rtesting instead of the block device file /dev/vgp01/testing etc. instead.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com