topic cp vs cpio in Operating System - HP-UX

cp vs cpio

Lai Nee Shyang_1 — Thu, 22 Mar 2001 03:53:58 GMT

Hi there,

I need to copy about 2.4 Tb of sapdata from 1 storage system to another storage system seen on the same host (with 10 CPU). I've been using "cp -pr", but someone suggest that I use cpio. May I know what are the benefits of using cpio over cp. Some say that cpio can be multi-threaded, meaning it will fully utilize available CPU during copy. Is that true?

Comments please.

Re: cp vs cpio

Peter Brimacombe — Thu, 22 Mar 2001 04:12:11 GMT

2 Tbs that's a lot
I use
find /directory -depth -print | cpio -pd /newdir
to copy anything bigger than 10 million bytes

I think that find | cpio would be faster than cp

Re: cp vs cpio

Printaporn_1 — Thu, 22 Mar 2001 04:20:51 GMT

Hi,

This is the fastest way
- goto the source directory
#find . | cpio -pdumv /newdir

Re: cp vs cpio

Tom Rosenfeld — Thu, 22 Mar 2001 07:31:36 GMT

If you are copying to another device of the exact same size e.g. using Logical Volumes of the same size, you can use dd(1) which is orders of magnitude faster (i.e. gigabytes per hour)

For example:
dd if=/dev/vg05/rlvol1 of=/dev/vginf1/rlvol1 bs=2048k

-tom

Re: cp vs cpio

Paula J Frazer-Campbell — Thu, 22 Mar 2001 08:48:15 GMT

hi
If you are going to use the find option then drop the "v" option as it will slow down the copy using verbose -

best is find .|cpio -pudvm /

HTH

Paula

Re: cp vs cpio

Paul Williams_1 — Thu, 22 Mar 2001 12:40:54 GMT

dd is definitely the quickest method of copying sap data

Re: cp vs cpio

Bernie Vande Griend — Thu, 22 Mar 2001 13:43:56 GMT

One thing to keep in mind is that cpio and tar do not support files over 2GB. I would recommend dd for this. I've also used fbackup|frecover for large copies.

Re: cp vs cpio

Lai Nee Shyang_1 — Fri, 23 Mar 2001 00:36:18 GMT

Thanks guys,

I guess I've to test it out to see which one suits best.

Lai

Re: cp vs cpio

Steven Sim Kok Leong — Fri, 23 Mar 2001 02:58:44 GMT

Hi,

What kind of storage systems are you using? If you are migrating eg. from a Hitachi box to an XP512, you can make use of Remote Copy to copy from one RAID array to another over ESCON links. In this case, cut-over downtime is minimized because data on both RAID arrays are synchronized at all times until the cut-over when the ESCON links are disconnected.

Are there differences between the filesystems residing on both storage systems? If you are copying at the OS level, do note that a difference in block size between the source filesystem and destination filesystem does matter, especially so when you have multiple small files. The default block size for vxfs is 1 kb while the default block size for hfs is 8 kb.

As for choice of OS-level copying tools, I would use one that allows me to continue an interrupted copying process (eg. network got disconnected) where I left off, rather than recopying everything again, especially so if I am going to copy 2 TB at one go.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com

Re: cp vs cpio

Lai Nee Shyang_1 — Fri, 23 Mar 2001 03:36:37 GMT

Hi Steven,

I'm migrating data from a XP512 box to a XP256 box frequently (about once per fortnight), both seen on the same hosts powered by 10 CPUs. We did ask HP about Remote copy via Escon link but it will take roughly 2 full days to do a complete resync and they cannot promise a delta resync after spliting. So I've to resort to copying. Both source and destination are using same block size (both for sapdata).

I've tried copying a 2Gig sapdata via cp, cpio and dd with "time" cmd to get the process timing. The cp took 51.3s and cpio took 58.5. I've problem with the dd command. Here's the syntax "time dd if=/dev/vgd02/testing of=/dev/vgd01/testing2 bs=1024k"
Anything wrong with that?

Thanks

Lai

Re: cp vs cpio

Steven Sim Kok Leong — Fri, 23 Mar 2001 06:50:41 GMT

Hi,

The character special (raw) device file should always be used for devices. Try using the raw device file /dev/vgp01/rtesting instead of the block device file /dev/vgp01/testing etc. instead.

Hope this helps. Regards.

Steven Sim Kok Leong
Brainbench MVP for Unix Admin
http://www.brainbench.com