1748006 Members
4725 Online
108757 Solutions
New Discussion юеВ

Copying the filesystem

 
Arun K
Occasional Contributor

Copying the filesystem

Hi All ,

I have a filesystem of 27 GB which contains lot of tiny files which is 2 KB .

I know the copying will take long time . Suggest me the best and fast method for copying ? cpio / tar / dd / rsync?

Regards
AK
7 REPLIES 7
Steven Schweda
Honored Contributor

Re: Copying the filesystem

In general, I'd normally use a "tar"
pipeline, but with 27GB you'd probably need
to use a better "tar" than the standard
HP-UX "tar". GNU "tar" should work. Add
compression, if the CPUs are faster than the
network.

For some examples (and excessive discussion),
try a forum search for:
tar pipeline
or:
tar pipeline rsync

If both ends are HP-UX, and you're doing the
whole file system, then vxdump+vxrestore may
be a better choice. Are there ACLs (or any
other OS-specific data) to preserve (which
"tar" may not do)?

Experiment is the most reliable way to
determine which method is fastest.
Yogeeraj_1
Honored Contributor

Re: Copying the filesystem

hi AK,

> I know the copying will take long time . Suggest me the best and fast method for copying ? cpio / tar / dd / rsync?

Question:
Copying from where to where?

If it is from a local file system to local file system, then
cp -p -R

If it is from a local file system to a nfs mount point, then
tar -cvf /largefs/ALLFILE.TAR ./
gzip -S .Z /largefs/ALLFILE.TAR
cp /largefs/ALLFILE.TAR.Z

We can many other scenarios, please clarify what scenario is yours!

revert!
kind regards
Yogeeraj
No person was ever honoured for what he received. Honour has been the reward for what he gave (clavin coolidge)
Victor Fridyev
Honored Contributor

Re: Copying the filesystem

The answer depends on the target. If you need to clone the source, i.e. target file system is empty and has exactly the same size, the best way is dd:
dd if=/dev/vgsrc/rlvolsrc of=/dev/vgtrgt/rlvoltrgt bs=32768k ,
otherwise cpio or rsync. Rsync is the best if you need to synchronize file systems daily/weekly.
HTH
Entities are not to be multiplied beyond necessity - RTFM
Arun K
Occasional Contributor

Re: Copying the filesystem

The scenario is i have to recreate the VG due to PE Size value set to 4 MB . So i have created a new vg and i am movinng the data from the old VG to new VG .

I have around small txt and html files for around 27 GB . So i started copying thru cpio but the data copying speed is less than a GB/hr .

Please let me know the inputs.
Steven Schweda
Honored Contributor

Re: Copying the filesystem

> cp -p -R

For why this may be a very bad idea, try that
forum search suggested earlier.

> tar -cvf /largefs/ALLFILE.TAR ./
> gzip -S .Z /largefs/ALLFILE.TAR
> cp /largefs/ALLFILE.TAR.Z

If you have unlimited space for the (totally
unnecessary) temporary files. (And ".Z" is
normally used with "compress", ".gz" with
"gzip". Why work so hard to add potential
confusion?)

> We can many other scenarios, [...]

Are all your suggestions as clever as these?

> Rsync is the best if you need to
> synchronize file systems daily/weekly.

Is that what's happening here?

> [...] i am movinng the data from the old VG
to new VG .

On one system? Local disks or network?
"less than a GB/hr" sounds pretty slow to me,
and I'm accustomed to old, weak junk.

> So i started copying thru cpio [...]

How, exactly? (Actual command(s)?)
Arun K
Occasional Contributor

Re: Copying the filesystem

I am copying the data in the local system , one VG to another VG . Total data - 201 GB

i have used the below command ..

find . -depth -print | cpio -pdlmv destination path

The above command copied 170 GB data in 18 hrs and due to the lack of time i have stopped coyping the data .

Again i have started copying the remaining 30 GB using the rsyn and it took 12 hrs still the data copying is not completed .

There are lot of tiny files of 1 KB size which rounds up to 30 GB.

After 12 hrs of copying iam still left with 6 GB data out of 30 GB

Regards
AK
Victor Fridyev
Honored Contributor

Re: Copying the filesystem

Hi,

In the case when you need to copy your data to a new destination once, dd is the best, you can copy either all disk or only one logical volume. My experience with rp3440 and internal disk gives 45 minutes for 73 GB

HTH
Entities are not to be multiplied beyond necessity - RTFM