topic Re: Copying the filesystem in Operating System - HP-UX

Copying the filesystem

Arun K — Sat, 17 May 2008 11:26:01 GMT

Hi All ,

I have a filesystem of 27 GB which contains lot of tiny files which is 2 KB .

I know the copying will take long time . Suggest me the best and fast method for copying ? cpio / tar / dd / rsync?

Regards
AK

Re: Copying the filesystem

Steven Schweda — Sat, 17 May 2008 12:50:13 GMT

In general, I'd normally use a "tar"
pipeline, but with 27GB you'd probably need
to use a better "tar" than the standard
HP-UX "tar". GNU "tar" should work. Add
compression, if the CPUs are faster than the
network.

For some examples (and excessive discussion),
try a forum search for:
tar pipeline
or:
tar pipeline rsync

If both ends are HP-UX, and you're doing the
whole file system, then vxdump+vxrestore may
be a better choice. Are there ACLs (or any
other OS-specific data) to preserve (which
"tar" may not do)?

Experiment is the most reliable way to
determine which method is fastest.

Re: Copying the filesystem

Yogeeraj_1 — Sat, 17 May 2008 13:49:23 GMT

hi AK,

> I know the copying will take long time . Suggest me the best and fast method for copying ? cpio / tar / dd / rsync?

Question:
Copying from where to where?

If it is from a local file system to local file system, then
cp -p -R

If it is from a local file system to a nfs mount point, then
tar -cvf /largefs/ALLFILE.TAR ./
gzip -S .Z /largefs/ALLFILE.TAR
cp /largefs/ALLFILE.TAR.Z

We can many other scenarios, please clarify what scenario is yours!

revert!
kind regards
Yogeeraj

Re: Copying the filesystem

Victor Fridyev — Sat, 17 May 2008 15:03:41 GMT

The answer depends on the target. If you need to clone the source, i.e. target file system is empty and has exactly the same size, the best way is dd:
dd if=/dev/vgsrc/rlvolsrc of=/dev/vgtrgt/rlvoltrgt bs=32768k ,
otherwise cpio or rsync. Rsync is the best if you need to synchronize file systems daily/weekly.
HTH

Re: Copying the filesystem

Arun K — Sat, 17 May 2008 16:18:47 GMT

The scenario is i have to recreate the VG due to PE Size value set to 4 MB . So i have created a new vg and i am movinng the data from the old VG to new VG .

I have around small txt and html files for around 27 GB . So i started copying thru cpio but the data copying speed is less than a GB/hr .

Please let me know the inputs.

Re: Copying the filesystem

Steven Schweda — Sat, 17 May 2008 16:43:40 GMT

> cp -p -R

For why this may be a very bad idea, try that
forum search suggested earlier.

> tar -cvf /largefs/ALLFILE.TAR ./
> gzip -S .Z /largefs/ALLFILE.TAR
> cp /largefs/ALLFILE.TAR.Z

If you have unlimited space for the (totally
unnecessary) temporary files. (And ".Z" is
normally used with "compress", ".gz" with
"gzip". Why work so hard to add potential
confusion?)

> We can many other scenarios, [...]

Are all your suggestions as clever as these?

> Rsync is the best if you need to
> synchronize file systems daily/weekly.

Is that what's happening here?

> [...] i am movinng the data from the old VG
to new VG .

On one system? Local disks or network?
"less than a GB/hr" sounds pretty slow to me,
and I'm accustomed to old, weak junk.

> So i started copying thru cpio [...]

How, exactly? (Actual command(s)?)

Re: Copying the filesystem

Arun K — Sat, 17 May 2008 17:11:23 GMT

I am copying the data in the local system , one VG to another VG . Total data - 201 GB

i have used the below command ..

find . -depth -print | cpio -pdlmv destination path

The above command copied 170 GB data in 18 hrs and due to the lack of time i have stopped coyping the data .

Again i have started copying the remaining 30 GB using the rsyn and it took 12 hrs still the data copying is not completed .

There are lot of tiny files of 1 KB size which rounds up to 30 GB.

After 12 hrs of copying iam still left with 6 GB data out of 30 GB

Regards
AK

Re: Copying the filesystem

Victor Fridyev — Sun, 18 May 2008 08:15:11 GMT

Hi,

In the case when you need to copy your data to a new destination once, dd is the best, you can copy either all disk or only one logical volume. My experience with rp3440 and internal disk gives 45 minutes for 73 GB

HTH