Copy thousands and thousands of file

[Hooi Siew Hoong_1]

Hi,

I have a VG with 1 filesystem, 28GB in size, stripped across 4 EMC 8GB luns. I wanted to migrate this filesystem to a 35GB XP lun.

The problem is since this filesystem is striped, I could not pvmove or perform a mirror extend and reduce online to migrate the data. We are forced to perform a manual copy offline.

The we realised that there are a few directories in the filesystem contains thousands of file. Several directories contain excess of 50k number of files. One directory contains 150k number of files and the one contains 600k files. We could not possibly copy all the files within the limited downtime.

Is there a faster way to migrate the filesystem? Or can we use dd command on the LV device file to copy?

Any suggestion is appreciated. Currently we are using "find -xdev -print|cpio -pxdm " to manual copy.

Thanks,

[Robert-Jan Goossens]

Hi,

This is always a problem; if you have a good backup product I would use it. Backup to tape, and restore to the new filesystem.

the find in combination with cpio is quit fast, but with a few hunderd thousand files this will take hours. I would remove any verbose output from the command to increase speed.

# find /souce -xdev | cpio -pcmud /target

Hope this helps a bit,

Kind regards,
Robert-Jan

[Roberto Polli]

why doesn't tar match for the quest?

Peace, R.

[Geoff Wild]

Try vxdump:

vxdump -0 -f - -s 1000000 -b 16 /zmnt/emc/mount | (cd /xpmount ; vxrestore rf -)&

where /zmnt/emc/mount is the mount point on the old frame and /xpmount is the mout on the new one.

I've done frame to frame migrations in the past, and when mirroring wasn't available, vxdump was the fastest.

Rgds...Geoff

[Volker Borowski]

Well,

I never did this for a striped filesystem, but for database-containers on rawdevices. A dd on lvol-level should work. The filesystem attributes should remain the same. You need to create an lvol of the same size as source in your 35G LUN, which you can extend later on.

LE-sizes and physical attributes should be the same. Unmount, dd the source-r(!)lvol to the target, fsck, mount (of course with same filesystem attributes) and there you are.

Of course you should do a test before, may be with a smaller filesystem of 1GB.

Good luck
Volker

[Todd McDaniel_1]

How big can these files be if there are 750k files? What is your window to cp? Are these dynamic files or static?



I would think the:

cd fromdir ; tar cf - . | ( cd todir ; tar xf - )


would work...

Not sure if DD would be faster, Just make sure the extent size is the same.

[Bruno Ganino]

Read this thread could be useful
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=27681
HTH
Bruno

[Hooi Siew Hoong_1]

Thanks for all the suggestions. I would try all these out and let you guys know which works the best.