General
cancel
Showing results for 
Search instead for 
Did you mean: 

rsync seems very slow between local volume groups

 
SOLVED
Go to solution
Timothy P. Jackson
Valued Contributor

rsync seems very slow between local volume groups

Hello,

I am working on a DR script on a system that is in production 24x7.

system: rx3600
OS: HP-UX 11i version 3

I have two logical volumes . I need to sync the data on lvol1 to lvol2. My first thought was to use rsync, but now it seems so slow I don't know if it is the best option.

I am syncing about 45 Gigs of data from /dir1 to /dir2 and the sync is taking about 40 minutes with out coping any data. Because I the company has been on holiday for the past week none of the data on the production side /dir1, there have been no updates.

The command I am using is....

rsync -avcu --delete --stat /dir1/ /dir2/

How can I speed this process up or should I be looking for another solution?

Any help would be greatly appreciated!

Thanks,
Tim
15 REPLIES 15
Mel Burslan
Honored Contributor

Re: rsync seems very slow between local volume groups

rsync is not a speed daemon by any means regardless you are doing it locally or to a remote system. Especially when you are talking about updating a 40+GB storage left out of sync for a long period of time.

But I have to ask one question:did this same setup work faster in the past ? If it did, what happened since then ? If this is a new setup and you are not happy with the speed, the only thing you can do is a) if the disk are on the SAN, use a SAN frame based data replication utility of your choice (business copy for HP XP, or Business Continuity Volumes aka BCVs for EMC are some examples); b) if the disks are local, you can use UX-Mirror to create a mirror copy of the disk drive but I am not sure how this can be sued as a DR solution.
________________________________
UNIX because I majored in cryptology...
Alzhy
Honored Contributor

Re: rsync seems very slow between local volume groups

If your Filesystems contain gazillion files and directories or are uber active -- RSYNC is not the correct prescription.

You will be better off using vxdump/vxrestore (if your filesystems are vxfs) or fbackup/frestore.

You may also want to look at using LVM Snapshots and pvmove.
Hakuna Matata.
Timothy P. Jackson
Valued Contributor

Re: rsync seems very slow between local volume groups

Hi Mel,

Both logical volumes are local and mirrored, so let me explain my logic which could have flaws in it.

Production is running 24x7 so my best bet to get a clean copy of the data is to...

1) pause the database engine
2) rsync the data to another file system
3) enable the database engine
4) transfer the synced data off site

Because it is taking so long to sync the database I cannot pause the database engine for that amount of time. So I modified my logic to the following.

1) pause the database engine
2) break the mirror on lvol1
3) enable the database engine
4) rsync lvol1 to lvol2
5) merge the mirror on lvol1
6) transfer the synced data off site.

I would like to keep the mirrors broke as short of time as possible.
Alzhy
Honored Contributor

Re: rsync seems very slow between local volume groups

SOunds like your DBAs need to learn how to use RMAN...

Then yo can ship out those RMAN files offsite
Hakuna Matata.
Timothy P. Jackson
Valued Contributor

Re: rsync seems very slow between local volume groups

I have never used vxdump or vxrestore before. Is it faster than fbackup/frecover? does it use a lot of resourses?
Mel Burslan
Honored Contributor
Solution

Re: rsync seems very slow between local volume groups

In which case, suggestion of vxdump/vxrestore made by Alzhy, may be a better approach for you speed-wise. rsync is a file-by-file operation whereas vxdump/vxrestore do their magic by copying volume data regardless of the number of files or their contents.

Years ago, during a SAN migration exercise, we have gained abiout 2.5 times speed advantage using vxdump/vxrestore over doing a plain cp (which is what rsync basically is) but this was a special case to our setup, using an EMC source array and XP1024 as a target. So, your mileage may vary.
________________________________
UNIX because I majored in cryptology...
Timothy P. Jackson
Valued Contributor

Re: rsync seems very slow between local volume groups

Oh Yeah...

The file systems are vfxs and are both local.
Mel Burslan
Honored Contributor

Re: rsync seems very slow between local volume groups

I second the RMAN option. Any modern backup solution have a rman module and back those puppies up to a tape or a remote disk. Takes the load off your shoulders as well. I like the idea :)
________________________________
UNIX because I majored in cryptology...
Timothy P. Jackson
Valued Contributor

Re: rsync seems very slow between local volume groups

unfortunately this is not an Oracle database it is a IBM/U2 unidata database. I am going to try the vxdump/vxrestore and see if it helps.