Help on "RSYNC" command.

[Mike_305]

Hello ITRC,

I am in process to copy huge numbers of NFS DATA to local drives and I know very little about "rsync". Does anyone has suggestion on copying the data in a script format.

I also know this works similar to "CP" command in Unix. This is what I am using and it's going from NFS mount to local file system on same HOST.

rsync -a -v -R -L /nfs/xyz /abc

Thanks in advance and appreciate your help.


Regards,

Mike

[Mike_305]

Hello Guru's,

Please help. Looks like no love on this post.

Regards,

M

[TwoProc]

OK... Mike.

First, you need a way to get those rsync's happenin'. I use ssh to make this work, myself, so I'm going to assume you've got that working too.

Suppose I've got data on /home/bob on a remote server and I want to get that sync'd with /home/bob on the destination, with the destination computer being kept in sync with the remote server.

$> rsync -avze ssh sourceserver:/home/bob/
/home/bob/

the "e" flag lets me run that rsync in the ssh tunnel. the "z" flag compresses the data in and out of the tunnel on each side. the first argument "sourceserver:/home/bob/" tells the rsync command where to sync from. The "/home/bob/" command tells rsync where to land the data.

It's pretty simple from there. The one thing to watch for is that the *TRAILING BACKSLASH /* is important! If you're doing directory to directory syncing, be sure and don't leav it out.

Of course the command would work fine without the "z" command to run without compression. Of course, it's possible that the transfer could be faster without compression, so be sure to test it and time the sync process both with and without compression.

There is one last item to discuss. You can use the "--delete" flag to have the local copy's file removed if the don't exist on the source server. By default, this does not occur. An example would be possibly log files that you accumulate on the dest server that you don't want deleted. Of course, that would be log files, that would be unique to each servers, if they had the same name, they'd just be overwritten. In any case you'll have to determine if you want a "FULL SYNC" with the source area, or if you just want new and different files, but not to lose any files unique to the destinaatin location only. So the above command would becomes

$> rsync -avze ssh --delete sourceserver:/home/bob/ /home/bob/

Oh, and keep in mind that an rsync command can rsync to even the same server that you're on, if you like to sync two directories on the same server.

$> rsync -avze ssh --delete /home/oldbob/ /home/newbob/

You can also use it to sync a remote source, and a remote destionation!

$> rsync -avze ssh --delete sourcserver:/home/oldbob/ \
destserver:/home/newbob/

(The backslash above is just because this posting formatted across two lines, if it were all on one line, you'd not need the backslash.)

HTH, and good luck!

[Mike_305]

Hello,

First I want to say thanks for the reply and the issue is not the rsync. I got the rsync working. The data that I am copying it's going from LOCAL NFS mount to the "LOCAL FILE SYSTEMS" on SAME HOST.

And, when I am using "z" options to compress all my DATA, output to the LOG files is not showing. Not sure if the OPTIONS "z" is supressing the error log that I am redirection to the file.

The command that I am using is as below.

/usr/local/bin/rsync -v -a -r -c -p -o -g -p -t -E -z /nfs/abc/ /nfs/xyz/ >> /home/tes/file_copy.log 2>&1

Any idea.

Regards,

Mike

[Mike_305]

Thanks Guys! appreciate your help.

[TwoProc]

Wow, Mike - sorry about that - but I can fix your problem. And, no - the "z" isn't eating the log file.

Funny thing is, I almost posted for you the way to make the log file - but didn't want to drag on longer than I was.


rsync -a -v -R -L /nfs/xyz /abc 2>&1 | tee -a /mylogdir/rsynclog.$$


This way, the conc mgr mailer will have a copy of the log file when it mail to system, and the logdir will have a copy too.

The 2>&1 command combines standard error into the standard output - and this, more than anything is probably where you were going wrong. A few programs send all or most of the output to standard error, even when it's not an error - and that could be what you're running up against.

[TwoProc]

Wellll Crud. I see that you did have standard error combined.

Then... try mine with the tee command. I use it that very way and get a log file from a background process every time.