topic Shell/perl script to sync data between remote systems in Operating System - HP-UX

Shell/perl script to sync data between remote systems

ShivKumar_1 — Wed, 17 Jun 2009 22:51:45 GMT

Hi
I am looking for a solution other than "rsync" to copy files between two systems.

I will be given a file which would contain enteries

---------------------------------------
1. abc 11111 c
2. bbb 22222 d
3. ccc 33333 c
4. ddd 44444 d

So all the columns which say 'c" have to be copied to remote system.

The ones which say "d" have to be deleted and I need to maintain a log of what was done.

rsync is not an option as its slow. Does any one have any ideas how to accomplish this?

Thnks
S

Re: Shell/perl script to sync data between remote systems

James R. Ferguson — Wed, 17 Jun 2009 23:11:11 GMT

Hi:

Why not 'rsync' which minimizes traffic by sending only what has changed? It handles deleted files, too.

If you are periodically refreshing an entire directory on a target server, you could craft a script that deletes the directory contents on the target; creates an archived copy of everything in the source directory (e.g. a 'tar'ball) on the source server; transfers the archive to the target server; and installs it there. This can all be done using 'ssh' for security, too.

Otherwise, I have to ask, is this merely an academic exercise?

Regards!

...JRF...

Re: Shell/perl script to sync data between remote systems

VK2COT — Thu, 18 Jun 2009 00:02:15 GMT

Hello,

As James said, I do not think rsync is a bad
option at all. It looks like your task
might be a school assignment :)

I used it at many commercial companies with
lot of success and throughput was never an
issue (unless your network is saturated
already).

Plus, there are a whole lot of tools to use
with rsync. Some of them are here:

http://unix.freshmeat.net/search/?Go.x=1&Go.y=1&q=rsync&section=projects

As well, rsync has many options. How are
you using it currently? What are the flags
you use on the command line?

Anyway, if you want some other alternatives:

Commercial software SyncDat:
http://www.dataexpedition.com/syncdat/features.html

Open-source (some of them are for Linux only):
Unison http://www.cis.upenn.edu/~bcpierce/unison/
plsync http://sourceforge.net/projects/plsync/
Zumastor http://www.zumastor.org/
PowerFolder http://sourceforge.net/projects/powerfolder-/

Ruby script:
http://eigenclass.org/hiki/cheap+rsync

WARNING: There is also a commercial product called PowerFolder for Microsoft Windows.
Do not mix it up with the SourceForge
project!

So, if you really want to use some other, plain Shell or Perl script, it is probably
easiest to implement some kind of diff(1)
or and or cmp(1) commands to check if local and remote files are different and if so,
copy a new one.

Cheers,

VK2COT

Re: Shell/perl script to sync data between remote systems

Steven Schweda — Thu, 18 Jun 2009 02:48:47 GMT

> rsync is not an option as its slow.

Says who? (And that's "it's".)

If you don't like rsync, how _were_ you
planning to copy files to and/or delete files
on the (unspecified) "remote system"?

> Does any one have any ideas how to
> accomplish this?

I might write a shell script to do it. Why
don't you?