rsync issue

[Jeff_Traigle]

We're trying to use rsync to synchronize a file system that resides on a VA7100 to a remote system with an AutoRAID. The VG, LV, and FS were all created with matching sizes (PE size, FS block size, and LV size). However, when we perform the rsync, utilization on the AutoRAID housed file system is twice the utilization of the VA housed file system. The one thing I found a little while ago is that the block length of the LUNs on the VA are 520 bytes while the block length of the LUNs on the AutoRAID are 512 bytes. So it seems to me that rsync is working at the physical block level and is sending over the 520 byte blocks, which require two 512 byte blocks on the receiving end. Does this sound like how rsync works?

[Steven E. Protter]

Shalom,

No, this is not proper funciton of rsync.

What really matters is if the same file ends up at both ends of the connection.

SEP

[A. Clay Stephenson]

The files sizes must be the same as well as the checksums; how the storage utilization of the underlying filesystem is handled is not rsync's problem. What are you using for your measurements? bdf? du? or array specific utilities?

[Jeff_Traigle]

bdf is what I used to compare utilization on each end.

[A. Clay Stephenson]

Taken from this document (which is a pretty good and not too technical read):
http://rsync.samba.org/how-rsync-works.html

"The block size and, in later versions, the size of the block checksum are calculated on a per file basis according to the size of that file."

Thus, rsync's notion of a block (upon which both sender and receiver must agree) is data dependent.

I suspect that the source array and the destination array are running vastly different RAID levels (maybe that double parity stuff on the VA) and you are seeing differences in array utilization not the logical utilization as seen by the UNIX hosts themselves.

[A. Clay Stephenson · Accepted solution]

Oh, and one other gotcha is sparse files. When sparse filled are read, the "missing" bytes are filled in automatically by the read() system call with zero's and the receiver dutifully writes them out as zeroes because it has no way of knowing this is a sparse file.

[Yogeeraj_1]

hi jeff,

allow me to also contribute in saying that you can modify your rsync script to display statistics about the operations going on.

e.g.
SERVER2: /home/yogeeraj/scripts>./rsync_server1.sh
receiving file list ...
2483 files to consider
applications/0 files...
application/scripts/nmon/observations/
application/scripts/logfiles/error-nmon.crn
0 100% 0.00kB/s 148:11:24 (1, 60.7% of 2483)
application/scripts/logfiles/error-sr710rp1.crn
0 100% 0.00kB/s 148:11:24 (2, 60.8% of 2483)
application/scripts/logfiles/output-nmon.crn
194 100% 189.45kB/s 0:00:00 (3, 60.9% of 2483)
application/scripts/logfiles/output-sr710rp1.crn
0 100% 0.00kB/s 148:11:24 (4, 61.1% of 2483)
application/scripts/nmon/observations/perf.nmon
172084 100% 3.49MB/s 0:00:00 (5, 61.7% of 2483)
application/scripts/nmon/observations/perf.nmon.csv.Z
54343 100% 541.52kB/s 0:00:00 (6, 61.8% of 2483)

wrote 2526 bytes read 121817 bytes 82895.33 bytes/sec
total size is 344642622 speedup is 2771.71
SERVER2: /home/yogeeraj/scripts>



Hence, you can do your test and verify where the problem really lies.

hope this helps too!

kind regards
yogeeraj

[Jeff_Traigle]

Looks like sparse files was the culprit. Using the -S option on rsync kept the file system utilization where it was on the source system.

I'll admit I was a little skeptical that could be the cause because I'd done fbackup/frecover of the file system and the file system utilization matched on both systems. The man page for frecover seems to indiciate (for lack of any mention to the contrary) that the -s option is necessary to optimize disk usage for sparse files. It seems that was not the case, however, as I only used -rv options.