gunzip file while it's still being copied?

[John J Read]

Here's a strange one. I found out that some of my developers are doing the following. I would like to tell them they are nuts and this shouldn't work but it seems to work just fine.. except now they are complaining about data integrity. Anyone have any experience or advice here?

A 1GB gzipped file "file1.gz" is being copied to file2.gz in the background to another directory but I simplified the commands below.

cp file1.gz file2.gz &

BEFORE the copy is finished, they start unzipping the target file.

gunzip file2.gz


I checked and md5sum's are the same when the operation is completed, at least for my tests. I would have thought that gunzip would complain that file2.gz is "busy" but it just starts unzipping and finishes up after the copy is complete.

This just sounds like a bad idea but I would like some evidence that this is either a supported or non-supported operation. Any thoughts?

[Leif Halvarsson_2 · Accepted solution]

Hi,
As both gzip and copy writes/reads the file sequential it may succeeds somtimes (if copy finishes before gunzip).

But, I don't belive it is recommended.

[Pete Randall]

John,

The simple fact that they're complaining about data integrity certainly makes this practice sound dubious.

How about some testing with files that are unzipped after the copy completes to see if you have any integrity problems then?


Pete

[MarkSyder]

This once happened to me.

Someone in a different office put a zipped file on one of my servers and emailed me with instructions as to what I should do with the file. It didn't occur to him that I would receive and read the email before the file transfer had completed - the result was a horrible mess, and it is not to be recommended!

Mark Syder (like the drink but spelt different)

[Fred Ruffet]

Assuming cp will write faster than gzip compresses, it will work. Now imagine, that for some reason (disks heavy use, OS scheduling...) gzip is faster than cp, then gzip will reach the point cp is actually writing and it will result in two possibilities :
. gzip consider it encountered end of file and will produce an uncomplete archive.
. gzip gets a bad data chunk to compress and produces a corrupted file.

Fact is that if they complained about corruption, one of these case may be occuring.

Regards,

Fred

[John J Read]

Thanks for all of your help! I appreciate it!!