topic Re: Partial Files ? in Operating System - HP-UX

Partial Files ?

mike boswell_3 — Thu, 17 Jun 2004 08:55:32 GMT

I use scripts to process a pile of files in a directory. I have been doing this on off hours when no one is around. I need to increase the frequency that it is run and will need to run it during business hours.

The senario goes like this. A Pc user w/ a samba connection to the hp-ux machine drops a file into the directory(INBOX). The script on the hp-ux machine is run via cron and moves the accumulated files out the the INBOX.

Could I end up moving a file before it is fully written to the INBOX and end up w/ a partial file? There must be a protection in place against this. Just need to confirm.

Thanks,
Mike B

Re: Partial Files ?

Mark Grant — Thu, 17 Jun 2004 09:04:35 GMT

Yes you could but it probably doesn't matter.

When you "mv" a file to another location on the same filesystem, the file doesn't change at all and, in fact, doesn't actually move anywhere. If the file is being written to when you "mv" it, it will still be being written to after the "mv".

There are two scenarios where you might hit a problem. Firstly, if your target directory for the INBOX files is on another fileystem then "mv" is just a "cp"+"rm" and then you will lose data from unfinished files.

Also, if, when files are written, the application closes the file and later re-opens it to append data, then, again you run the risk of copying half a file.

I would do some tests but my guess is that you will be OK.

However, if unsure then just play safe.

Re: Partial Files ?

RAC_1 — Thu, 17 Jun 2004 09:09:45 GMT

On each file you can do fuser/lsof to know, if it is being used by a process or not.

You can also compare the size of the file on Windows to that on the hp-ux

Anil

Re: Partial Files ?

mike boswell_3 — Thu, 17 Jun 2004 10:11:36 GMT

Thanks,
That makes me more comfortable.
Mike

Re: Partial Files ?

harry d brown jr — Thu, 17 Jun 2004 10:28:01 GMT

Mike,

>>>>The senario goes like this. A Pc user w/ a samba connection to the hp-ux machine drops a file into the directory(INBOX).

How are the files dropped off into "INBOX"? Does the user manually do it or does some application do it?

>>>>The script on the hp-ux machine is run via cron and moves the accumulated files out the the INBOX.

If an application does it, then have the application drop the file off, then have it drop off a file name with a suffix of "RDY" - change your cron script to look for "RDY" suffixes only.

If it is a user manual file movement to "INBOX", then change your script to use "lsof" on the file to make sure no one has it open. lsof: http://hpux.cs.utah.edu/hppd/hpux/Sysadmin/lsof-4.70/

live free or die
harry