FTP interface techniques

[Volker Borowski]

Hi friends,

did you ever deal with ftp interfaces ?
I mean like server A sends a file to server B and an application on server B processes this file.
Fairly simple :-)

Well, some questions:
- what if the file to be transmitted has not been processed on server B, because the application has been down for a while, and gets overwritten ?
- what if server B reads the file while it is still in transmission ?
- what if server A transmitts a new file while the application on server B is still reading it.

In the past, I did a lot of scripting with checks and renaming to catch those possible pitfalls. In this case there had always been the problem, that you might need to setup some versioning of files, mostly on both sides (sending AND receiving), in case a server or the application is down.

Now I saw a cool setup at a customer site. They used a plain lpd printer for this. They get all those nice queueing elements for free.

The file to be transmitted is queued (printed) on server A, which is a remote printer on server B.
The corresponding printer is active and enabled for printing and prints to a LPD-printer on server B that is enabled for queueing but disabled for printing.
The application checks the printer with lpq for jobs to be processed, processes them and cancels them when ready.

Beside the fact, that there was no authentification on it (means everyone could setup a lpd printer and send b...s... to this one), the idea to set something up like this appeared brilliant to me, in terms of "transmission" security (not "access" security).

Is there a way to do authentication on a lpd-connect ?
Did anyone manage to setup such a queueing with ftp in a secure way, which means no sending or received files get lost ?
How do you proceed with these problems in your ftp-interfaces.

Comments welcome
Volker

[Rodney Hills]

Volker,

Interesting concept. I wonder if you could do the same thing using email (smtp).

my 2 cents

-- Rod Hills

[Dave La Mar]

Our shop simply puts a file_name.running from server A to the server B directory then deletes it following the put of the desired file.
Server B does not touch the desired file until the file_name.running is no longer there.
Though we had contemplated putting the file under some other name and renaming following ftp transfer completion, the first method mentioned has worked best for us.

Just another way.

dl

[Rory R Hammond · Accepted solution]

Volker,

This is a very interesting thing, it does have possibilities.

I have an old unix favorite that people seems to forget about. UUCP supports tcp/ip

setup took me about 2 hours for 14 sites. 13 of the sites collect and gather Purchase Order data and then uucp/uux the data to the master site. Security is handled via uucp Systems and Permission files. UUCP support queing and after trying awhile will notify you about que failures. The last one I setup was 2 years ago, and I have not received any problems since.

Ror

[Volker Borowski]

Thanks for feedback so far,

Rodney: I think e-mail would work quite the same way, although I am not so sure if the queuing is really id based as it is for printjobs. I think the sequence might get mixed up, if a server is down for a while.

Dave: that is nearly the same I did in the past....
put sendfile sendfile.in_transmission
rename sendfile.in_transmission sendfile.transmitted
... and so on. It worked for me in the past, but every time I set this up at a new site, there comes up a special case which I did not cover yet. So the next time I needed *.in_process and *.ready_to_archive and still no end in sight. The lpd-way seems to cover it all.

Rory: I barely know more about uucp than it is "old stuff" :-) Then I know about uuencode/uudecode from this forums as a solution for various mail-attachment problems.
I think I will read some manpages on this.

More ideas wanted
Volker

[Rory R Hammond]

Sorry about following up so late.

If you like.
I can send you examples of
System, Permission, Devices files.
And commands

Might email directly for quicker response
Rory