Operating System - HP-UX
1745925 Members
4079 Online
108723 Solutions
New Discussion юеВ

How to use pipes to communicate with ftp?

 
Yury_7
Occasional Contributor

How to use pipes to communicate with ftp?

Hello

I want to optimise my ftp sesions.
Now I have one production server and one ftp server.
Clients put files to ftp. When all files uploaded then lock file has been created.

Production server by timeout creates session to ftp server and checks is
lock file exists?
If lock exists than I get list of all files which client send to me and download
them one by one to production server. After each file I check ftp log and if there is
no errors, than I delete file from ftp.
You can see this process in attached file.

But,
I don't want to open each time separate session.
I think that it's possible to open one ftp session as background job
and send commands to it using pipes. Results also get from pipes.
So I only once open session. Each time I send commands to it I will read results
from pipes.

I write next script:

#! /bin/ksh

alias errmsg='echo "($(date +%d.%m.%Y%3H:%M:%S)) ERROR: "'
alias infmsg='echo "($(date +%d.%m.%Y%3H:%M:%S)) INFO: "'

download()
{
#create ftp session in background
# read commands from 'input' pipe and write commands to 'reply' pipe
(
tail -f -c+1 input | ftp -vn | tail -f -c+1 > reply
) &
ftppid=$!

# download data files
{
echo "open ${ftpserver}"
echo "user ${ftplogin} ${ftppasswd}"
} > input

if [[ $(cat reply | grep "230 User ${ftplogin} logged in.") = "" ]]
then
errmsg "Can't login to ftp server."
echo "by" > input
return
fi

echo "ls ${client_lock}" > input
if [[ $(cat reply | grep ${client_lock}) = "" ]]
then
infmsg "Lock not found"
echo "by" > input
return
fi

echo "ls ${filemask}" > input
cat reply | sed -n "s/.* \(${filemask}\).*/\1/g;/${filemask}/p" | {
#find files by mask from ftp-log
while read file
do
echo "get $file" > input
if [[ $(cat reply | grep "226 Transfer complete.") != "" ]]
then
infmsg "File $file downloaded. Deleting it."
echo "del $file" > input
else
errmsg "Can't download $file"
fi
done
}
echo "by" > input

}

mkfifo input
mkfifo reply
download


This version of 'download' function doesn't working.
Maybe, I can't using pipes in this way?
Also I found that if pipe contains no data, than command 'cat reply' will
waiting until data appeared.


Can you help me to write correct script which can communicate
with ftp server via only one ftp session?


Thanks.
3 REPLIES 3
Fred Ruffet
Honored Contributor

Re: How to use pipes to communicate with ftp?

Some points :
. You'd better be using "ftp reply 2>&1 &" instead of using pipes and tails.
. Add a "sleep ..." command after ftp to wait for connection establishment.
. Note that you may encounter timeouts and that your ftp could close connection.

Could you post output from your script ?

Regards,

Fred
--

"Reality is just a point of view." (P. K. D.)
Dave La Mar
Honored Contributor

Re: How to use pipes to communicate with ftp?

Yury -
We perform similar, but do all the gets then check the log for errors on any given file.

snippet -

ftp -n -v << endl >> /FTP_LOG (Record process)
open servername
user username userpassword
prompt
lcd TEMP1_DIR
mget RC* (perform a specific get) (or)
mget * (just get all)
quit
endl

Followin the process, we grep the FTP_LOG for errors. At that point you could delete or whatever based on your needs. No change to your logic for checking the lock file.

Best regards,

dl
"I'm not dumb. I just have a command of thoroughly useless information."
Yury_7
Occasional Contributor

Re: How to use pipes to communicate with ftp?

Thank you for your replyes.

As you can see in attached file. now I already communicate with ftp server
using log files.
But I must first off all check lock file and then download all files by mask.
I want do it in one ftp session.
Is anybody know how should I do it?