- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - Linux
- >
- Re: Automate FTP
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 07:56 AM
тАО11-09-2005 07:56 AM
I need help with figuring out how to script the following. There is an FTP server that contains a file that I need to download. A new one is put there almost every day...I don't know when, it is random. If there is not a new file put there, then the old one remains until the new arrives (different name each time). I know how to automatically ftp, but want I want to do is:
1. only download the file if it is new (in other words, I don't have it downloaded already)
2. email myself as a notification that a new file has been obtained.
Thanks!
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:10 AM
тАО11-09-2005 08:10 AM
Solutionftp -n -v hostname > $LOG << ENDFTP
user username passwd
prompt off
binary
cd wherever
lcd wherever
mget *
quit
If you are wanting to do some comparisons for newer file, will need a script that contains logic to compare the date/time stamp of the file to be downloaded.
- Tags:
- ftp
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:16 AM
тАО11-09-2005 08:16 AM
Re: Automate FTP
S.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:27 AM
тАО11-09-2005 08:27 AM
Re: Automate FTP
The ftp client is not very good at knowing whether the file is newer or not.
You could however insert some code into the excellent response you have already received.
ls -la | awk '{print $5 $6 $7 $8} > /tmp/localfile
# now you can compare the time and date stamp and size to what you already have on your local system.
I'm not sure it will be 100% accurate.
A better methodology perhaps would be for the ftp server to run a command like this
find /ftpdir -mtime -exec 'ls -1 > /ftp/downloadlist'
or maybe
find /ftpdir -ctime -exec 'ls -1 > /ftp/downloadlist'
Then you download that list every day, feed it into your ftp scripot and get the new files.
A little help at ftp side is needed.
Or you can download and compare later, but it sounds like you are trying to conserver bandwitch.
Hope I've given you an idea on how to approach the problem.
As for the email part:
echo -e "To: $EMAIL_TO" > $fmesg
echo -e "From: $EMAIL_FROM" >> $fmesg
echo -e "Cc: prottertemp@investmenttool.com" >> $fmesg
echo -e "Subject: $EMAIL_SUBJECT" >> $fmesg
echo -e "" >> $fmesg
/bin/cat $fbody >> $fmesg
cat $fmesg | /usr/sbin/sendmail -t
> $fbody
rm -f $fbody
> $fmesg
rm -f $fmesg
> $ftext
rm -f $ftext
just set the variables and decide what the text is. fmesg is a temporary holding file for the message. Should be pretty obvious.
This is production code from a Linux box, but it is also running on HP boxes I used to admin in the US. Linux boxes are mine, I'm allowed to peak.
Good Luck,
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:32 AM
тАО11-09-2005 08:32 AM
Re: Automate FTP
There are probably more eloquent ways to do this, but ...
From and ftp session:
1. Set the prompt off by issuing the ftp prompt command.
2. ls yourfile* dummy
The above will list yourfile with date/timestamp in the file dummy on your local machine.
3. Quit the ftp.
4. Compare your old file to that in the dummy file created in #2.
5. Based on the comparison, go on with your business. i.e. set off the email and second ftp job to pick up the file.
Best of luck.
Regards,
dl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:34 AM
тАО11-09-2005 08:34 AM
Re: Automate FTP
See the man pages for 'ftp(1)'. You have a 'newer' variation of 'get':
# newer filename
...gets the file only if the modification time of the remote file is more recent that the file on the current system.
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:36 AM
тАО11-09-2005 08:36 AM
Re: Automate FTP
Get the file only if the modification time of the remote file is
more recent that the file on the current system. If the file
does not exist on the current system, the remote file is
considered newer. Otherwise, this command is identical to get.
As to sending mail, can have a job that periodically checks the date/time of the file. If newer, then send mail.
DATE=`ll
if [ $DATE != date +%e` ]
then
mailx ....
fi
Hope that helps.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 08:37 AM
тАО11-09-2005 08:37 AM
Re: Automate FTP
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 09:06 AM
тАО11-09-2005 09:06 AM
Re: Automate FTP
ftp.sh
--------------------------------
#!/usr/bin/sh
PATH=${PATH}:/usr/local/bin
export PATH
typeset PROG=${0##*/}
typeset TDIR=${TMPDIR:-/var/tmp}
typeset PID=${?}
typeset T1=${TDIR}/X${PID}_1.txt
trap 'eval rm -f ${T1}' 0 1 2 15
typeset SRCDIR="/home/cstephen"
typeset REMHOST="bugs"
typeset DESTDIR="/tmp/cstephen"
typeset U="cstephen"
typeset P="-p secret"
typeset ADDRESS="mmouse@disney.com"
typeset -i STAT=0
# make sure destination directory exists"
if [[ ! -d "${DESTDIR}" ]]
then
echo "Destination ${DESTDIR} does not exist." >&2
exit 1
fi
cd "${DESTDIR}"
STAT=${?}
if [[ ${STAT} -ne 0 ]]
then
echo "Cannot cd to ${DESTDIR}; status ${STAT}." >&2
exit 2
fi
#Get a list of files in directory ${DIR} on remotehost {REMHOST}
#Login as user ${U} using password ${P}.
#If a .netrc file is located on the client then no passwd is needed.
ftpget.pl -h ${REMHOST} -l ${U} ${P} -L '' -d "${SRCDIR}" > ${T1}
STAT=${?}
if [[ ${STAT} -eq 0 ]] # dirlist okay
then
if [[ -s ${T1} ]]
then
typeset FNAME=""
cat ${T1} | while read FNAME
do
if [[ ! -f ${FNAME} ]]
then
ftpget.pl -h ${REMHOST} -l ${U} ${P} -d "${SRCDIR}" "${FNAME}"
STAT=${?}
if [[ ${STAT} -eq 0 ]]
then
echo "New File: ${FNAME}" | mailx -s "New File" ${ADDRESS}
else
echo "Failed to get ${FNAME}; status ${STAT}" >&2
exit ${STAT}
fi
fi
done
fi
else
echo "${PROG} dirlist failed; status ${STAT}" >&2
fi
exit ${STAT}
-------------------------------
That should do it if I haven't made any typo's. You can add a patterm to the ftpget.pl -L argument to match only certain patterns. e.g -L 'Data*.txt'. Using Perl is a much better choice because you get error checking for free -- something that is very tedious in the shell. Note that you can also setup a .netrc file on your end and you don't need to pass the password. Man .netrc for details. If you want to do this more secure there is also a Net::SFTP module. You can then make a cron wrapper script for this and you are done.
Here's the perl piece, ftpget.pl; the above script assumes that you install it in /usr/local/bin. I suggest that you invoke it as ftpget.pl -u for full usage because you may want to set binary or ASCII mode for the gets.
- Tags:
- Perl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-09-2005 09:28 AM
тАО11-09-2005 09:28 AM
Re: Automate FTP
James nailed it with newer.
James -
This is a great tip! Wasn't aware of it, but will definitely use it.
Thanks.
And Sally, please no points on this reply, just wanted to acknowledge anothers great contribution.
Regards,
dl