topic Re: Compress a file before ftp in Operating System - HP-UX

Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 09:20:10 GMT

Hi,

I have a script that ftp's to over 100 deifferent servers in turn, gets a specific file, renames it and drops it onto a local backup server. The files vary in size from 4mb to 150mb. I am within a secure intranet to security with ftp is not an issue. I want to auto compress the file before I ftp it over the network, either as a seperate script or as a command contained within my original.

Can anyone help?

the current script is as follows:

hostFile=/data01/global_backup/scripts/GLOBAL_hostFile.txt
while read ip sd
do
echo "connecting to $sd on $ip"
ftp -dv $ip <get /global/prd/cycle/export/full_export.dmp /data01/global_backup/files/monday/$sd.full_export.dmp
bye
EOF
if [ $? -ne 0 ]; then
echo "Had a problem connecting to $sd on $ip"
else
echo "Connected to $sd on $ip"
fi
done < ${hostFile}

Re: Compress a file before ftp

Coolmar — Tue, 21 Nov 2006 09:27:23 GMT

Hi Mac,

I found the following link and I think it might help.

S.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=840556&admit=-682735245+1164119155839+28353475

Re: Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 09:37:52 GMT

thanks for the reply, my problem is that I have more than 80 servers to connect to in turn. I have edited the .netrc file to contain the user password details to enable autologin. This only seems to work with ftp, is there anyway I can include a telnet session in the script to compress the file in the same script prior to the ftp transfer?

Re: Compress a file before ftp

Christian Tremblay — Tue, 21 Nov 2006 10:41:33 GMT

echo "connecting to $sd on $ip"
remsh gzip

add .gz extension to your ftp get in your script

Re: Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 10:52:01 GMT

for some reason when I add the line:

remsh gzip {filename}

it does not connect using that command, it seems to pass it by and run the get command direct, falling over because the .gz file is not there yet.

Re: Compress a file before ftp

OldSchool — Tue, 21 Nov 2006 11:00:11 GMT

appears to be a small typo in Christian's reply. try:

remsh $sd gzip

needs to know server to run command on

Re: Compress a file before ftp

OldSchool — Tue, 21 Nov 2006 11:03:28 GMT

or it might be $ip (if $sd isn't the dns name)

Re: Compress a file before ftp

Christian Tremblay — Tue, 21 Nov 2006 11:03:37 GMT

not a typo, i avoid putting actual working commands because a lot of people rely on this forum to do their job for them.

I put just enough for a knowledgable user to figure it out.

Re: Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 11:06:25 GMT

again thanks for the reply guys - i already tried both the $sd and $ip before asking, it seems that the redhat box I am running the script from doesn't understand the remsh command.....

Re: Compress a file before ftp

Christian Tremblay — Tue, 21 Nov 2006 11:12:22 GMT

You can substitute ssh to remsh ( which is a lot better and secure ) but you would have to deploy you ssh keys to all your servers to enable passwordless remote execution.
Which is a good thing to do if you have not already done so.

man ssh

Chris

Re: Compress a file before ftp

Christian Tremblay — Tue, 21 Nov 2006 11:17:28 GMT

PS: I'm not sure, but I think in redhat remsh is called rsh

You may try that....

Re: Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 11:45:53 GMT

got the rsh part - but for some reason it falls over on the login, doesnt want to use the details included in the .netrc.

Re: Compress a file before ftp

Christian Tremblay — Tue, 21 Nov 2006 11:52:54 GMT

I don't have access to a Redhat box to check the rsh use of .netrc, try man rsh on your Redhat box for more info.

You really should try to implement ssh in your environment. .netrc files are considered a security risk and will not survive a SOX audit. In fact their use is prohibited at most security conscious sites.

Re: Compress a file before ftp

Peter Nikitka — Tue, 21 Nov 2006 12:14:25 GMT

Hi,

.netrc is used for ftp only - rsh uses a .rhosts or /etc/hosts.equiv file on the remote host to check for logins without the need of an interactive login.

Even for ssh connections (which I recommend) you will have to add data to all involved clients you want to connect to.
If you setup sftp, you could reuse your script(s) nearly unchanged.

mfG Peter

mfG Peter

Re: Compress a file before ftp

MacMcDonald — Tue, 21 Nov 2006 12:17:08 GMT

it seems that rexec uses the .netrc file as well, could i use that to initiate the gzip command?

Re: Compress a file before ftp

Volker Borowski — Tue, 21 Nov 2006 14:23:47 GMT

Hi,

you could set up your ftp server to allow the execution of some commands, i.E. compression via script or likewise.

Get manpage of ftp and dig into the
"site exec"
section

i.E. here: http://docs.hp.com/en/B2355-60105/ftpd.1M.html

Watch it, there are scurity issues with this setup !

Volker

Re: Compress a file before ftp

Arturo Galbiati — Wed, 22 Nov 2006 03:37:09 GMT

Hi Mac,
you can use the rexec command:
rexec host [-l username] [-n] command

HTH,
Art