Compress a file before ftp

[MacMcDonald]

Hi,

I have a script that ftp's to over 100 deifferent servers in turn, gets a specific file, renames it and drops it onto a local backup server. The files vary in size from 4mb to 150mb. I am within a secure intranet to security with ftp is not an issue. I want to auto compress the file before I ftp it over the network, either as a seperate script or as a command contained within my original.

Can anyone help?

the current script is as follows:


hostFile=/data01/global_backup/scripts/GLOBAL_hostFile.txt
while read ip sd
do
echo "connecting to $sd on $ip"
ftp -dv $ip <get /global/prd/cycle/export/full_export.dmp /data01/global_backup/files/monday/$sd.full_export.dmp
bye
EOF
if [ $? -ne 0 ]; then
echo "Had a problem connecting to $sd on $ip"
else
echo "Connected to $sd on $ip"
fi
done < ${hostFile}

[Coolmar]

Hi Mac,

I found the following link and I think it might help.

S.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=840556&admit=-682735245+1164119155839+28353475

[MacMcDonald]

thanks for the reply, my problem is that I have more than 80 servers to connect to in turn. I have edited the .netrc file to contain the user password details to enable autologin. This only seems to work with ftp, is there anyway I can include a telnet session in the script to compress the file in the same script prior to the ftp transfer?

[Christian Tremblay]

echo "connecting to $sd on $ip"
remsh gzip

add .gz extension to your ftp get in your script

[MacMcDonald]

for some reason when I add the line:

remsh gzip {filename}

it does not connect using that command, it seems to pass it by and run the get command direct, falling over because the .gz file is not there yet.

[OldSchool]

appears to be a small typo in Christian's reply. try:

remsh $sd gzip

needs to know server to run command on

[OldSchool]

or it might be $ip (if $sd isn't the dns name)

[Christian Tremblay]

not a typo, i avoid putting actual working commands because a lot of people rely on this forum to do their job for them.

I put just enough for a knowledgable user to figure it out.

[MacMcDonald]

again thanks for the reply guys - i already tried both the $sd and $ip before asking, it seems that the redhat box I am running the script from doesn't understand the remsh command.....

[Christian Tremblay]

You can substitute ssh to remsh ( which is a lot better and secure ) but you would have to deploy you ssh keys to all your servers to enable passwordless remote execution.
Which is a good thing to do if you have not already done so.

man ssh

Chris

[Christian Tremblay]

PS: I'm not sure, but I think in redhat remsh is called rsh

You may try that....

[MacMcDonald]

got the rsh part - but for some reason it falls over on the login, doesnt want to use the details included in the .netrc.

[Christian Tremblay]

I don't have access to a Redhat box to check the rsh use of .netrc, try man rsh on your Redhat box for more info.

You really should try to implement ssh in your environment. .netrc files are considered a security risk and will not survive a SOX audit. In fact their use is prohibited at most security conscious sites.

[Peter Nikitka]

Hi,

.netrc is used for ftp only - rsh uses a .rhosts or /etc/hosts.equiv file on the remote host to check for logins without the need of an interactive login.

Even for ssh connections (which I recommend) you will have to add data to all involved clients you want to connect to.
If you setup sftp, you could reuse your script(s) nearly unchanged.

mfG Peter

mfG Peter

[MacMcDonald]

it seems that rexec uses the .netrc file as well, could i use that to initiate the gzip command?

[Volker Borowski]

Hi,

you could set up your ftp server to allow the execution of some commands, i.E. compression via script or likewise.

Get manpage of ftp and dig into the
"site exec"
section

i.E. here: http://docs.hp.com/en/B2355-60105/ftpd.1M.html

Watch it, there are scurity issues with this setup !

Volker

[Arturo Galbiati]

Hi Mac,
you can use the rexec command:
rexec host [-l username] [-n] command

HTH,
Art