Running a copy command stored into a variable with/without gzip

[Gilbert Standen_1]

I'm having problem with the following. Here is the code I run.

if [ "$GZIP" = 'N' ]
then
grep $sfname ${REFDIR}/${TSID}/file_map | sed 's/^/cp -p /' | read CMD
fi

if [ "$GZIP" = 'Y' ]
then
grep $sfname ${REFDIR}/${TSID}/file_map | sed 's/^/gzip < /' | sed 's/dbf/dbf > /' | sed 's/$/\.gz/' | read CMD
fi

if [ "$RUNN" = 'Y' ]
then
echo $CMD
$CMD
else
echo $CMD
fi
echo ' '
fi

Here is the output if GZIP=Y:

gzip < /global/u123/app/oracle/oradata/PROD/banlob.dbf > /u09/app/oracle/oradata/EVAL/PROD_banlob.dbf.gz
gzip: Y: No such file or directory
gzip: <: No such file or directory

You see how gzip gives errors when it runs "$CMD" if it is the gzip version. Looking at the above code, I don't get these errors when I set it to GZIP=N. In that case, $CMD is set to the following line:

cp -p /global/u123/app/oracle/oradata/PROD/banlob.dbf /u09/app/oracle/oradata/EVAL/PROD_banlob.dbf

which runs fine within the script.

So why will one command successfully store to $CMD and run, while the gzip version, stores to CMD ok (I can echo it and cut and paste it to the command line and it runs fine) but when the script tries to run it as $CMD it gives the errors show above (and I don't know where the "Y" comes from in the gzip error -- there aren't even any "Y's" in the code being run!!).

[Bill Hassell · Accepted solution]

When assigning anything but a simple command to a variable and then executing, use eval as in:

eval "$CMD"

enclosing $CMD in quotes will take care of any special characters like spaces and redirection characters like < or >.

[Gilbert Standen_1]

Hi Bill,
I tried your suggestion, but I still get the error. Here is the revised script and execution result. I still get this "gzip: Y: No such file or directory" message. And, as shown, I can still cut and paste the echoed command and it runs fine!
Gil

fsscttest:oracle:CONV$ vi test_copy_files.ksh
"test_copy_files.ksh" 36 lines, 653 characters
#!/bin/ksh

export SSID=$1
export TSID=$2
export RUNN=$3
export GZIP=$4

export REFDIR=/home/oracle/refresh

export sfname='refresh.current.dbf'
echo $sfname

if [ "$GZIP" = 'N' ]
then
grep $sfname ${REFDIR}/${TSID}/file_map | sed 's/^/cp -p /' | read CMD
fi

if [ "$GZIP" = 'Y' ]
then
grep $sfname ${REFDIR}/${TSID}/file_map | sed 's/^/gzip < /' | sed 's/dbf/dbf > /' | sed 's/$/\.gz/' | read CMD
fi

if [ "$RUNN" = 'Y' ]
then
echo $CMD
sleep 5
eval "$CMD"
else
echo $CMD
fi
~
"test_copy_files.ksh" 30 lines, 476 characters
fsscttest:oracle:CONV$ test_copy_files.ksh PROD EVAL Y Y
refresh.current.dbf
gzip < /home/oracle/refresh/EVAL/refresh.current.dbf > /home/oracle/refresh/refresh.current.dbf.gz
gzip: Y: No such file or directory
fsscttest:oracle:CONV$ ls -l /home/oracle/refresh/EVAL/refresh.current.dbf
-rw-r----- 1 oracle dba 291031040 Jan 7 12:37 /home/oracle/refresh/EVAL/refresh.current.dbf
fsscttest:oracle:CONV$ gzip < /home/oracle/refresh/EVAL/refresh.current.dbf > /home/oracle/refresh/refresh.current.dbf.gz
fsscttest:oracle:CONV$

(command runs ok when cut and pasted to command line)

[OldSchool]

A couple of things that *might* be issues:

1) gzip *uses* $GZIP to determine gzip opts, at least in later versions, so you may be inducing side-effects using it in your script. I'd try renaming to something like GZ_switch?

2) I've never tried, nor do I see in the man pages that "gzip < file.name > file.name.gz" is supported. I'd remove the redirected std input "<", as it expects a filename anyway.

[OldSchool]

oh yeah, btw: (1) above explains the "Y"

[Gilbert Standen_1]

OldSchool,

That did the trick! It works now in the script as a result of your suggestion #1. Appreciated!

As to suggestion #2, I had previously been using a different approach, i.e.:

cat filename | gzip -1 -c > filename.gz

which also works. Would this be better? Testing shows that as expected it runs about twice as fast and produces a file that is about 21% larger than the "vanilla" gzip.

[Heironimus]

"gzip < file > file.gz" works fine. "cat file | gzip -c > file.gz" should be about the same in all respects if you specify the same compression level. Use whichever one you like better.

[OldSchool]

ok,

the -1 is the compression level 1-9, where 1 is least/fastest and 9 is slowest/most compression. w/o the option the default is approx 6, middle of the road compression with middle of the road speed. that's why the file is somewhat bigger than the "vanilla" gzip.

what I meant w/ #2 was I've never seen
"gzip < a > a.gz" before

the "cat" approach you noted keeps the original file and produces a .gz file. the usual command "gzip a" will *replace* a w/ a.gz

so....it's gonna depend on what you want when your done zipping.....

[Gilbert Standen_1]

All excellent replied and so much of a help. It is very much appreciated.

This script is a component of a larger set of scripts which is used to refresh a test oracle database from a production system by doing a fully-automated "clone" of the production oracle system. In this case, we want to produce a gzipped copy of the source files, and not gzip the source file itself.

That's the explanation for these approaches to the coding with the redirects.

Gil

[Gilbert Standen_1]

ok, so just as a followup I scripted it as GZSW where GZSW is of the command line form [Y|N]:[1-9] and then within the script
echo $GZSW | cut -f1 -d':' | read GZS1
echo $GZSW | cut -f2 -d':' | read GZS2
so now a user of the script can choose (faster/larger) or (slower/smaller) (1-9) as they wish as well as choosing whether to GZIP or not. I used a compound variable because I've already got 9 total variables in the calling script so the caller will pass GZSW as GZS1:GZS2