UNIX shell script - Reading directory contents and tar-ing/gzip all files - HELP NEEDED

[Danny Fang]

Hi,

I have a script which is to perform the following task:
1) check for files in a particular directory.

2) The directory should contain 4 files in the form of:
__BSC...

whereby:
OSSInstance - a combination of alphanumeric characters
OSSName -a combination of alphanumeric characters
Nokia BSC ID - a combination of alphanumeric characters
counter no - Digits 1,2, 72 and 79 only
YYYYMMDDHHMM - basically a numeric value denoting timestamp of the file i.e. 200603130500
gid - a numeric value for a particular process

An example of the file is shown below as:
1003_oxnn2_BSC48379.72.200603130500.768751

3)There is a timeout period in the script which checks if these 4 files are available within the particular directory.These 4 files are being transferred remotely from some machine.

4)If ALL 4 files are already in the directory within the specified timeout period, then it's copied to another directory where it's tar-ed and gzip-ed.

5)If one(1) of the 4 files are not the directory within the specified timeout period, these files are copied elsewhere and their names written into a log file.

6) The timeout period in waiting for ALL 4 files to arrive is 5 mins.

However I've encountered some problems within the script, which is captured below:
prod-cingtuna\ :/mkl/users/lows >./Nokia_RAN-tarNew.sh
1003_oxnn2_BSC48379.1.200603130500.768741: dskjfhkdjs: not found
1003_oxnn2_BSC48379.1.200603130500.768741: djkdahlk: not found
1003_oxnn2_BSC48379.1.200603130500.768741: anjhsahdklsad: not found
1003_oxnn2_BSC48379.1.200603130500.768741: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_INSTANCE
1003_oxnn2_BSC48379.1.200603130500.768741: dskjfhkdjs: not found
1003_oxnn2_BSC48379.1.200603130500.768741: djkdahlk: not found
1003_oxnn2_BSC48379.1.200603130500.768741: anjhsahdklsad: not found
1003_oxnn2_BSC48379.1.200603130500.768741: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_NAME
1003_oxnn2_BSC48379.1.200603130500.768741: dskjfhkdjs: not found
1003_oxnn2_BSC48379.1.200603130500.768741: djkdahlk: not found
1003_oxnn2_BSC48379.1.200603130500.768741: anjhsahdklsad: not found
1003_oxnn2_BSC48379.1.200603130500.768741: askjdhasas^Jdaskj^Jakjchdajk^J: not found
TMP_VAR
BSC_VAL
NOKIA_BSC_ID
TIMESTAMP
GRP_DAY 20
hrmin 17:50
GRP_HOUR 17
GRP_MIN 50
NUM_FILES 1
1003_oxnn2_BSC48379.2.200603130500.768750: dskjfhkdjs: not found
1003_oxnn2_BSC48379.2.200603130500.768750: djkdahlk: not found
1003_oxnn2_BSC48379.2.200603130500.768750: anjhsahdklsad: not found
1003_oxnn2_BSC48379.2.200603130500.768750: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_INSTANCE
1003_oxnn2_BSC48379.2.200603130500.768750: dskjfhkdjs: not found
1003_oxnn2_BSC48379.2.200603130500.768750: djkdahlk: not found
1003_oxnn2_BSC48379.2.200603130500.768750: anjhsahdklsad: not found
1003_oxnn2_BSC48379.2.200603130500.768750: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_NAME
1003_oxnn2_BSC48379.2.200603130500.768750: dskjfhkdjs: not found
1003_oxnn2_BSC48379.2.200603130500.768750: djkdahlk: not found
1003_oxnn2_BSC48379.2.200603130500.768750: anjhsahdklsad: not found
1003_oxnn2_BSC48379.2.200603130500.768750: askjdhasas^Jdaskj^Jakjchdajk^J: not found
TMP_VAR
BSC_VAL
NOKIA_BSC_ID
TIMESTAMP
GRP_DAY 20
hrmin 17:50
GRP_HOUR 17
GRP_MIN 50
NUM_FILES 2
1003_oxnn2_BSC48379.3.200603140500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.3.200603140500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.3.200603140500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.3.200603140500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_INSTANCE
1003_oxnn2_BSC48379.3.200603140500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.3.200603140500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.3.200603140500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.3.200603140500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_NAME
1003_oxnn2_BSC48379.3.200603140500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.3.200603140500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.3.200603140500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.3.200603140500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
TMP_VAR
BSC_VAL
NOKIA_BSC_ID
TIMESTAMP
GRP_DAY 20
hrmin 17:50
GRP_HOUR 17
GRP_MIN 50
NUM_FILES 3
1003_oxnn2_BSC48379.72.200603130500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.72.200603130500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.72.200603130500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.72.200603130500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_INSTANCE
1003_oxnn2_BSC48379.72.200603130500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.72.200603130500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.72.200603130500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.72.200603130500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
OSS_NAME
1003_oxnn2_BSC48379.72.200603130500.768751: dskjfhkdjs: not found
1003_oxnn2_BSC48379.72.200603130500.768751: djkdahlk: not found
1003_oxnn2_BSC48379.72.200603130500.768751: anjhsahdklsad: not found
1003_oxnn2_BSC48379.72.200603130500.768751: askjdhasas^Jdaskj^Jakjchdajk^J: not found
TMP_VAR
BSC_VAL
NOKIA_BSC_ID
TIMESTAMP
GRP_DAY 20
hrmin 17:50
GRP_HOUR 17
GRP_MIN 50
NUM_FILES 4
a /tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.1.200603130500.768741 1K
a /tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.2.200603130500.768750 1K
a /tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.3.200603140500.768751 1K
a /tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.72.200603130500.768751 1K
prod-cingtuna\ :/mkl/users/lows >

From the script output above:
1) I'm not sure why it is producing the error"1003_oxnn2_BSC48379.1.200603130500.768741: dskjfhkdjs: not found"
The file contents are:
prod-cingtuna\ :/mkl/users/lows >cat 1003_oxnn2_BSC48379.1.200603130500.768741
dskjfhkdjs
djkdahlk
anjhsahdklsad
askjdhasas'
daskj
akjchdajk

2) It seems that for every line in the file, the script outputs the "not found" error to the standard output - I'm not able to see where it's coming from witin the script.

3) Some of the values for the variables shown below within the script are not having any values inspite being assigned values from AWK:
TMP_VAR
BSC_VAL
NOKIA_BSC_ID
TIMESTAMP
The AWK statement has been tested on the command line and it's producing correct

4)Once all 4 files are in the specified directory, they all do get copied to another directory (specified as /tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out in the script)
However, tar and gzip process failed to tar these 4 files correctly(according to the name specified in the directory) into a single file and zip them.
Instead, the file ._.._.BSC.BSC_VAL..tar.gz
is produced:
prod-cingtuna\ :/mkl/users/lows >ls -la
total 112
-rw-r--r-- 1 lows metrica 322 Mar 20 19:44 ._.._.BSC.BSC_VAL..tar.gz

Could anyone help me out by pointing out where did I go wrong in the script?

Feel free to modify the script.

Thanks in advance.
Danny

[Danny Fang]

Hi,

Please note that the line
"counter no - Digits 1,2, 72 and 79 only" from this posting actually meant:

counter no - Digits 1,2, 3 and 72 only.

Apologies for the confusion caused.

Thanks
Danny

[Danny Fang]

Hi,
kindly note that the line from this forum posting:
"counter no - Digits 1,2, 72 and 79 only"

actually refers to:
counter no - Digits 1,2, 3 and 72 only.

Thanks
Danny

[Peter Godron · Accepted solution]

Danny,
point 3:
you need to wrap "`" around your assigns i.e.
OSS_NAME=`echo $file | awk 'BEGIN { FS = "_"} {print $2}'`

For a start include line
set -x
as the second line of your script.
This will give your more debug info when you run the script.

[Antonio Cardoso_1]

2) "not found" output comes from
OSS_INSTANCE=echo $file | awk 'BEGIN { FS = "_"} {print $1}'

and similar lines,
=> you should have instead :
OSS_INSTANCE=`echo $file | awk 'BEGIN { FS = "_"} {print $1}'`

(back quotes missing)

[Antonio Cardoso_1]

2) "not found" output comes from
OSS_INSTANCE=echo $file | awk 'BEGIN { FS = "_"} {print $1}'

and similar lines,
=> you should have instead :
OSS_INSTANCE=`echo $file | awk 'BEGIN { FS = "_"} {print $1}'`

(back quotes missing)

And this is also the reason for point 3) I think.

[Peter Nikitka]

Hi,

4) Even if you correct the setting of your variables (3.) in the way, Peter G. advised (and there are several places you have to change!):

You have misspelt OSS_INSTANCE an used OSS_INTANCE in your tar parameters.

mfG Peter

[Jean-Yves Picard]

Hello,

point 1:
the line
OSS_INSTANCE=echo $file | awk 'BEGIN { FS = "_"} {print $1}' ##1003
start a sub shell
in this subshell OSS_INSTANCE is given the value of echo
then the commande
$file | awk 'BEGIN { FS = "_"} {print $1}' ##1003
is run
since $file is executable, it is.
since it contents does not contains valid command you receive the error above.

are you sure
tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.2.200603130500.768750
and
tmp/osspkg2s/uma/umabass/cs_bsspm_aif/client/out/1003_oxnn2_BSC48379.3.200603140500.768751

came within 5 minutes ???
look one day apart to me.

you may whish to use ls -t to order the file, or any sort option.

since you only have four file
I would create and instanciate four variables
name FILE1, FILE2, FILE4, FILE72
with initial value of "unknow"

have a PREVIOUS_VAL and a VAL
VAL is concatenation of all token save NOKIA_BSC_ID (which by the way I am not sure you caught it right)

initiate all VAR's
initiate VAL
if [ $PREVIOUS_VAL != $VAL ]
then
## check for all file
if echo "$FILE1 $FILE2 $FILE4 $FILE72" | grep unknow
then
## there is a missing file
else
## all file here
fi
## reset all $FILE1 $FILE2 $FILE4 $FILE72
PREVIOUS_VAL=$_VAL
fi

Hope this help.

Jean-Yves Picard

[Peter Nikitka]

Hi,

4) Even if you correct the setting of your variables (section 3.) in the way, Peter G. advised (and there are several places you have to change!):

You have misspelt OSS_INSTANCE an used OSS_INTANCE in your tar parameters.

mfG Peter

[Peter Godron]

Danny,
did these suggestions help?
Please update.

[Danny Fang]

Problem resolved based on suggestions provided in this forum.

Thanks to all whom had helped out.