Awk script problem

[Stephen Hughes]

Hi

I run an awk command in a script named script + error.txt attached. When it reaches 54 lines in the output file, it gives me the error seen in script + error.txt which is attached. Is there a way to resolve this or do you know why it happens so that I can look to work round it?

Regards
Stephen

[Alex Glennie]

try looking at what your maxfiles kernel parm is set to 60 ? maybe worth increasing to 200, otherwise it may be a hard coded value inside of awk(1).

[John Palmer]

The problem is that awk is opening a file in each iteration. This file doesn't get closed and there is a system limit of (in you case) 64 open files per process.

You need to ensure that awk is closing the file.

[Guy Draper]

I am guessing you are hitting the system limit of number of open files per process. You could try increasing this to say 200, but if your list of files is likely to get higher than this you'll be constantly changing it. I would suggest you get your script to close the file once you have finished with it. Sorry I don't have a system at the moment to try it on.

[James R. Ferguson]

Stephen:

It appears that the objective is to copy files based on some criteria. May I suggest that instead of using the system call feature of awk, that you simply build copy commands as output of the awk script. That is, the total script might start by building a script "preamble"; append the copy command with the awk process to the running script file, and then execute the dynamically constructed script.

...JRF...

[John Palmer]

I think that what you are tring to do could be done with a fairly simple shell script. Something like:-

cat $WORK/acc_reformat |
while read ARG1 ARG2 ARG3 ARG4
do
FILE=$(ls -l /u/odexcc/FTP/$ARG3 |grep "$ARG4" | grep "$ARG2" |cut -c59-73)
echo /u/odexcc/FTP/$ARG3/$FILE >> $WORK/files.out
cp /u/odexcc/FTP/$ARG3/$FILE /u/stephen/shell/recon/ftp_files
done

This could be made much more efficient if you supplied more details of what you are trying to achieve.

[Andreas Voss · Accepted solution]

Hi,

i've put you the script with a close command in it. Give it a try.

Regards