topic Re: Awk script problem in Operating System - HP-UX

Awk script problem

Stephen Hughes — Tue, 03 Oct 2000 13:08:55 GMT

Hi

I run an awk command in a script named script + error.txt attached. When it reaches 54 lines in the output file, it gives me the error seen in script + error.txt which is attached. Is there a way to resolve this or do you know why it happens so that I can look to work round it?

Regards
Stephen

Re: Awk script problem

Alex Glennie — Tue, 03 Oct 2000 13:13:53 GMT

try looking at what your maxfiles kernel parm is set to 60 ? maybe worth increasing to 200, otherwise it may be a hard coded value inside of awk(1).

Re: Awk script problem

John Palmer — Tue, 03 Oct 2000 13:16:52 GMT

The problem is that awk is opening a file in each iteration. This file doesn't get closed and there is a system limit of (in you case) 64 open files per process.

You need to ensure that awk is closing the file.

Re: Awk script problem

Guy Draper — Tue, 03 Oct 2000 13:20:54 GMT

I am guessing you are hitting the system limit of number of open files per process. You could try increasing this to say 200, but if your list of files is likely to get higher than this you'll be constantly changing it. I would suggest you get your script to close the file once you have finished with it. Sorry I don't have a system at the moment to try it on.

Re: Awk script problem

James R. Ferguson — Tue, 03 Oct 2000 13:39:16 GMT

Stephen:

It appears that the objective is to copy files based on some criteria. May I suggest that instead of using the system call feature of awk, that you simply build copy commands as output of the awk script. That is, the total script might start by building a script "preamble"; append the copy command with the awk process to the running script file, and then execute the dynamically constructed script.

...JRF...

Re: Awk script problem

John Palmer — Tue, 03 Oct 2000 14:02:08 GMT

I think that what you are tring to do could be done with a fairly simple shell script. Something like:-

cat $WORK/acc_reformat |
while read ARG1 ARG2 ARG3 ARG4
do
FILE=$(ls -l /u/odexcc/FTP/$ARG3 |grep "$ARG4" | grep "$ARG2" |cut -c59-73)
echo /u/odexcc/FTP/$ARG3/$FILE >> $WORK/files.out
cp /u/odexcc/FTP/$ARG3/$FILE /u/stephen/shell/recon/ftp_files
done

This could be made much more efficient if you supplied more details of what you are trying to achieve.

Re: Awk script problem

Andreas Voss — Wed, 04 Oct 2000 06:49:18 GMT

Hi,

i've put you the script with a close command in it. Give it a try.

Regards