output redirection not working

[Kenneth_18]

I'm executing a perl script from within a csh script as follows:


====
#!/usr/bin/csh

./perlscript options options >>& /tmp/perlscript.log
====


I execute the csh script in cron like this:

xx * * * * csh -c '/directory/csh_perlscript.sh' >> /tmp/perlscript.log 2>&1

or directly from a terminal like this:

./csh_perlscript.sh >>& /tmp/perlscript.log


What I notice is that the output of the csh_perlscript is sent to the perlscript.log but the outout of the perlscript itself is not sent to the log file.

What could be wrong? Sometimes it works but it takes a very long time before the file is updated.

Thanks in advance!
Kenneth

[Biswajit Tripathy]

Kenneth wrote:
> or directly from a terminal like this:
>
> ./csh_perlscript.sh >>& /tmp/perlscript.log

Change that to:
./csh_perlscript.sh >> /tmp/perlscript.log 2>&1

Same as the way you used in cron.

- Biswajit

[Noel Miranda]

Remove the & in
./perlscript options options >>& /tmp/perlscript.log

and try it out.

[Kenneth_18]

The "&" after ">>" doesn't actually matter if its there or not as it's just the csh equivalent to the "2>&1" in posix sh.

After more observation, the redirection seems to be working but the file is not being updated in real time. It is as if it is waiting for a certain memory buffer to fill up before it dumps the contents to the logfile.

How can I tune this so that the output is immediately dumped into the log file so I could monitor the output in real time.

Thanks in advance!
Kenneth

[Noel Miranda]

Hi Kenneth,

Thanks for the tip. I was just checking out the same with a simple script here and it works pretty fine, maybe you are doing something within your perl script which take time to generate the output. Did you try running your script with a csh -x?

[Kenneth_18]

Hi Noel,

If I ran the perl script directly from within a csh enviroment, the output is immediately sent to the screen.

Now I'm wandering if there is some kernel tuning parameter needed or maybe the server is just so heavily loaded. The perl script is doing an sql query to an oracle server.

Thanks for the update.

Kenneth

[Noel Miranda]

Hi Kenneth,

Try runnning the script with
csh -x <scriptname>

This might give you an idea as to where your script is taking up the most time.

[Rodney Hills]

Is it possible you need to move the quote around the entire command?

csh -c '/directory/csh_perlscript.sh >> /tmp/perlscript.log 2>&1'

Remember that cronjobs are run with the standard /bin/sh and as such your "csh" is being called from "sh". Putting the quotes around the whole line will force the output redirection to be processed by "csh" and not "sh".

HTH

-- Rod Hills

[Kenneth_18]

Thanks guys for the trouble...hope you can keep 'em coming still. I think my initial problem description was a bit blurred.

I have a perl script named perlscript.pl that has a multi argument input. It is being executed inside a csh enviroment.

When I directly run the script through a terminal, it outputs the result of an sql query to the screem almost immediately.

When I try to redirect the output to a log file, it is taking considerable time for the log file to be updated, sometimes it takes up to 10 minutes or more.

The perl script is run through cron but as it contains a long argument list, I also placed it inside a csh script. The cron runs the csh script. The csh script runs the cron.

My problem is actually when redirecting the output of the perl script itself. The output logfile is not being updated in realtime although the script is working normally.

Hope you guys can keep the suggestions and possible solutions coming!

Thanks in advance.

[Rodney Hills · Accepted solution]

perl like other languages buffers it output, so if you have a long running process that outputs a few short lines, you won't see the output on the disk file immediatelly until the program exits.

You can force perl to flush it's buffer after every print line, by setting $|=1;

HTH

-- Rod Hills

[Kenneth_18]

Hey Rod,

Thanks! That did it. I thought it was the csh shell whose bufferring the input to the logfile and never occurred that it was perl who was bufferring its output.

Again many thanks!
Ken