Archive of Quickly growing log files

[Richard Amick]

I have a couple of logfiles which I would like to preserve. These files grow pretty fast and result from processes which cannot be shutdown or suspended. Also, the file name must exist for one of the processes (the process will die if the logfile is not present.
I would like to keep the information stream uninterrupted while either copying or moving the older info off to tape or another file. I have tried using cp -p but between the time the copy is started and when it is completed, info is lost.
Thank you for your help.
Rich

[MARTINACHE]

It may be a better solution, but this is very quick :

Supposing your two files are log1 and log2.

create a link to log1 : ln -s log1 log
use log to save your logs.

when you need to backup :
rm log
ln -s log2 log
backup log1 to tape

Regards,

Patrice.

[John Palmer]

Hi Rich,

Patrice's suggestion will work providing your processes do not have the logfiles open for writing all the time.

I would suggest though that instead of doing
rm log
ln -s log

you do 'ln -sf log'
as this will reduce the time when 'log' does not exist.

Regards,
John

[James R. Ferguson]

Richard:

Here's a mechanism you can try. Assume that your logging script is called "mylogger". Execute it as follows:

# mylogger | sh -c 'while read LINE ; do echo $LINE >> /tmp/log ; done'

...and so it runs for awhile, and then do:

# mv /tmp/log /tmp/log.old

...and now mylogger continues to run and you have TWO log files

Try this with a simple script for that looks like:

#!/usr/bin/sh
while true
do
date
sleep 1
done

...JRF...

[Dan Hetzel]

Just my 2 cents...

What about having your program writing
its log into a named pipe (see mkfifo)
and having another process reading from
the same named pipe and writing to one or
many different files ?
The named pipe would allow some write buffering
while the reading process swaps output files.

[Carlos Fernandez Riera]

Try to implement signals in your processes.

Treat for example USER SIGNAL 1 ( singal 16) to close your log file, rename and, reopen again.