Auto Trimming of log files

[Henry Chua]

Hi,

I have written a script to capture activities on my system. However I am afraid that the log might grow too large in future.. Is there any ingenious way to write another script so as to check and delete the lines in the log that is say 30days old?

Thank u..

[Fred Ruffet]

What about a simple rotate log every 30 day and keep 1 or 2 archived log file ?

Regards,

Fred

[Sanjay Kumar Suri]

You can use SAM also to trim the log files.

sks

[Thierry Poels_1]

hi,

you can write your own script to cut off lines when it's bigger than n lines or n MB.
But if you do not want to reinvent the wheel then have a loot at:
http://hpux.its.tudelft.nl/hppd/hpux/Sysadmin/logrotate-2.5/

regards,
Thierry.

[Nguyen Anh Tien]

Hi Henrry.
Intead of using one file for all. My crontab script write out one file for one day. like this:
[TIAN:/] cat script/ia_start.sh
################# Applciation Start Script for cron file #################

#### Make the log_file ####

GetDate=`date +"%Y%m%d"`
LOG_NM=/script/log/ia_start/ia_start.$GetDate

echo "=====[Today is `date`] Application Start Script START...]=====" >> $LOG_NM


##------------------------ Oracle Start -----------------------------
echo " Firstly, Start Oracle : `date` " >> $LOG_NM

[Nguyen Anh Tien]

Another way.
1, Create mark point for each lines by this shell variable
GetDate=`date +"%d"`+"HenryChua"
2, at script you adding this variable to the end (or begin) OF each line.
3. at trimming script. you look for string:
30HenryChua and delete it.
tienna

[Peter Godron]

Hi,
I would suggest creating a directory and writing one file per day, based on the day of the month.
This will automatically overwrite your log from one month ago and is very is to analyse.

./script > `date +"%d"`.log 2>&1

Regards

[twang]

the way i am using is to gzip the old file and remove after one month, the following actions run daily,
find $LOGDIR -name appllog\*.log -mtime +1 -exec /usr/contrib/bin/gzip {} \;
find $LOGDIR -name appllog\*.gz -mtime +31 -exec rm {} \;

[Michael D. Zorn]

Rather than 'rm', ypu might want to do

cat /dev/null > theLogFile

right after the save.

Some programs don't automatically create the log file if it isn't there.

As the others said, it's a lot easier to save the whole log file every 30 days (or however often makes sense) and start a new one, than try to pull out lines.

I usually save log file with names like

wtmp2004_12.gz

[Daavid Turnbull]

I have used some thing like the script below for years.

Organising archives directories named by month and year means that archives can be easily removed one month at a time.
_________________

#!/bin/ksh
#
# retarc - RETire and ARChive
#
# files in the specified directory which are over a certain size are
# 1) renamed to filename.date
# 2) compressed
#
# Author: Daavid June 1, 2000

yearStr=`date +"%E"`
monthStr=`date +"%m"`
dayTimeStr=`date +"%d_%H_%M"`
archiveDir="archive"
dest=

startDir=$1

if test -z "$2"; then
# 20 Mb
minSize=20971520
else
minSize=$2
fi


if test -z "$startDir"; then
echo "Usage : $0 dirName [minFileSize]"
echo " where dirName is the location of files to be retired and archived"
echo " minFileSize is the minimum file size in bytes to be processed."
echo " (default is $minSize bytes)"
else
# Ok - time for action, first make sure the correct dir exists
dest="${startDir}/$archiveDir"
mkdir $dest > /dev/null 2>&1
dest="${dest}/$yearStr"
mkdir $dest > /dev/null 2>&1
dest="${dest}/$monthStr"
mkdir $dest > /dev/null 2>&1
for file in `ls $startDir`
do
if test -n {`echo $file | grep "\.Z\$"`} ; then
cd $startDir
if test -f $file ; then
size=`ls -sn1 $file | awk '{ print \$6 }'`
if (( $size > $minSize )) ; then
cp $file $dest/$file.$dayTimeStr
if (( $? == 0 ))
then
# 'zero' the original
>$file
compress $dest/$file.$dayTimeStr
else
echo "Copy Failed!"
fi
fi
fi
fi
done
fi