topic Re: Auto Trimming of log files in Operating System - HP-UX

Auto Trimming of log files

Henry Chua — Thu, 13 Jan 2005 03:08:52 GMT

Hi,

I have written a script to capture activities on my system. However I am afraid that the log might grow too large in future.. Is there any ingenious way to write another script so as to check and delete the lines in the log that is say 30days old?

Thank u..

Re: Auto Trimming of log files

Fred Ruffet — Thu, 13 Jan 2005 03:23:11 GMT

What about a simple rotate log every 30 day and keep 1 or 2 archived log file ?

Regards,

Fred

Re: Auto Trimming of log files

Sanjay Kumar Suri — Thu, 13 Jan 2005 03:33:30 GMT

You can use SAM also to trim the log files.

sks

Re: Auto Trimming of log files

Thierry Poels_1 — Thu, 13 Jan 2005 03:39:19 GMT

hi,

you can write your own script to cut off lines when it's bigger than n lines or n MB.
But if you do not want to reinvent the wheel then have a loot at:
http://hpux.its.tudelft.nl/hppd/hpux/Sysadmin/logrotate-2.5/

regards,
Thierry.

Re: Auto Trimming of log files

Nguyen Anh Tien — Thu, 13 Jan 2005 03:50:20 GMT

Hi Henrry.
Intead of using one file for all. My crontab script write out one file for one day. like this:
[TIAN:/] cat script/ia_start.sh
################# Applciation Start Script for cron file #################

#### Make the log_file ####

GetDate=`date +"%Y%m%d"`
LOG_NM=/script/log/ia_start/ia_start.$GetDate

echo "=====[Today is `date`] Application Start Script START...]=====" >> $LOG_NM

##------------------------ Oracle Start -----------------------------
echo " Firstly, Start Oracle : `date` " >> $LOG_NM

Re: Auto Trimming of log files

Nguyen Anh Tien — Thu, 13 Jan 2005 03:59:51 GMT

Another way.
1, Create mark point for each lines by this shell variable
GetDate=`date +"%d"`+"HenryChua"
2, at script you adding this variable to the end (or begin) OF each line.
3. at trimming script. you look for string:
30HenryChua and delete it.
tienna

Re: Auto Trimming of log files

Peter Godron — Thu, 13 Jan 2005 04:15:28 GMT

Hi,
I would suggest creating a directory and writing one file per day, based on the day of the month.
This will automatically overwrite your log from one month ago and is very is to analyse.

./script > `date +"%d"`.log 2>&1

Regards

Re: Auto Trimming of log files

twang — Thu, 13 Jan 2005 04:29:16 GMT

the way i am using is to gzip the old file and remove after one month, the following actions run daily,
find $LOGDIR -name appllog\*.log -mtime +1 -exec /usr/contrib/bin/gzip {} \;
find $LOGDIR -name appllog\*.gz -mtime +31 -exec rm {} \;

Re: Auto Trimming of log files

Michael D. Zorn — Fri, 14 Jan 2005 11:55:44 GMT

Rather than 'rm', ypu might want to do

cat /dev/null > theLogFile

right after the save.

Some programs don't automatically create the log file if it isn't there.

As the others said, it's a lot easier to save the whole log file every 30 days (or however often makes sense) and start a new one, than try to pull out lines.

I usually save log file with names like

wtmp2004_12.gz

Re: Auto Trimming of log files

Daavid Turnbull — Sun, 16 Jan 2005 17:03:22 GMT

I have used some thing like the script below for years.

Organising archives directories named by month and year means that archives can be easily removed one month at a time.
_________________

#!/bin/ksh
#
# retarc - RETire and ARChive
#
# files in the specified directory which are over a certain size are
# 1) renamed to filename.date
# 2) compressed
#
# Author: Daavid June 1, 2000

yearStr=`date +"%E"`
monthStr=`date +"%m"`
dayTimeStr=`date +"%d_%H_%M"`
archiveDir="archive"
dest=

startDir=$1

if test -z "$2"; then
# 20 Mb
minSize=20971520
else
minSize=$2
fi

if test -z "$startDir"; then
echo "Usage : $0 dirName [minFileSize]"
echo " where dirName is the location of files to be retired and archived"
echo " minFileSize is the minimum file size in bytes to be processed."
echo " (default is $minSize bytes)"
else
# Ok - time for action, first make sure the correct dir exists
dest="${startDir}/$archiveDir"
mkdir $dest > /dev/null 2>&1
dest="${dest}/$yearStr"
mkdir $dest > /dev/null 2>&1
dest="${dest}/$monthStr"
mkdir $dest > /dev/null 2>&1
for file in `ls $startDir`
do
if test -n {`echo $file | grep "\.Z\$"`} ; then
cd $startDir
if test -f $file ; then
size=`ls -sn1 $file | awk '{ print \$6 }'`
if (( $size > $minSize )) ; then
cp $file $dest/$file.$dayTimeStr
if (( $? == 0 ))
then
# 'zero' the original
>$file
compress $dest/$file.$dayTimeStr
else
echo "Copy Failed!"
fi
fi
fi
fi
done
fi