topic Re: How to truncate Log file to contain one month data only in Operating System - HP-UX

How to truncate Log file to contain one month data only

Robert Peregrin — Sun, 15 Aug 2010 07:50:19 GMT

Hi,

I have a log file which is growing to almost 2 GB in size. I need to truncate this log file to contain only one month old data in order to reduce its size.

Please help in giving a procedure for this.

Regards,

Robert Peregrin

Re: How to truncate Log file to contain one month data only

Suraj K Sankari — Sun, 15 Aug 2010 13:44:04 GMT

Hi,

to truncate you 0 you just need to give a > infront of your file.
>filename

to truncate file with condition then you need to write a script.

Suraj

Re: How to truncate Log file to contain one month data only

Dennis Handly — Sun, 15 Aug 2010 15:28:28 GMT

Which logfile? Some are ascii text (syslog.log) and some are binary (wtmps).

It also depends on what is the date string in the file. And whether you need exactly 31 days or just the current month or you are only going to run the script on the last/first day of the month.

Re: How to truncate Log file to contain one month data only

Emil Velez — Mon, 16 Aug 2010 00:05:51 GMT

in SAM or the SMH tools there is a section under routine tasks where you can automatically trim log files and you can add non-standard log files to this tool and define a max size for the log file. YOu can then trim all of them at the same time and periodically using this tool.

Re: How to truncate Log file to contain one month data only

Raj D. — Mon, 16 Aug 2010 06:09:19 GMT

Robert,

If you want to trim the file , by reducing contents from the log file you can do so by removing first few lines,
1. count the lines, with wc -l logfile ,
2. trim it as desired.

- If you want a gzip copy of backup of the file elsewhere you can keep a copy.

-Example: To trim first 2000 lines from the file:
- stop the process writing the file.
- trim
# perl -i -ne 'print unless 1 .. 2000' logfile.log
- start the process, writing to it.
- check size/lines.

Hth,
Raj.

Re: How to truncate Log file to contain one month data only

rariasn — Mon, 16 Aug 2010 07:19:43 GMT

Hi Robert:

Create a script to switch and compress the log file.

- Stop application
- mv logfile to logfile.old
- Initialize new logfile
- Start application
- compress the old logfile

schedule (cron, ovo, ...) the script in specified dates and times.

rgs,

Re: How to truncate Log file to contain one month data only

Robert Peregrin — Mon, 16 Aug 2010 09:17:08 GMT

Thanks for your suggestions guys.

The log file is a text file.

The SAM utility is nice but we need the triming to be automatic. A simple script would be useful which I can run as a cron job.

We are currently using the below script. If you can have better and easier suggestions, then it would be helpful.

#!/usr/bin/sh
#
# this script should be run (via cron) daily to keep DMC log at a
# reasonable size
#
WHOAMI=/usr/bin/whoami
ECHO=/usr/bin/echo
CP=/usr/bin/cp
DATE=/usr/bin/date
CAT=/usr/bin/cat
ED=/usr/bin/ed
LOGGER=/usr/bin/logger
DMCLOGDIR=/var/opt/dmc/logs
DMCLOG=dmc.log

#
# make sure script is running as root
#
if [ `$WHOAMI` != "root" ] ; then
$ECHO you must be root to run this script
exit
fi

#
# copy the DMC log file
#
cd $DMCLOGDIR
$CP $DMCLOG ${DMCLOG}.Day`$DATE -u +%d`

#
#
# truncate the DMC log file
#
$CAT - <1,$ d
w
q
EOF

#
# put log entry in the syslog file
#
$LOGGER $DMCLOG truncated at `$DATE`

Re: How to truncate Log file to contain one month data only

rariasn — Mon, 16 Aug 2010 09:26:30 GMT

Hi Robert:

#
#
# truncate the DMC log file
#

> $DMCLOG

rgs,

Re: How to truncate Log file to contain one month data only

Robert Peregrin — Mon, 16 Aug 2010 09:33:51 GMT

Note: I need to truncate the log file but it should still contain 1 month old data.

Re: How to truncate Log file to contain one month data only

rariasn — Mon, 16 Aug 2010 09:47:22 GMT

Hi Rorbert:

Sample scheduled script with cron:

# cd /var/spool/cron/crontabs

# cat root

30 1 1 * * /tmp/test.sh #Test

# ll root

-rw------- 1 root sys 30 Aug 16 12:43 root

The entries in a crontab file are lines of six fields each. The fields are separated by spaces or tabs. The lines have the following format:

minute hour monthday month weekday command

# crontab -l

rgs,

Re: How to truncate Log file to contain one month data only

Dennis Handly — Mon, 16 Aug 2010 11:03:27 GMT

>it should still contain 1 month old data.

What is the format of the file and what are the answers to my other questions?

Re: How to truncate Log file to contain one month data only

Robert Peregrin — Mon, 16 Aug 2010 11:17:05 GMT

The file is called dmc.log. Its an application specific log file in text format.

Sample date string in the file is "Sat Aug 14 18:42:19 2010."

Yes, I need to keep exactly 1 month old data (number of days depends on the month as some months have 30 days, some have 31 and February have 29 or 28 days) in the file.

It doesn't matter when I run the script for as long as I will be able to truncate the file with 1 month old data kept.

Re: How to truncate Log file to contain one month data only

Elmar P. Kolkman — Mon, 16 Aug 2010 11:52:02 GMT

This should work for selecting the lines from the log file.

#!/usr/bin/perl -w

use strict;

my @prev_months= qw(Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

my $fh;
open($fh,$ARGV[0]);
my $line;
my @parts;

while ($line = <$fh>) {
@parts=split(' +',$line);
if (($parts[1] eq $prev_months[$mon]) && ($parts[2] > $mday)) {
print $line;
}
if (($parts[1] eq $prev_months[($mon+1)]) && ($parts[2] <= $mday)) {
print $line;
}
}

Re: How to truncate Log file to contain one month data only

Dennis Handly — Mon, 16 Aug 2010 11:55:04 GMT

>I need to keep exactly 1 month old data (number of days depends on the month as some months have 30 days, some have 31 and February have 29 or 28 days) in the file.

The number of days in a month is useless unless you are at the end of the month. If you are in the middle, what does it mean?

>It doesn't matter when I run the script for as long as I will be able to truncate the file with 1 month old data kept.

I assume we can make the assumption that if today is Aug 16, you want the info from July 16 to today?

You could make an awk script to scan every record for these fields:
Sat Aug 14 18:42:19 2010

if ($2 == prevmon && $3 >= currday && $5 >= prevyear ||
$2 == currmon && $5 == curryear) copy=1

You would have to do date arithmetic to figure out what the previous month's name and year is. The current month, day and year can be gotten from date(1).

Once "copy" is set, just read & print to the end of the file.

Re: How to truncate Log file to contain one month data only

Robert Peregrin — Wed, 18 Aug 2010 10:33:57 GMT

Thanks Guys. Your suggestions are good but we chose to retain the running script we are using.

Re: How to truncate Log file to contain one month data only

Robert Peregrin — Wed, 18 Aug 2010 10:41:20 GMT

I have already found an alternative solution to my question.