How to grep string from the dynamic file and send it by mail on real time

[subrata Sur]

Hi
I have a huge file(131 GB ) which is written logs on every secs . I need to capture certain string like 'error' from that logs and wanted to send it by mail whenever encountered with min resource .

Any idea how to do that

Thanx
With regds,
Subrata

[Steven E. Protter]

Shalom,

grep criteria > file

http://www.hpux.ws/mailfile2

Use the attachment feature of that program to send the file as an attachment. Its production code, fully documented.

perhaps tail -n 500 | grep critera > file

SEP

[Yogeeraj_1]

hi Subrata,

we have implemented something similar for monitoring of our Oracle Database log file.

See if you can adapt for your own environment.

file attached.

hope this helps!
kind regards
yogeeraj

[Yogeeraj_1]

hi again,

Forgot script.

Also, try to review the size of the "huge file". Not too practical performance wise. You may wish to use the logrotate tool so that it creates better managed file sizes.

kind regards
yogeeraj

[spex]

Hi Subrata,

The following script monitors for keywords specified in and sends an alert to ${EMAIL} upon a match.

# cat filemon.sh
#!/usr/bin/sh

EMAIL=your@email.addr

if [ ${#} -lt 2 ]
then
echo "usage: ${0} " >&2
exit 1
fi

MONFILE=${1}
PATFILE=${2}

if [ ! -r ${2} ]
then
echo "error: file ${2} does not exist or is not readable" >&2
exit 2
fi

tail -f ${MONFILE} |&
CPID=${!}
trap "kill -9 ${CPID}" 1 2 3 15

while read -p LINE
do
echo ${LINE} | grep -f ${PATFILE} -F -q -i
RC=${?}
if [ ${RC} -eq 0 ]
then
echo ${LINE} | mailx -m -s "Error!" ${EMAIL:=root}
echo error reported: ${LINE}
fi
done
exit 0

Syntax is:
filemon.sh
where contains fixed strings representing error keywords of interest.

You can pseudo-daemonize the script via:
# nohup filemon.sh hugefile patfile &

PCS

[A. Clay Stephenson]

Probably the ideal, ready-made command to do what you want to do is "tail -f logfile". It will poll your file once per second to see if the length has changed and output those changes to stdout. You then pipe that output (if any) to a filter that looks for any keywords and mails the output. Your filter could be grep, awk, Perl, ... . Tail -f is a good choice because other than initially, it doesn't read the entire file; instead it lseeks() to its last position and reads from that point and then records the new last position. Because Perl can seek, you could choose to do everything inside a Perl script and not use tail.

Note that I have intentionally not written the code for you because that really does you no favors.