help with script

[so.nimda]

Hi,

I need the help of scripting gurus out there with this :

There's a need to monitor a directory to see if a new file has been created and dropped there. If there is, I need this same script to run an application to process this file.

How should I go about it? I don't think the script should be run as a cron job, but should be something (a daemon, maybe) in the background running all the time.

I'm not at all familiar with shell scripts, would appreciate if someone point me in the right direction.

At the same time, instead of shell script, would perl script be an alternate option ?

Thanks in advance.

[Dennis Handly · Accepted solution]

>There's a need to monitor a directory to see if a new file has been created and dropped there.

Does the file get removed if it is processed?
If it gets removed you can just do:
while : ; do
for file in directory/*; do
process-file-here
done
sleep 60
done

>I don't think the script should be run as a cron job

This would be the easiest. You just need a way to prevent another copy from running. It could look at ps(1) to see if there is another.

[OFC_EDM]

I'd use cron. It's the easiest and poses less issues.

Along with Dennis's suggestion. I looked more at the steps to finding a newer file. Without using Perl etc.

I've attached a file with a simple script.
Note that this script will also search any Sub-Directories. Don't know if that's an issue for you.

Is simply lists the files to standard output. You can adjust the script to redirect to a file. Or in your cron entry you could redirect to a file.

Cheers

[OFC_EDM]

Sorry meant to mention. That script was written on Tru64. The options to commands such as "date" and "find" may be different on your OS.

So test it first you may have to change some syntax.

Cheers

[OFC_EDM]

If you're curious about writing daemons here's a link to a snippet from an O'Reilly book that talks about it.

But I don't recommend writing daemons if you're new to scripting on Unix.

http://www.unix.com.ua/orelly/unix/upt/ch38_11.htm

Regards

[H.Merijn Brand (procura]

Here's an example of something like that in perl. This not only monitors files that have been dropped here, but also files that have been removed.

--8<---
#!/pro/bin/perl

use strict;
use warnings;

sub usage
{
my $err = shift and select STDERR;
print "usage: $0 [-t ]\n";
exit $err;
} # usage

use Cwd;
use Getopt::Long qw(:config bundling nopermute passthrough);
my $wait = 1;
GetOptions (
"help|?" => sub { usage (0); },

"t|w=i" => \$wait,
) or usage (1);

sub current
{
my %dir;
opendir my $dh, "." or die "Cannot open .: $!\n";
foreach my $e (readdir $dh) {
(-f $e && $e =~ m/\.xml$/) or next;
$dir{$e} = [ "f", (stat $e)[7, 9, 10] ];
}
closedir $dh;
\%dir;
} # current

my @state;

sub state
{
my ($state, $f, $d) = @_;
my @date = localtime $d->{$f}[2];
my $date = sprintf "%4d-%02d-%02d %02d:%02d:%02d",
1900 + $date[5], ++$date[4], @date[3,2,1,0];
printf "%s %s %5d %s %s\n", $state, $d->{$f}[0], $d->{$f}[1], $date, $f;
} # state

my $prev = current;
my $cwd = getcwd ();

while (1) {
sleep ($wait);
my $dir = current;

# Deleted files
foreach my $f (sort keys %$prev) {
exists $dir->{$f} and next;
state "DEL", $f, $prev;
# Do something for deleted files here ...
}

foreach my $f (sort keys %$dir) {
if (exists $prev->{$f}) {
# Changed files
$f =~ m/^\.+$/ || "@{$prev->{$f}}" eq "@{$dir->{$f}}" and next;
state "CHG", $f, $dir;
# Do something for changed files here ...
}
else {
# New files
state "NEW", $f, $dir;
# Do something for new files here ...
}
}

$prev = $dir;
}
-->8---

Enjoy, Have FUN! H.Merijn

[Eric Lipede]

Hi

I have a script that checks for new "dump/ core files" if a new one is found an email is sent to some users. This email is only sent out ONCE.
Works a treat and its nice and simply. Hope it helps.

#!/sbin/sh
# WARNING: Do not modify this script
# Author : Eric A .lipede

###############################################

ODIR="/VMD1/acudump"
DDIR="/hpuxsw/utilities/scratch/acudump"
DLIST="$DDIR/acudump_list_orig"
ELIST="$DDIR/email_list"
SCRATCH="$DDIR/acudump_scratch.$(date +%y%m%d)"
HEADER="$DDIR/header.$(date +%y%m%d)"
MACHINE=`uname -a | awk '{ print $2 }'`
GROUP="itops itdevelopment el"

email()
{
export NAME="$MACHINE:Acuconnect dump monitor"
export USER="$MACHINE:Acuconnect dump monitor"
if [ $# -eq 2 ]
then
cat $2 | mailx -s "$1" $GROUP
else
echo " " | mailx -s "$1" $GROUP
fi
}

check_original()
{
if [ ! -f $DLIST ]
then
email "$DLIST not found - investigate"
exit
fi
}

create_new_list()
{
ll $ODIR | grep -v "\$mp" | grep rtf | awk '{ print $5"_"$8"_"$9 }' > $SCRATCH
if [ $? != 0 ]
then
email "$SCRATCH creation failed - investigate"
exit
fi
}

compare_files()
{
cat $SCRATCH | while read
do
FILEINSCRATCH=`echo ${REPLY}`
if
! grep $FILEINSCRATCH $DLIST > /dev/null 2>&1 &&
! grep $FILEINSCRATCH $ELIST > /dev/null 2>&1
then
NEWFILE=$ODIR/`echo $FILEINSCRATCH | awk -F_ '{print $3}'`
echo "Title:\t$NEWFILE" > $HEADER
echo "Contents:\t 1st 100 lines" >> $HEADER
head -100 $NEWFILE >> $HEADER
email "$MACHINE:Report - New Acuconnect exception file created: `echo ${REPLY} | awk -F_ '{print $3}'`" "$HEADER"
if [ $? = 0 ]
then
echo $FILEINSCRATCH >> $ELIST
rm $HEADER
fi
fi
done
}

clean_up()
{
if [ -f $SCRATCH ]
then
rm $SCRATCH
fi
}

# Start

check_original
create_new_list
compare_files
clean_up



###########################
The email list (that contains unique entries so we dont get duplicate emails going out) looks like ...
1017_10:46_dump.8975.rtf
1024_10:46_dump.9008.rtf
1032_10:46_dump.27552.rtf
1045_10:46_dump.27631.rtf
1051_10:46_dump.7946.rtf
etc

[so.nimda]

Hi,

@Dennis : thanks for the reply and the sample script. If it's to run as a crontab, how should the frequency be set? That's what stumped me. =(

@Kevin : thanks for the reply and sample script and link to writing daemons (yes, I should refrain from that as a noob =)). Most helpful.

@H.Merijn Brand (procura) : thanks for your reply and the sample perl script. It's an alternative.

@Eric : thanks for your reply. The sample script was helpful, too! =)

[Dennis Handly]

>If it's to run as a crontab, how should the frequency be set?

How often does your system go down?
If it does go down, will someone remember to start this script up if it only restarts once a day?

You may want it to restart every hour.

How to check if already running:
base=$(basename $0)
running=$(UNIX95=EXTENDED_PS ps -C $base -opid= | wc -w)
if [ $running -gt 1 ]; then
exit 0 # already running
fi