script to count files on daily basis

[NDO]

Hello!

I am trying to write a script that will count files with *.cdr and *.tap extensions and will sent it to me via email.

At this point I have try the following:

ls -lrt *.cdr | awk '{if(($6 == "Jul") && ($7 == "11")) print $0}' | wc -l

but that has a problem that will only give me 11th of july files, so I will have to edit the script everyday....

[Bill Hassell]

I think there is an additional requirement that you did not mention.

This will count files with *.cdr and *.tap extensions:

ls -1 *\.cdr *\.tap | wc -l

Did you leave out a requirement that the test only counts files created on today's date?

If that is the real requirement, using ls -l very cumbersome.

Use find, something like this:

find . \( -name *.cdr -o -name *.tap \) -mtime -1 | wc -l

-mtime -1 shows files that were created or modified in the last 24 hours.

[NDO]

the idea is to run the script after the midnight (put on crontab) to count those files of the previuos day.

Your

find

example returned :

find . \( -name *.cdr -o -name *.tap \) -mtime -1 | wc -l
find: bad option ICTEMSC32015061614501134843.cdr
0

as an example:

ls -lrt *.cdr | awk '{if(($6 == "Jul") && ($7 == "14")) print $0}' | wc -l
1570 

[Patrick Wallek]

You need quotes around the name specifications, like so:

find . \( -name "*.cdr" -o -name "*.tap" \) -mtime -1 | wc -l

[NDO]

its actually giving me a total:

find . \( -name "*.cdr" -o -name "*.tap" \) -mtime -1 | wc -l
1789

But what I really want is an output like this:

cdr: 1197
tap: 229

[Patrick Wallek]

Try this:

CDRCOUNT=$(find . -name "*.cdr" -mtime -1 | wc -l)
TAPCOUNT=$(find . -name "*.tap" -mtime -1 | wc -l)
echo "cdr: ${CDRCOUNT}"
echo "tap: ${TAPCOUNT}"

[NDO]

Patrick:

I did run your script which gave the following output:

cdr: 1588
tap: 224

But if I just do :

ls -lrt *.cdr | awk '{if(($6 == "Jul") && ($7 == "14")) print $0}' | wc -l
1570

and :

ls -lrt *.tap | awk '{if(($6 == "Jul") && ($7 == "14")) print $0}' | wc -l
247

I have different values... 

[Patrick Wallek]

The "find" with the '-mtime -1' will shows files created within the last 24 hours.  It will NOT just show files created yesterday.

If the find command is run at midnight, then the count should be pretty accurate.  However, if you want to be able to specify the date to find the files for, then that will take some more work...

[Patrick Wallek]

OK, try this.  This assumes you have 'perl' in your path.  If you don't find where the perl executable is and add the full path to the perl commands.

#!/usr/bin/sh

# Get yesterdays date
YESTMON=$(perl -MPOSIX -le '@t=localtime;--$t[3];print strftime "%Y %b %d",@t' | awk '{print $2}')
YESTDAY=$(perl -MPOSIX -le '@t=localtime;--$t[3];print strftime "%Y %b %d",@t' | awk '{print $3}')

CDRCOUNT=$(ls -lrt *.cdr | awk '{if (($6 == "'${YESTMON}'") && ($7 == "'${YESTDAY}'")) print $0}' | wc -l)
TAPCOUNT=$(ls -lrt *.tap | awk '{if (($6 == "'${YESTMON}'") && ($7 == "'${YESTDAY}'")) print $0}' | wc -l)

echo "cdr: ${CDRCOUNT}"
echo "tap: ${TAPCOUNT}"

Here is an ls of my working directory:

# ls -l
total 80
drwxr-xr-x 2 root sys 8192 Jul 15 10:31 .
drwxrwxrwt 16 root root 8192 Jul 15 10:42 ..
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file1.cdr
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file2.cdr
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file3.cdr
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file3.tap
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file4.tap
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file5.tap
-rw-r--r-- 1 root sys 0 Jul 14 10:00 file6.tap
-rwx------ 1 root sys 1790 Apr 8 14:22 generate_cfg2html-int.sh
-rwx------ 1 root sys 1794 Apr 8 14:02 generate_cfg2html.sh
-rwxr--r-- 1 root sys 499 Jul 15 10:42 pwtest.sh

Here is the script running:

# ./pwtest.sh
cdr: 3
tap: 4

[Dennis Handly]

>ls -1 *\.cdr *\.tap | wc -l

There is no need to use "\." for filename generation, a period isn't special like for REs.

>Did you leave out a requirement that the test only counts files created on today's date?

That requirement can't be satisfied, only "modified" today.

>what I really want is an output like this:

If you have zillions of files, it is better to use find to get both and then separate the files after.

>find with the '-mtime -1' will shows files created within the last 24 hours.

Actually modified.

>This assumes you have 'perl' in your path.

If you have perl, you can do all the work there.  :-)

Otherwise have perl return two timestamps and create two reference files and pass them into find(1).

[NDO]

Patrick 

your second script worked fine, but I need to mail that output to an email address

[Patrick Wallek · Accepted solution]

Adding more requirements to the script?

Add this line to the end of the script:

echo "cdr: ${CDRCOUNT} \n tap: ${TAPCOUNT}" | mailx -s "cdr and tap file counts" you@somedomain.com

[Steven Schweda]

> [...] but I need to mail that output to an email address

   What's stopping you?  Did you try a  Forum (or Web) search for, say:

      e-mail from shell script hp-ux

???

[NDO]

Thanks a lot, I knew that I needed to use mailx, but I was not sure how because of the requirement to display 2 outputs. But thank you

[NDO]

Hi

for some reason and since 31st of july, the script is providing erroneous output:

#!/usr/bin/sh

# Get yesterdays date
YESTMON=$(perl -MPOSIX -le '@t=localtime;--$t[3];print strftime "%Y %b %d",@t' | awk '{print $2}')
YESTDAY=$(perl -MPOSIX -le '@t=localtime;--$t[3];print strftime "%Y %b %d",@t' | awk '{print $3}')

CDRCOUNT=$(ls -lrt *.cdr | awk '{if (($6 == "'${YESTMON}'") && ($7 == "'${YESTDAY}'")) print $0}' | wc -l)
TAPCOUNT=$(ls -lrt *.tap | awk '{if (($6 == "'${YESTMON}'") && ($7 == "'${YESTDAY}'")) print $0}' | wc -l)

echo "cdr: ${CDRCOUNT}"
echo "tap: ${TAPCOUNT}"

and the output is:

cdr: 0
tap: 0

can you help

[Dennis Handly]

>for some reason and since 31st of july, the script is providing erroneous output:

What part isn't working?

Add: echo "Yesterday: $YESTMON|$YESTDAY"

As far as I can tell, this never worked during the first 9 days of a month.

You need to do a numeric compare:

CDRCOUNT=$(ls -l *.cdr | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 ==
 '${YESTDAY}' {++sum} END {print sum}')
TAPCOUNT=$(ls -l *.tap | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 ==
 '${YESTDAY}' {++sum} END {print sum}')

[NDO]

when adding echo "Yesterday: $YESTMON|$YESTDAY"

the result is:

Yesterday: Aug|03

and when changing with the piece of code you suggested, it does come right:

cdr: 1608
tap: 194

and if I place the script under a different directory and modify the script acordingly:

CDRCOUNT=$(ls -lrt /data/ICTPRD/bmd1/rating/processed/*.cdr | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 == '${YESTDAY}' {++sum} END {print sum}')
TAPCOUNT=$(ls -lrt /data/ICTPRD/bmd1/rating/processed/*.tap | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 == '${YESTDAY}' {++sum} END {print sum}')

produces the following results:

./contar[12]: /usr/bin/ls: The parameter list is too long.
cdr: 0
tap: 194

So...

[Dennis Handly]

>if I place the script under a different directory and modify the script accordingly:

"accordingly" means you have to handle mass quantities, which means you need to use find, not ll(1).

Or you can use this bandaid:

CDRCOUNT=$(cd /data/ICTPRD/bmd1/rating/processed;  ls -lrt *.cdr | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 == '${YESTDAY}' {++sum} END {print sum}')

TAPCOUNT=$(cd /data/ICTPRD/bmd1/rating/processed; ls -lrt *.tap | awk 'BEGIN {sum = 0} $6 == "'${YESTMON}'" && $7 == '${YESTDAY}' {++sum} END {print sum}')

[NDO]

thanx a lot, already assigned kudos