1745823 Members
3784 Online
108722 Solutions
New Discussion юеВ

Re: Scripting issue

 
SOLVED
Go to solution
Dadski
Advisor

Scripting issue

Hi All

I am attempting to create a quick and simple skulker script. This is what I intended.

A text file containing some thing similar to
cat HOUSEKEEP_FILES
-------------------------------------------------------------
FileSys Strin ACTION DAYS OLD
-------------------------------------------------------------
/tmp/martin/1 file1001 REMOVE 2
/tmp/martin/1 file1002 REMOVE 2
/tmp/martin/1 file1003 REMOVE 2
/tmp/martin/2 file1004 REMOVE 3
/tmp/martin/2 file1005 REMOVE 3
/tmp/martin/2 file1006 REMOVE 3


I wanted to use a sed and awk command, lets say to fin files older than 2 or 3 ($4) in the directory ($1) the actual file ($2). I dont think I'm to far away bu reading that but the command blurps. By the way I am just trying to list initially. The command I am using is,

sed -n '/REMOVE/p' HOUSEKEEP_FILES | awk '{system(find $1, " " -name $2," " -print)}'

is there any awk gurus who can simplify this?

thanks very much

Martin

 

 

P.S. this thread has been moved from HP-UX > System Administration to HP-UX > languages - HP Forums Moderator

8 REPLIES 8
Autocross.US
Trusted Contributor

Re: Scripting issue

Here's a solution that doesn't use sed/awk:

egrep "REMOVE" HOUSEKEEP_FILES | while read DIR FILE ACTION DAYS_OLD ; do
[[ $DAYS_OLD -gt 2 ]] && find $DIR -name "$FILE" -print
done
I drive way too fast to worry about calories.
Matti_Kurkela
Honored Contributor

Re: Scripting issue

Looks a bit dangerous. For each line, you're about to remove potentially several files.

For example, when receiving the first non-header line of HOUSEKEEP_FILES, your script would not remove only:
/tmp/martin/1/file1001

but also anything matching:
/tmp/martin/1/*/file1001
/tmp/martin/1/*/*/file1001
/tmp/martin/1/*/*/*/file1001
...

(When creating automated file removers, you *must* be as specific and predictable as possible. The hard way of learning this lesson involves restoring files from backups and restarting long-running processing jobs while thinking about how to explain this to an angry customer :-)

If the directory structure is predictable, it would be much safer to directly assemble the name of the file to be removed instead of fishing around with the find command.

If the file paths are of the form "/FileSys/Strin", then you could do something like this:

#!/bin/sh
grep '^/.*REMOVE' HOUSEKEEP_FILES | \
while read FileSys Strin junk; do
echo "rm \"$FileSys/$Strin\""
# rm "$FileSys/$Strin" # uncomment after testing
done

The first grep command passes through only the lines beginning with a '/' character, so we get rid of all the header lines. After that it's just a matter of reading each line, collecting the columns to variables and assembling the necessary command line.

Of course, the above approach is slow and causes excessive forking of new processes.
So, here's version 2:

#!/bin/sh
grep '^/.*REMOVE' HOUSEKEEP_FILES | \
awk '{ print $1 "/" $2 };' | \
xargs echo rm
# Replace "xargs echo rm" with "xargs rm" if OK

Note that there is no loop at all, just a single pipeline. You could even write it as a one-liner, but I think it's easier to understand when written on multiple lines.

MK
MK
OldSchool
Honored Contributor

Re: Scripting issue

be aware that the above solutions do not address the "age" of the file. And you've not defined what you mean by age. There are access times and modification times, but "creation" time is not available (unless the file is absolutely never touched after creation)
VK2COT
Honored Contributor
Solution

Re: Scripting issue

Hello,

Hmm, lot of possibilities.

I was even thinking about generalizing the
script. What if HOUSEKEEP_FILES could contain
different keywords in the third column (not only "REMOVE"). For example:

-------------------------------------------------------------
FileSys Strin ACTION DAYS OLD
-------------------------------------------------------------
/tmp/martin/1 file1001 REMOVE 2
/tmp/martin/1 file1002 REMOVE 2
/tmp/martin/1 file1003 REMOVE 2
/tmp/martin/2 file1004 REMOVE 3
/tmp/martin/2 file1005 KEEP 3
/tmp/martin/2 file1006 REMOVE 3

Here is my little script:

#!/bin/sh

awk '! /---|DAYS OLD/ {print}' HOUSEKEEP_FILES | while read SEARCHDIR FILE ACTION HOWOLD
do
case $ACTION in
REMOVE|remove) ACTION="rm -f" ;;
*) ACTION="ll" ;;
esac
find $SEARCHDIR -name $FILE -type f -mtime $HOWOLD -exec $ACTION {} \;
done


And here is what it would run:

find /tmp/martin/1 -name file1001 -type f -mtime 2 -exec rm -f {} \;
find /tmp/martin/1 -name file1002 -type f -mtime 2 -exec rm -f {} \;
find /tmp/martin/1 -name file1003 -type f -mtime 2 -exec rm -f {} \;
find /tmp/martin/2 -name file1004 -type f -mtime 3 -exec rm -f {} \;
find /tmp/martin/2 -name file1005 -type f -mtime 3 -exec ll {} \;
find /tmp/martin/2 -name file1006 -type f -mtime 3 -exec rm -f {} \;

I am sure there are many more ways to
do it :)

Cheers,

VK2COT
VK2COT - Dusan Baljevic
Peter Nikitka
Honored Contributor

Re: Scripting issue

Hi,

it is not clear, in which way you want to use the file name.
I suggest, that if there is no value of 'fil' found in awk, use any filespec found in the list. Else use only the specified filespec.

Wrap my awk in a script of your choice;
start with something like this in the commandline to test your expectations:

#filespec=file1002 # may be set
awk -v age=3 -v dir=/tmp/martin/1 -v fil="$filespec" 'NR<4 {next}
($3 == "REMOVE") && ($4 >= age) && ($1 == dir) { if(fil && $2 != fil) next
sys_str="find "dir" -type f -name "$2" -mtime +"age" -print"
print sys_str
system(sys_str)}' HOUSEKEEP_FILES

You could implement the handling of directory names the same way as I did with the filespec.


mfG Peter

The Universe is a pretty big place, it's bigger than anything anyone has ever dreamed of before. So if it's just us, seems like an awful waste of space, right? Jodie Foster in "Contact"
Dadski
Advisor

Re: Scripting issue

Thanks all for replying VK2COT you've nailed it, thats exactly what I was looking for. thanks to all.
Peter Nikitka
Honored Contributor

Re: Scripting issue

Hi,

nice, that we could help you!
Because you are new to this forum - Welcome! - I want to put yout attention to its unique point system:

http://forums11.itrc.hp.com/service/forums/helptips.do?#33

mfG Peter

The Universe is a pretty big place, it's bigger than anything anyone has ever dreamed of before. So if it's just us, seems like an awful waste of space, right? Jodie Foster in "Contact"
Mark McDonald_2
Trusted Contributor

Re: Scripting issue

Just a note - we have a similar script.

We use ZIP as an option too. So 2 day old logs get zipped, then deleted after 7 days.

So you would have:
/tmp/martin/1 file1001 ZIP 2
/tmp/martin/1 file1001 REMOVE 7