Operating System - HP-UX
1834355 Members
2453 Online
110066 Solutions
New Discussion

Re: How can I do this with Unix?

 
SOLVED
Go to solution
Rob Johnson_3
Regular Advisor

How can I do this with Unix?

I wasn't too sure where to post this question so I'm starting here. I have a need to do a search against several thousand files that DON'T contain any one of five to ten different strings of text. I know how to do a find command for any file that contain a certain string but don't know how to search for files that DON'T contain one of several things. How can I do this with the Unix OS?

The strings are something like:
1) aaa authentication + I don't remember the rest.
2) aaa authentication + I don't remember the rest + more text.
3) set authentication + I don't remember the rest.
4) set authentication + I don't remember the rest + more.
5) there may even be a couple of more strings of text.

P.S. I'm not very good with the Unix shell.

Thanks for any input on how I can do this.
23 REPLIES 23
Sandman!
Honored Contributor
Solution

Re: How can I do this with Unix?

use the "exec" option of the find command

# find -type f -exec egrep -ilv "(aaa|set) authentication" {} \;
A. Clay Stephenson
Acclaimed Contributor

Re: How can I do this with Unix?

Well, if you are going to do UNIX, you better learn to script at some level. One approach would be to leverage the find command and use the grep command to test it.



First create this command to test each file, call it scanit.sh:

--------------------------------------
#!/usr/bin/sh
FNAME=${1}
# first make sure this is some form of text
# file
file ${FNAME} | grep -i "text"
STAT=${?}
if [[ ${STAT} -eq 0 ]]
then # okay, it is a text file
grep -q -e "aaa auth xxxx" \
-e "bbb auth yyyy" \
-e "I'm not very good"
STAT=${?}
if [[ ${STAT} -ne 0 ]]
then
echo "${FNAME}" # no patterns match
fi
else
echo "${FNAME} ain't no text file!!" >&2
fi
exit 0
----------------------------------
Save scanit.sh and chmod 755 scanit.sh

Now cd to your desired starting directory:

find . -type f -exec scanit.sh {} \;


Man file, find, and grep to see how this works. ${?} is the status of the last command executed and 0 means that grep matched at least 1 of the patterns otherwise there were no matches. I haven't tested this but it should work.

If it ain't broke, I can fix that.
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

I'll try this tomorrow...I don't have a unix box and home.

Question on the firt response?
# find -type f -exec egrep -ilv "(aaa|set) authentication" {} \;

Where in the command could I put the different strings of text to search for? Also, I'm looking for files that don't contain the text. How would I know which ones don't?

Sandman!
Honored Contributor

Re: How can I do this with Unix?

What different strings you want to search for...could you provide a sample.

the "egrep" section of the find command contains the search strings. man egrep

Option "-v" to egrep excludes all files that don't contain the search strings.

cheers!
Muthukumar_5
Honored Contributor

Re: How can I do this with Unix?

To search a pattern which will not be contained in a file then use -v option on grep or egrep.

You can -exec option with find command. There is some limitation in that usage. In stead of that you can use xargs for that as,

find -type f | xargs egrep -iv '(aaa|set) authentication'

hth.
Easy to suggest when don't know about the problem!
Muthukumar_5
Honored Contributor

Re: How can I do this with Unix?

If you want to add more patterns that will not be in the file then,


find -type f | xargs egrep -iv '(aaa|set) authentication|||'

It will do grep with all pattens available.

hth.
Easy to suggest when don't know about the problem!
Amit Agarwal_1
Trusted Contributor

Re: How can I do this with Unix?

I don't think 'grep -v' would achieve what Rob wants.

grep -v simply means that display all the lines that do not match the pattern. It does not tell you whether the pattern matched any line or not. Rob wants to list the files that do not contain the pattern.

I would agree with Clay's script. You need to do positive grep and then check the return value to see whether any pattern matched or not. Similar script would be

for i in `ls -R`
do
if [ -f $i ]
then
grep -q -e -e $i
if [ $? -ne 0 ]
then
echo $i
fi
fi
done


Muthukumar_5
Honored Contributor

Re: How can I do this with Unix?

We can use grep with -e option to get egrep's effect. To negate some pattern in a file then,

grep -ev '||'

It will do the same to,

egrep -v '||'

hth.
Easy to suggest when don't know about the problem!
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

I want to find all files that DON'T contain any one of the following strings of text.

1)aaa authentication login default group tacacs+ local

2)aaa authentication login default tacacs+ local

3)aaa authentication login default local tacacs+

4)aaa authentication login default local group tacacs+

5)set authentication login tacacs enable telnet primary

6)set authentication login local enable
Simon Hargrave
Honored Contributor

Re: How can I do this with Unix?

This should do what you want: -

find . -type f -exec grep -lv -e "string1" -e "string2" -e "string3" {} \;

eg: -

find . -type f -exec grep -lv -e "aaa authentication + I don't remember the rest" -e "aaa authentication + I don't remember the rest + more text" -e "set authentication + I don't remember the rest" -e "set authentication + I don't remember the rest + more" -e "there may even be a couple of more strings of text" {} \;

This will execute a find from the current directory on all files (not directories), and for each execute a grep -v (-v means DOESN'T include), specifying a -e for each option you want to search for. The -l makes the grep simply display the name of the file that matches the (not found due to -v) files. See?

Therefore running this will provide a list of all files from the current directory that contain none of your strings. This is what you want?
Simon Hargrave
Honored Contributor

Re: How can I do this with Unix?

^^^^^ actually scrap that ^^^^^

As has correctly been said earlier the -v will show every file that contains at least one line without every clause you want so it's not correct.

An easy alternative may be: -

for file in `find . -type f`
do
grep -ql -e "string1" -e "string2" -e "string3" $file
if [ $? -ne 0 ]; then
echo $file
fi
done
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

I'm getting the following error when running the command. The command is:
find . -type f -exec grep -lv -e "aaa" -e "local" {} \;

Usage: grep -hblcnsviw pattern file . . .
grep: illegal option -- e
Usage: grep -hblcnsviw pattern file . . .
grep: illegal option -- e
Usage: grep -hblcnsviw pattern file . . .
grep: illegal option -- e
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

Oh yea, the box I'm executing this from is running SunOS
Simon Hargrave
Honored Contributor

Re: How can I do this with Unix?

It would appear you are using the script given to you on a HPUX forum on a Solaris server... The grep command is different, most likely you will have to use egrep without multiple -e parameters and specify a suitable regular expression.
Simon Hargrave
Honored Contributor

Re: How can I do this with Unix?

More specifically try this: -

find . -type f -exec egrep -lv "aaa|local" {} \;

I don't have a Sun box to test it so it's a guess...
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

I've put a script together as shown below to search all configs for certain strings. The problem I'm having is all configs are getting returned.
Any ideas?

#!/usr/bin/ksh
for files in `find /var/adm/CSCOpx/files/archive/shadow/Cisco_Router -type f`
do
egrep -l -e "aaa authentication login default group tacacs+ local" -e "aaa authentication login d
efault tacacs+ local" -e "aaa authentication login default local tacacs+" -e "aaa authentication
login default local group tacacs+" $files
if [ $? -ne 0 ]; then
echo $file
fi
done
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

Oh yea. I know the strings exist in most of the config files. I want it to only return the files that don't have the string.

A. Clay Stephenson
Acclaimed Contributor

Re: How can I do this with Unix?

That's the beauty of invoking grep -q (quiet); you should really only be interested in the return status (0 it found a match; anything else it didn't). When I said read the man pages, I meant it.
If it ain't broke, I can fix that.
Sandman!
Honored Contributor

Re: How can I do this with Unix?

You could put the search strings in a file and supply that as an argument to "egrep" within the find command.

find -type f -exec egrep -ilvf srchstrngs.in {} \;

cheers!
Rob Johnson_3
Regular Advisor

Re: How can I do this with Unix?

The problem I'm having is it's returning all the files that have a string of text in the search list.

The script I'm using is this. I'm saying to return files not equal to zero but it's still displaying all the files that do have a match.
I'm going nuts over this!!!

#!/usr/bin/ksh
for files in `find /var/adm/CSCOpx/files/archive/shadow/Cisco_Router -type f`
do
egrep -e "aaa authentication login default group tacacs+ local" -e "aaa authentication login default tacacs+ local" -e "aaa authentication login default local tacacs+" -e "aaa authentication login default local group tacacs+" $files
if [ $? -ne 0 ]; then
echo $files
fi
done
Sandman!
Honored Contributor

Re: How can I do this with Unix?

Rob,

You need to add "-v" as an option to your egrep command for excluding files that match the strings you'ave specified. See below:


=============================================
#!/usr/bin/ksh
for files in `find /var/adm/CSCOpx/files/archive/shadow/Cisco_Router -type f`
do
egrep -v -e "aaa authentication login default group tacacs+ local" -e "aaa authentication login default tacacs+ local" -e "aaa authentication login default local tacacs+" -e "aaa authentication login default local group tacacs+" $files
if [ $? -ne 0 ]; then
echo $files
fi
done
=============================================

best of luck!
Jan van den Ende
Honored Contributor

Re: How can I do this with Unix?

Rob,


I am also a bit of a stranger in Unix land, but since my boss requires me to aquire Unix skills, I decide that following this forum would not be the worst option.


Following this thread set me to thinking.
Reading the various answers, I slowly grew an idea about what __I__ would do in this situation, if no immediate *X solution would transpire:

1. Export the directory as an NFS mount point.
2. On a VMS system, MOUNT that point (which thereby on VMS becomes a DNFS type disk)
3. SET DEFAULT DNFSxxx:[000000]
( xxx is whatever the previous MOUNT generated as the next unit number; SET DEFAULT is more or less equivalent to cd).

(any coming wrappings to be attributed to ITRC, they are NOT part of the command)

$ SEARCH [...]*.*
"

Explanation:
FIND [...] :operate on all files in this directory and all subdirectories
*.* : operate on all file names of all file types
(in VMS one could add another ".*" to also search all previous versions, but since they are deleted in Unix anyway, that is not applicable)
^searchtext","searchtext2"
search all specified files for any of the comma-separated list of quoted string.
,... : not part of the command syntax per se. Just a placeholder for any amount of next elements of the list of strings to be searched for.
/MATCH=NOR: EXclude if found
/LIMIT=1: stop searching this file after first find
/SILENT : do not echo records that are selected (otherwise any record NOT containing a search string would be echoed!)
/HEADER (this one just for clarity: it is the default anyway) : echo names of files that meet search criteria
/OUTPUT= : any filename you want to hold the resulting output, being the filenames that do NOT contain any of those strings.
No special file naming, so placed in the default (Unix, say work:) directory, ie, on the *X system.
4. behave, so DISMOUNT DNFS.

----

Okay, I do admit that all details of the command made me thinking for a while, but I thought it was a nice challenge.

Caveat: you need a VMS system for this, that can reach your Unix, and you need to be allowed to export on your Unix (or Linux) box, and you need (assistence of someone with) sufficient privileges on VMS for the MOUNT and DISMOUNT

Nevertheless: I enjoyed the challenge!

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Amit Agarwal_1
Trusted Contributor

Re: How can I do this with Unix?

If you are on Solaris, use '-s' option and if you are on HP-UX use '-q' option with egrep.

The optioans turns on the quiet mode of egrep facility. Do not use '-v' option.