topic Re: search and delete script in Operating System - Linux

search and delete script

Shivkumar — Thu, 25 Jan 2007 13:39:44 GMT

Hi,

I want to search and delete using sed some lines from a text files containing some config info.

Say for example, file.txt has content like:-

|abc |123|
|bcg 345989|
|fdk |8977r89q|
|jkhssdf |987q89r89|
|iuhuihsfdah |897q89r|
|sadjfiouoas |jnasdfjkhasdjk|
|jkhjkfasfa| 901590fasklfja

I want to search "jkhssdf" and "sadjfiouoas" and delete them and put the content in a file.
Can someone suggest a quick shell or perl script ?

Appreciate your help.

Thanks,
Shiv

Re: search and delete script

James R. Ferguson — Thu, 25 Jan 2007 13:58:10 GMT

Hi Shiv:

This will find the tokens you indicated and funnel those matching lines into the STDERR stream. The non-matching lines will be directed to STDOUT:

# perl -ne 'if (m{^\|jkhssdf|^\|sadjfiouoas}) {print STDERR} else {print}' file > file.1 2> file.2

...Thus the original file ("file") is left intact and "file.1" contains the non-matching lines; "file.2" contains the matching lines.

Regards!

...JRF...

Re: search and delete script

Sandman! — Thu, 25 Jan 2007 14:18:20 GMT

Re: search and delete script

Haridharan — Thu, 25 Jan 2007 20:50:46 GMT

Shiv,

Try the below....

awk '{ if ($1=="|abc" || $1=="|bcg") print $2; else print}' file > file.out

Re: search and delete script

Dennis Handly — Thu, 25 Jan 2007 21:28:34 GMT

If you don't mind two passes over your data file you can just use grep:

$ grep -e jkhssdf -e sadjfiouoas file.txt > file-extract.txt
$ grep -v -e jkhssdf -e sadjfiouoas file.txt > new_file.txt

Re: search and delete script

perumal_2 — Fri, 26 Jan 2007 01:03:39 GMT

Hi Shiv

You can do this in simple vi.
Open your config file in vi editor
press "Esc"
you will be given with the edit prompt :
The type the below to replace jkhssdf
%s/jkhssdf//g
and to replace the sadjfiouoas
%s/sadjfiouoas//g
g stands to do the change globally in the whole file.
Be aware the change wil be made in the current file. If you want to have old file also for reference take a copy of the same before doing the change.
sed would be the best option to find and replace and keep the original intact
sed -e "s/jkhssdf//" -e "s/sadjfiouoas//" original_file_name >new_file_name

TQ
Perumal

Re: search and delete script

Peter Nikitka — Fri, 26 Jan 2007 12:42:07 GMT

Hi,

I assume you will specify your string to delete being a complete content of a field.
I that is right, I recommend NOT using grep for this filtering, as Dennis suggested.
Having data
|abc |123|
|bcg 345989|
|fdabck |8977r89q|

and trying to drop 'abc'
will lead to a deletion of the last line as well.

Question: I see '|' as delimiter, and your content contains sometimes space as well.
Does it have any meaning - so you see it as a part of the content as well - or is it just a formatting character which could be dropped?
The procedure we have to use to detect - and remove - a field depending on user supplied strings will depend heavily on that.

mfG Peter

Re: search and delete script

Shivkumar — Fri, 26 Jan 2007 12:52:16 GMT

Yes. pipe and space both are delimiter here.

Regards,
Shiv

Re: search and delete script

Peter Nikitka — Fri, 26 Jan 2007 13:33:39 GMT

Hi,

preface:
- the first field is always empty (before the first delimiter), so I deal with fieldnumbers >1 only.
- a space is a field delimiter (and not just to ignore), a string
|a |b|
has five fields: EMPTY,a,EPMTY,b,EMPTY

case 1) The search string has to be looked up in the second field only
awk -F'[ |]' -v sea='str1 str2 ...' 'BEGIN {
s=split(sea,ars," ") }
{ for(j=1;j<=s;j++) if($2==ars[j]) next }
{print}' inputfile

case 2) If you want to lookup in all fields, enclose the 2nd line with an additional loop, e.g.
{ for(i=NF;i>1;i--) if($i) { for(j=1;j<=s;j++) if($2==ars[j]) next } }

mfG Peter

Re: search and delete script

James R. Ferguson — Fri, 26 Jan 2007 14:38:36 GMT

Hi SHiv:

> Yes. pipe and space both are delimiter here.

In that case, I might use:

# perl -ne 'if (m{([\s|])(jkhssdf|sadjfiouoas)([\s|])}) {print STDERR} else {print}' file > file.1 2> file.2

The '\s' is any whitespace character.

Now, if you want this generalized a bit, so that you could *pass* the argument(s) for which you want to search and delele, do this:

# perl -ne 'BEGIN{$re=shift};if (m{([\s|])($re)([\s|])}) {print STDERR} else {print}' jkhssdf\|sadjfiouoas file > file.1 2> file.2

Notice that now you pass the pattern to be found and then the name of the input file.

Notice too, that I escaped the "|" in the pattern so that the shell would not interpret it. The notation "a|b" means match "a" OR "b".

Regards!

...JRF...

Re: search and delete script

Dennis Handly — Fri, 26 Jan 2007 16:50:10 GMT

>Peter: I recommend NOT using grep for this filtering

There is nothing wrong with grep and in fact that issue will appear in awk, sed or perl. (JRF fixed his.)

You are correct that in all cases you need to add the field delimiters to make sure you don't pick up substrings.

So a fix to my example would be:
$ grep -e "|jkhssdf " -e "|sadjfiouoas "

Though a grep -w may work with my original example.

Re: search and delete script

Peter Nikitka — Sat, 27 Jan 2007 09:10:44 GMT

Hi,

remarks for Dennis:
I told, I recommend NOT using grep == when the whole field content is specified for filtering ==
In other cases a grep is prefectly okay.

In my awk solution, I did use a compare
... $2==ars[j] ...
an NOT a match operator: the latter would in deed nothing change to a grep solution.

The option '-w' is not available at HP-UX's grep, IMHO.

mfG Peter

Re: search and delete script

Dennis Handly — Sat, 27 Jan 2007 09:17:33 GMT

>recommend NOT using grep == when the whole field content is specified for filtering

This will work if you anchor it with the delimiter. (If you remember to do so, as you correctly pointed out.)

>The option '-w' is not available at HP-UX's grep

It is there in 11.11.