search and delete script

[Shivkumar]

Hi,

I want to search and delete using sed some lines from a text files containing some config info.

Say for example, file.txt has content like:-

|abc |123|
|bcg 345989|
|fdk |8977r89q|
|jkhssdf |987q89r89|
|iuhuihsfdah |897q89r|
|sadjfiouoas |jnasdfjkhasdjk|
|jkhjkfasfa| 901590fasklfja

I want to search "jkhssdf" and "sadjfiouoas" and delete them and put the content in a file.
Can someone suggest a quick shell or perl script ?

Appreciate your help.

Thanks,
Shiv

[James R. Ferguson · Accepted solution]

Hi Shiv:

This will find the tokens you indicated and funnel those matching lines into the STDERR stream. The non-matching lines will be directed to STDOUT:

# perl -ne 'if (m{^\|jkhssdf|^\|sadjfiouoas}) {print STDERR} else {print}' file > file.1 2> file.2

...Thus the original file ("file") is left intact and "file.1" contains the non-matching lines; "file.2" contains the matching lines.

Regards!

...JRF...

[Sandman!]

Try the shell script below:


#!/usr/bin/sh

ex -s file <<-EOF
/^|jkhssdf/w file.out | /^|jkhssdf/d
/^|sadjfiouoas/w >> file.out | /^|sadjfiouoas/d
wq
EOF

[Haridharan]

Shiv,

Try the below....

awk '{ if ($1=="|abc" || $1=="|bcg") print $2; else print}' file > file.out

[Dennis Handly]

If you don't mind two passes over your data file you can just use grep:

$ grep -e jkhssdf -e sadjfiouoas file.txt > file-extract.txt
$ grep -v -e jkhssdf -e sadjfiouoas file.txt > new_file.txt

[perumal_2]

Hi Shiv

You can do this in simple vi.
Open your config file in vi editor
press "Esc"
you will be given with the edit prompt :
The type the below to replace jkhssdf
%s/jkhssdf//g
and to replace the sadjfiouoas
%s/sadjfiouoas//g
g stands to do the change globally in the whole file.
Be aware the change wil be made in the current file. If you want to have old file also for reference take a copy of the same before doing the change.
sed would be the best option to find and replace and keep the original intact
sed -e "s/jkhssdf//" -e "s/sadjfiouoas//" original_file_name >new_file_name

TQ
Perumal

[Peter Nikitka]

Hi,

I assume you will specify your string to delete being a complete content of a field.
I that is right, I recommend NOT using grep for this filtering, as Dennis suggested.
Having data
|abc |123|
|bcg 345989|
|fdabck |8977r89q|

and trying to drop 'abc'
will lead to a deletion of the last line as well.

Question: I see '|' as delimiter, and your content contains sometimes space as well.
Does it have any meaning - so you see it as a part of the content as well - or is it just a formatting character which could be dropped?
The procedure we have to use to detect - and remove - a field depending on user supplied strings will depend heavily on that.

mfG Peter

[Shivkumar]

Yes. pipe and space both are delimiter here.

Regards,
Shiv

[Peter Nikitka]

Hi,

preface:
- the first field is always empty (before the first delimiter), so I deal with fieldnumbers >1 only.
- a space is a field delimiter (and not just to ignore), a string
|a |b|
has five fields: EMPTY,a,EPMTY,b,EMPTY

case 1) The search string has to be looked up in the second field only
awk -F'[ |]' -v sea='str1 str2 ...' 'BEGIN {
s=split(sea,ars," ") }
{ for(j=1;j<=s;j++) if($2==ars[j]) next }
{print}' inputfile

case 2) If you want to lookup in all fields, enclose the 2nd line with an additional loop, e.g.
{ for(i=NF;i>1;i--) if($i) { for(j=1;j<=s;j++) if($2==ars[j]) next } }

mfG Peter

[James R. Ferguson]

Hi SHiv:

> Yes. pipe and space both are delimiter here.

In that case, I might use:

# perl -ne 'if (m{([\s|])(jkhssdf|sadjfiouoas)([\s|])}) {print STDERR} else {print}' file > file.1 2> file.2

The '\s' is any whitespace character.

Now, if you want this generalized a bit, so that you could *pass* the argument(s) for which you want to search and delele, do this:

# perl -ne 'BEGIN{$re=shift};if (m{([\s|])($re)([\s|])}) {print STDERR} else {print}' jkhssdf\|sadjfiouoas file > file.1 2> file.2

Notice that now you pass the pattern to be found and then the name of the input file.

Notice too, that I escaped the "|" in the pattern so that the shell would not interpret it. The notation "a|b" means match "a" OR "b".

Regards!

...JRF...

[Dennis Handly]

>Peter: I recommend NOT using grep for this filtering

There is nothing wrong with grep and in fact that issue will appear in awk, sed or perl. (JRF fixed his.)

You are correct that in all cases you need to add the field delimiters to make sure you don't pick up substrings.

So a fix to my example would be:
$ grep -e "|jkhssdf " -e "|sadjfiouoas "

Though a grep -w may work with my original example.

[Peter Nikitka]

Hi,

remarks for Dennis:
I told, I recommend NOT using grep == when the whole field content is specified for filtering ==
In other cases a grep is prefectly okay.

In my awk solution, I did use a compare
... $2==ars[j] ...
an NOT a match operator: the latter would in deed nothing change to a grep solution.

The option '-w' is not available at HP-UX's grep, IMHO.

mfG Peter

[Dennis Handly]

>recommend NOT using grep == when the whole field content is specified for filtering

This will work if you anchor it with the delimiter. (If you remember to do so, as you correctly pointed out.)

>The option '-w' is not available at HP-UX's grep

It is there in 11.11.