topic Re: Using sed with wildcards in Operating System - Linux

Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 10:37:23 GMT

Hi,

I've a need to substitute some string in a line where I do not what the string is currently, only what I need it to be. I've been trying, with no luck to use sed so I'm turning to the group. A sample line follows:

TXT=getsearch ("THISISIT", "This is always the same: ",3);

Now I have a need to replace THISISIT which I will not know with something else that I do know. The TXT= is unique so I could key off that to get the target line I want to modify. I've tried a number of things, but I'm either not qualifying any lines or I get something like TXT=getsearch ("THISISIT"TXT=getsearch ("THISISIT", "This is always the same: ",3);

Hope I've described it well enough.

Re: Using sed with wildcards

Pat Lieberg — Fri, 26 Aug 2005 11:05:35 GMT

How about:

sed 's/"[^"]*"/"replacement string"/' filename

Re: Using sed with wildcards

curt larson_1 — Fri, 26 Aug 2005 11:07:21 GMT

how about trying something in awk

awk -F"\"" '
/TXT=/ {printf("%s\"%s",$1,"yourtext");
for ( i=3;iprint $NF;
}'

sed could be something like this

sed -e 's/^$TXT=getsearch ("$[:alnum:]*$", "This is always.*$$/\1yourtext\2/

Re: Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 11:25:38 GMT

Pat,

It worked to a point. It changed any occurance of that to THISISIT. Since there are other instances of it in the file they were all changed to THISISIT.

What I did do, taking your suggestion is add the qualifier PW1 to the sed like:

sed '/PW1=/s/"[^"]*"/"replacement string"/' filename.

It changed the one line with PW1.

Thanks a bunch.

Re: Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 11:32:06 GMT

Sorry Pat, the correct line was

sed '/TXT=/s/"[^"]*"/"replacement string"/' filename.

Not PW1. PW1 was still in my cut/paste buffer.

Re: Using sed with wildcards

TwoProc — Fri, 26 Aug 2005 11:36:02 GMT

Jeff,

I've handled this same problem in the past by generating a sed command file on each occurence of the new "unknown" with a replacement command for sed.
I generate that file into /var/tmp/sed.$$
and then run my sed against that new command file with the "-f" option for sed.

Probably a little clunky, but it has allowed me to get the job done everytime, as it is easy to build rather complex multiple sed command sets into a file and use it a base to change many files using the one command file.

Re: Using sed with wildcards

Pat Lieberg — Fri, 26 Aug 2005 11:53:14 GMT

oops, good catch. I overlooked that compeltely.

Glad I at least got you on the right track.

Re: Using sed with wildcards

James R. Ferguson — Fri, 26 Aug 2005 11:53:19 GMT

HI Jeff:

How about:

perl -ne 's!(.+),(.+),(.+)!"newstring",$2,$3!;print' yourfile

...where "newstring" is the replacement for the THISISIT.

Regards!

...JRF...

Re: Using sed with wildcards

Hein van den Heuvel — Fri, 26 Aug 2005 12:02:29 GMT

perl -pe 's/"$1"/"new text"/ if /^TXT.*\("(\w+)"/' old > new

$1 gets the (\w+) on a match.

The "" around $1 and new word in the substitute are belts and suspenders.... in case the the old text is 'TXT' or such.

The $1 can be remembered eg: $thisisit=$1
and used for further substitutes in different context in the file.

perl -pe '$thisisit = $1 if /^TXT.*\("(\w+)"/;
s/$thisisit/new text/ if defined ($thisisit)' old > new

Check out the perl -i option for 'in place' edits.

Hein.

Re: Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 13:30:59 GMT

James,

Thanks, but your solution changed every occurance and chopped off any text preceeding what I changed.

ie: every changed line started with ^"THISISIT

Trouble is, unlike sed, my breadth of knowledge in perl is not that much so I can make modifications to your suggestion.

Re: Using sed with wildcards

Sandman! — Fri, 26 Aug 2005 14:43:04 GMT

Jeff,

Try this with awk:

# awk '{gsub("THISISIT","replacement_string"); print $0}' input_file

regards!

Re: Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 14:51:51 GMT

Hi Sandman,

You're assuming I'll know what THISISIT will be, but I wont. From the looks of the awk, it will also change every occurance which I might not want.

Re: Using sed with wildcards

Sandman! — Fri, 26 Aug 2005 15:14:18 GMT

Thanks for the clarification. So you don't know what the substitution string looks like. Are you going to input that from the keyboard e.g. with a read statement?

Also if you don't want to change all occurances, then how many do you want to change? An example would certainly help.

regards!

Re: Using sed with wildcards

Jeff Gyurko — Fri, 26 Aug 2005 15:31:27 GMT

Hi Sandman,

I'll know what I'm changing the string TO, but I don't know what the current string looks like to key off it. I can key off the beginning of the line where ^TXT is since that will be unique, but I'll only want it on the one matched line. The example is like the original question.

ie: ^TXT=getsearch ("THISISIT", "This is always the same: ",3);

I'll want to change THISISIT, but I won't know what THISISIT is. I'll only know THISISWHATIWANT. It 'should' only occur once which is why I can key off of ^TXT to get the line I want to operate on.

Re: Using sed with wildcards

Mel Burslan — Fri, 26 Aug 2005 15:48:25 GMT

Jeff,

first off, I am notg a perl expert
sencondly, this may require more than a single line of commands, so it may entail being an external function in script form you need to call on a single line where you need it.

After saying that, if this might be of interest to you, read-on.

Assuming the beginning of the string being constant, i.e.,

TXT=getsearch ("

part never changes (this is what I understand from your original post), you can determine the position of first " sign, which is location-1 of your target string start point, using index function in perl something like

#!/usr/bin/perl
$MYSTRING=;
first_pos=index ($MYSTRING, '"');
$start4second=$first_pos+1;
second_pos=index ($MYSTING, '"', $start4second);
$length=$second_pos-$first_pos;
str2replace=substr($MYSTRING, $first_pos, $length);
print $str2replace;

let's say you put this little code snippet in a file called extractor.pl and made it executable

now, if the THISISIT going to be the same all thru one file, you can go with something like this

SourceString=`head -1 myfile` #assuming one string per line in this myfile

ReplString="Something I Know"

hunt_string=`echo $SourceString | ./extractor.pl`

cat myfile | sed -e "1,1s/$hunt_string/$repl_string/" > myoutfile

whereas, if the THISISIT different on each line, you can go with something like this

ReplString="Something I Know"

cat myfile | while read SourceString

do

hunt_string=`echo $SourceString | ./extractor.pl`

echo $SourceString | sed -e "1,1s/$hunt_string/$repl_string/" >> myoutfile

done

DISCLAIMERS:
I need to emphasize I am !NOT! a perl expert hence no clue on interpretation of double and single quotes. As I used double quotes as delimiters for your hunt_string, I enclosed them in single quotes but it may not work. There are guru level people on PERL matters among us here and I am sure one can tell you the correct syntax if this does not do what you need.

I know it is not the most elaborate solution but it might work. Take it as more of guideline than its syntax value.

Good luck.

Re: Using sed with wildcards

Patrick Wallek — Fri, 26 Aug 2005 15:49:32 GMT

IF the string "TXT=getsearch " only occurrs in the file once, you should be able to do something like this little awk script:

awk -F , '{ \
if (substr($1,1,13) == "TXT=getsearch") \
$1="TXT=getsearch (\"YOURNEWTEXTHERE\"" \
print $1","$2","$3 \
}' yourfile

This is assuming that your example is accurate and there are only 3 fields if you use the comma as a delimiter.

Re: Using sed with wildcards

Sandman! — Fri, 26 Aug 2005 16:11:32 GMT

Jeff,

Here's what I was able to do with sed based on your explanation of the problem at hand. This uses the hold buffers within sed to reproduce substitution patterns for later replay:

# sed '/^TXT/s/$get.* ("$$.*$$", "This is.*$/\1replacement_string\3/'

hope it helps!!!

Re: Using sed with wildcards

Sandman! — Fri, 26 Aug 2005 16:31:25 GMT

Actually you can compact the sed command based on the fact that the target line is unique and it begins with "TXT=" i.e.

# sed '/^TXT=/s/$.* ("$$.*$$", .*$/\1replacement\3/'

regards!

Re: Using sed with wildcards

James R. Ferguson — Fri, 26 Aug 2005 19:16:10 GMT

Hi (again) Jeff:

Try this. It limits replacements to lines beginning with "TXT=" and replaces the first quoted string found. Thus, in you example. "THISISIT" or "WHATEVER" would be replaced with "new_string":

I choose perl to exploit its exceedingly powerful regular expression engine.

# perl -pe 's!(^TXT=.*?")(.*?")!\1new_string"!' your_file

Regards!

...JRF...

Re: Using sed with wildcards

Bob_Vance — Sat, 27 Aug 2005 14:08:07 GMT

I like Pat's solution (with the final tweak of matching "^TXT") -- simple one-liner

BTW,
I usually generalize a bit, allowing for variable whitespace.
So, I would have written the 'sed' script as:

'/^[[:space:]]*TXT[[:space:]]*=/s/"[^"]*"/"replacement string"/'

This would allow for any additional whitespace at start of line or around the "=".

Note also, that it would have been not as elegant if we were trying to change the *second* quoted string and both strings were variable ;>)

my $.02
bv