Automating changes in text files

[unixnewbie]

Hi alll,

I am new to unix,partcularly to shell scripting.I have an appplication which generates data files which I have to edit (using vi) and search for a line starting with a particular pattern (numeric or alphanumeric eg: "003" or "abc" 0r "a13" etc....).If i find that particular line I have check whether the next line starts with another particular pattern and if it is NOT I need to combine both the lines in to one (ie, no CR or "enter") in between.I tried with sed and some scripting but still no progress. I will be extremely happy if I get some script in which I can pass on the file name and the patterns to be searched and have the file changed (as I have a lot ).Please help .Thanks in advance

[Dennis Handly]

It would probably be simpler to do it in awk where you can actually program.
awk '
/^003/ || /^abc/ || /^a13/ {
printf "%s", $0
getline
if ($0 ~ "^next" )
print "" # newline
}
{ print } ' file

If you want the patterns to be variables:
awk -v p1="^003" -v p2="^abc" -v p3="^a13" -v p4="^next" '
$0 ~ p1 || $0 ~ p2 || $0 ~ p3 {
printf "%s", $0
getline
if ($0 ~ p4)
print "" # newline
}
{ print } ' file

And since you are using EREs, you can combine the N patterns:
awk -v p1="^(003)|(abc)|(a13)" -v p4="^next" '
$0 ~ p1 {
printf "%s", $0
getline
if ($0 ~ p4)
print "" # newline
}
{ print } ' file

[unixnewbie]

Dear dennis,

Thanks for that. But please clarify the following (may be too silly..)
1) Can I just type in all this to file and run that as a script ?
2) will the same work on Linux also ??
3) Can u suggest changes so that I can use it as a script and I can run it just by typing

./script

eg: ./scr myfile 000 003

and all the lines in myfile starting with 000 will combine with the next line if it is NOT starting with 003.(other lines no changes)

Thanks again,

[john korterman]

Hi,

try running the attached as you have suggested; it may or may not work.

regards,
John K.

[Hein van den Heuvel]

Here is a perl one-liner that works for the problem as stated

$ perl -e '$a=shift; $b=shift; while (<>) {if (/^$a/){chomp;print;$_=<>;$_="\n".$_ if /^$b/} print}
'


It user "^" to anchor the match pattern to the beginning of lines as specified.
Questions to ask yourself.

- What to do if there are mutliple starts
- Glue together just one next line or everything up to the next end
- What to do if there is no end in sight
- What to do if there extra ends
- Can the start line ever be a last line in a file.

Here is the perl script in slow motion:

$ perl -e ' ' # start perl with scrip in text
$a=shift; # pick up start pattern from command line
$b=shift; # pick up stop
while (<>) { # loop through input file
if (/^$a/) { # looks like the trigger?
{chomp; # prepare by removing new-line
print; # pritn first chunk already
$_=<>; # read next chunk
$_="\n".$_ # pre-pend a new-line only...
if /^$b/ # if end pattern seen
}
print # print the current line
} # loop


Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Sandman!]

Try the sed construct below:

# sed '/^000/N;/.*\n003/s/\n/ /p' file

~cheers

[Sandman!]

To execute sed construct from within a shell script do as shown:

# ./uxscript file 000 003

...contents of uxscript are listed below.

======================= uxscript =======================

#!/usr/bin/sh

sed "/^$2/N;/.*\n$3/!s/\n/ /p" $1

[Dennis Handly]

>1) Can I just type in all this to file and run that as a script?

Yes.

>2) will the same work on Linux also??

It should.

>3) Can you suggest changes so that I can use it as a script and I can run it just by typing
./script
eg: ./scr myfile 000 003

Note the current script just creates a new file, if you want to update it, you would need to rename it.

Ah, it is much easier than what I thought you needed.

#!/bin/sh
# Usage: $0 file pat1 pat2
awk -v p1="^$2" -v p2="^$3" '
$0 ~ p1 {
printf "%s", $0
getline
if ($0 ~ p2)
print "" # newline
}
{ print } ' $1

>Sandman: To execute sed construct from within a shell script do as shown:

Yes, I thought about using sed like that. I thought I had to look for N patterns at the same time.
Your script inserts a blank, mine doesn't.

[unixnewbie]

Sandman, The sed construct on execution shows the result but the file is not changing. If I redirect the o/p to another file ./sand myfile 000 003 > myfile1 I get what I want but the line replicates.ie, it identifies the lines, joines them perfect, but it will add one more line in myfile1 (a duplicate the same joined line).Any changes ?? ..Thanks..

Handly, I tried ur script in Linux (#!/bin/sh)(not yet on HP)
./denn newfile orgfile 000 003.
If I am not wrong the newfile should contain what I need.But it is still 0Kb.Please advise. Thank you

John and Hein,I will try your suggestions and get back. Thanks a lot....

[unixnewbie]

sandman sorry I forgot sed don't change the input file.