output file redirect from sed is empty when run from HPUX, Solaris7or AIX4.3

[Phuc Nguyen_1]

I am having some problem with my script, hopefully someone can help. When I ran the script in bourne shell in Redhat then everything works fine. But when I ran in bourne shell in HPUX11.x, Solaris7 or AIX4.3 the output from my second and third sed command are empty files.

---sed part of script---
case "$OPTION_IN" in
.
.
.
f|F) sed -e 's/^/'"$Z"'/' file1 | tr -s "\n" " " > out1;
sed -e 's/^/'"$Z1"'/' out1 > out2;
sed -e 's/$/'"$Z2"'/' out2 > out3;
;;
q|Q) break ;;
*) echo "Invalid choice." ;;
esac
done
exit
----------------------
where Z1="some_text_string"
Z2="another_string"

Any help is appreciated.
Thank you

[Mark Greene_1]

are Z1 and Z2 set in the script, or are they environment variables?

mark

[Phuc Nguyen_1]

Z1 and Z2 are variables define in the script.

[Miguel Covas]

I think that since you are stripping the new lines on the first output file you get a out1 file with no end line, just a bunch of characters in which sed
is unable to locate the end of line and therefore the whole file is end of line hence the odd behavior.

To stick to the proper behavior perhaps you would have to get a file containing $Z1$Z2 without no new line, but anyway, trying to substitute the end of the lines with something where there is no lines at all...

[Miguel Covas]

I supose that you want to change several empty lines by a single blank space. The behavior in Linux is OK, two or more new lines are collapsed in a single blank, but even in that case, perhaps you will not get what you want, since the bunch of new lines will include the previous not empty line and you will get two lines joined by a blank like in:
----------
line 11111


line 22222
line 33333
----------
becomes:
----------
line 11111 line 22222
line 33333
----------

[Mark Greene_1]

wait a minute, in your version of linux, is end-of-line a newline, or a newline and carriage return? this may be the difference.

HTH
Mark

[S.K. Chan]

The best approach to debug this is run your sed statement manually and check the output file after each run. I've tested your first sed statement with the "tr" and it works fine, it deleted all my blank lines. You might want to use sed also to do this ..
example ..

sed '/^$/d'

I find this very useful, you might want to use it .. it's up to you..
http://www-h.eng.cam.ac.uk/help/tpl/unix/sed.html

hope it helps ..

[Miguel Covas]

I've tried the shell on Mac OS X
and it works.I mean tr works as it should do.

Anyway, as it is used in the example it will not work (even behaving properly and replacing two or more \n with a single blank). The first sed inserts $Z at the beginning of each line, so no more groups of \n together.

tr -s '\012' < file1
works by the book, but again this is not just getting rid of the empty lines

[Phuc Nguyen_1]

To Mark Greene:
I don't think there are any carriage return character (\r) in the out1 file. Only newline character. How can I check to see if there are any carriage return character.

[Mark Greene_1]

>>How can I check to see if there are any carriage return character. <<

this will make the non-printable characters and control charactes visible:

cat -v {filename} |pg


HTH
mark

[Phuc Nguyen_1]

------ cat -v |pg ---------
Don't see any carriage return (\n) or new-line (\n) (But tab, new-line and form-feed are not suppose to show (from man cat).
Actual output of cat -v pout1 |pg.
----------------
string1 string2 string3
(EOF):
----------------

[A. Clay Stephenson]

Use od -c filename to show all the characters; cat -v as you have found out is not the weapon of choice.

[Mark Greene_1]

oops, my apologies for that, try od -c [file]

mark

[Phuc Nguyen_1]

------ od -c out1 -----
00000000 s t r i n g 1 = aa s t r i n g 2 = b
00000020 b
00000022
----------------------
don't see any \n or \r in out1 file.

In the file1 for the 1st sed, there are \n after each string ( s t r i n g 1 = a a \n)

[A. Clay Stephenson]

I see one potential big problem with your approach. If you are trying to replace LF's with a space and then feed that to more sed's, it is possible to end up with an EXTREMELY long input record which is too big to process.
I would also use an eval to get rid of the hokey quoting that you are being forced to do.

I think you will get a better answer if you post some actual input, actual target and replacement strings, and a sample of your desired output. We are all on the same page then.

You probably should also indicate if other approaches (awk, perl, ...) are acceptable.

[Phuc Nguyen_1]

Thank you to everyone that had posted.

--re: command too long to process--
for this task, argument length is not a concern, but your comments are greatly appreciated.

-- sed is just not working when file have no \n (on unix side). I will try other approach.

We can stop posting for this post at this point. Once again, Thank you.