topic awk - searching/replacing control characters in Operating System - HP-UX

awk - searching/replacing control characters

Lemaitre — Mon, 02 Dec 2002 10:52:24 GMT

People,

I am trying to delete a load of control characters from a file that has acquired a load of new text via conversion problems. I.e. the pipe delimitted file looks like :

aaaa|bbbb|cccc^M
dddd|eeee|ffff^M
gggg|hhhh|iiii^M

what I am trying to do is convert it to

aaaa|bbbb|cccc|eor
dddd|eeee|ffff|eor
gggg|hhhh|iiii|eor

I have tried the following :

cat filename | awk -F"|" '
{printf("%s|%s|%s|eor",$1,$2,$3)}'

but this gives :

aaaa|bbbb|cccc^M|eor
dddd|eeee|ffff^M|eor
gggg|hhhh|iiii^M|eor

With my limited knowledge of awk, could someone suggest a script that would do this, or better still, identify how awk (if it can that is), searches for this type of character (^M indicates a newline I think)

thanks to you all!
John

Re: awk - searching/replacing control characters

RAC_1 — Mon, 02 Dec 2002 10:59:05 GMT

do this
cat test | awk -F"|" {printf("%s|%s|eor\n",$1,$2,$3)}'

Hope this helps.

Re: awk - searching/replacing control characters

Chris Wilshaw — Mon, 02 Dec 2002 11:04:37 GMT

As an alternative, using sed you could try

cat | sed s/.$/\|eor/g

This will replace any character followed by the end of line marker ($) with |eor

Re: awk - searching/replacing control characters

Jean-Louis Phelix — Mon, 02 Dec 2002 11:05:07 GMT

hi,

You should perhaps try to delete these ^m before reformatting using either sed ot dos2ux.

dos2ux filename | awk '{printf "%s|eor\", $0)}

or the most simple :

sed 's/^$.*$^M/&|eor/' filename

use ^V^M to avoid ^M interpretation.

Regards.

Re: awk - searching/replacing control characters

u856100 — Mon, 02 Dec 2002 11:07:26 GMT

Hi,

I missed the \n in my previous response!

I have been using :

cat filename | awk -F"|" '
{printf("%s|%s|%s|eor\n",$1,$2,$3)}'

The awk script you mentioned, I think would miss out a column I need (cccc - eeee and iiii)

thanks
John

P.s. I don't know why, but I have just logged in as myself, and got logged in as some person called Lemaitre!

whats going on?????

Re: awk - searching/replacing control characters

Jean-Louis Phelix — Mon, 02 Dec 2002 11:07:55 GMT

Hi Chris,

Clever '.$' :^) , not to deal with ^M interpretation. I like it !

Jean-Louis.

Re: awk - searching/replacing control characters

u856100 — Mon, 02 Dec 2002 11:08:17 GMT

people!

in addition to the fact that I have somehow logged in as another person (le-maitre).... I CANT assign points!!!

NOOOO!

Re: awk - searching/replacing control characters

H.Merijn Brand (procura — Mon, 02 Dec 2002 11:44:54 GMT

# perl -pi -e's/[\r\n]+$/|eor\n/' filename

Re: awk - searching/replacing control characters

John Meissner — Mon, 02 Dec 2002 12:32:23 GMT

did you ftp this file to your server? if you did ftp it... did you use bin or ascii mode? sometime i noted that using the wrong mode will add those characters to the end of each line. Try ftp'ing the file again and see if you get the same results.

If you didn't ftp the file then just follow the suggestions above.

Re: awk - searching/replacing control characters

V. Nyga — Mon, 02 Dec 2002 14:08:59 GMT

Hi,

only an additional info:
this ^M is the UX-interpretation of a DOS carrige return.
To eliminate this character you only need a dos2ux!
dos2ux file>file2
and the ^M will disappear!

Regards
Volkmar