topic Re: Urgent: Help with sed command in Operating System - HP-UX

Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 00:32:28 GMT

i need help with sed command. i have to release this ASAP. this is because some user did cut and paste from word.

i have a file with following format.

field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4

All the fields are text field. Sometime some of the fields of a line contains "\r \n". this is where problem start.

if any filed contains "\r \n", cause that line to be devided into two lines.

for example field2 of first line contains
"\r \n" then it looks like that
we have five lines instead of 4 lines.

field1 field2
field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4

please help me. i need help to remove "\r \n" from a line. i tried following
comand but it didn't remove "\n".

sed 's/^V^M//g' filename > newfilename

please help me.

thanks

Re: Urgent: Help with sed command

Sudhakar.J — Fri, 08 Nov 2002 00:46:31 GMT

Hi Vivek,

You can try to remove these characters thru vi editor.
Open the file in vi (vi ),press colon(:) at colon prompt u enter this string to replace all the occurences of "\r\n" (1,$s/\\r\\n//g).It does global replacement of \r\n to null.

regds
sudhakar

Re: Urgent: Help with sed command

Sudhakar.J — Fri, 08 Nov 2002 00:47:17 GMT

Hi Vivek,

You can try to remove these characters thru vi editor.
Open the file in vi (vi ),press colon(:) at colon prompt u enter this string to replace all the occurences of "\r\n" to null(:1,$s/\\r\\n//g).It does global replacement of \r\n to null.

regds
sudhakar

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 01:06:59 GMT

Thanks Sudhakar,

sorry for any confusion. actually in the file the is no occurrence of "\r \n" itself. actually what I meant was, there is a carriage return and a line feed character in some of the fields.

if I run following commands

#1
> cat filename
field1 adgak field3

#2
> cat -vet filename
field1 adgak^m$ field3

Because of this it looks like that there are two lines as following.

field1 adgak
field3

if I run following commands it does not solve the problem.

Sed ???s/^M$//g??? filename >newfilename

Or

Sed ???s/^M//g??? filename >newfilename

Any help would be appreciated.

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 01:15:56 GMT

There could be one solution to my problem. i just thought of it. if we can replace every ocuurence of ^M$ with a special character which will cause current line and next line to be merge.

botom line , is there any way to merge tow line using unix script?

please help me.

Re: Urgent: Help with sed command

Darrell Allen — Fri, 08 Nov 2002 03:12:53 GMT

Try using the tr command. I'm not at a system right now but I think it would be something like:
tr -d "\015" file2

That should get rid of the ^M characters.

Darrell

Re: Urgent: Help with sed command

Chia-Wei — Fri, 08 Nov 2002 03:21:53 GMT

sed '
# look for "^M" at end of line
/^M$/{
#if found, read the next line
N
# merge line
s/^Mn//
}' filename

Re: Urgent: Help with sed command

curt larson_1 — Fri, 08 Nov 2002 03:26:24 GMT

to merge your two lines together try something like this

awk '{if (NF == 4 ) print $0;
else {
printf("%s",$0);
getline
print $0;
}
}'

Re: Urgent: Help with sed command

Christian Gebhardt — Fri, 08 Nov 2002 06:55:53 GMT

Hi
- to replace a ^M yor are right to use ^V^M (typing Cntr-V Cntr-M)
- to replace \n you can use standard unix-command cat | tr "\012" "x"

Chris

Re: Urgent: Help with sed command

Jean-Luc Oudart — Fri, 08 Nov 2002 09:09:30 GMT

tr will do the job :
oudartj:/home/oudartj/k1 $ more aa
asasdsdsa
sdsfdsfdsfds

oudartj:/home/oudartj/k1 $ od -xc aa
0000000 6173 6173 6473 6473 610d 0a73 6473 6664
a s a s d s d s a \r \n s d s f d
0000020 7366 6473 6664 730d 0a0a
s f d s f d s \r \n \n
0000032

oudartj:/home/oudartj/k1 $ cat aa | tr -d "\r" | od -xc
0000000 6173 6173 6473 6473 610a 7364 7366 6473
a s a s d s d s a \n s d s f d s
0000020 6664 7366 6473 0a0a
f d s f d s \n \n
0000030

Jean-Luc

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 18:01:34 GMT

Thanks,

Nothing helped me.

in my case i have to remove any occurance of "^M$" or "\r\n" from the middle of a line, not from the end of a line.

for example:

> cat -vet file

abcd efgh^M$
ijklmn ghkj 48

i have to remove "^M$" from the middle of line and output should be

abcd efgh ijklmn ghkj 48

so basically we have to merge next line with current line at each occurance of "^M$".

i don't have to delete line feed from the end of file.

please help me.

thanks

Re: Urgent: Help with sed command

John Meissner — Fri, 08 Nov 2002 18:30:30 GMT

you are getting the ^M at the end of your lines because of the way the file was transfered with ftp to your system. When you launch ftp type:
ascii
to change the transfer mode to text. This will fix the ^M and the end of each line

as for the /r or \n :
sed 's/\\r//g' file > file2
sed 's/\\n//g' file2 > file

Re: Urgent: Help with sed command

John Meissner — Fri, 08 Nov 2002 18:33:14 GMT

ok - ignore my previous post - i was skimming when i saw the ^M$ comment...

Re: Urgent: Help with sed command

James R. Ferguson — Fri, 08 Nov 2002 18:34:01 GMT

Hi:

Try:

# dos2ux myfile|awk '{if (NR%2==0) print X$0;X=$0};END{if (NR%2==1) print X}'

Regards!

...JRF...

Re: Urgent: Help with sed command

John Meissner — Fri, 08 Nov 2002 18:37:13 GMT

James - how do you get that good :)

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 18:57:44 GMT

Thanks James,

It seems that your solution may work but it still need some change.

let me explain again:

i have file name "vv2". if run following commands

#1

> cat -vet vv2

NASSCAR on Fox. ^M$
^I ^I^I^INASCAR^I^IY^I48$
afsdfsdfsd$
sfs$

explination: it looks like that there are four lines. in fact there are only three line originaly. at the time of data entry user did cut & paste. because of it there is a carriage ruturn after " Fox. " because of this first line is broken into two lines.

what i want is , just remove any occurance "^M$" from the middle of the line so that it will look line as follows.

> cat -vet vv2

NASSCAR on Fox. NASCAR^I^IY^I48$
afsdfsdfsd$
sfs$

so as per your suggestion i ran following command
> dos2ux vv2 |awk '{if (NR%2==0) print X$0;X=$0};END{if (NR%2==1) print X}' > vv3

> cat -vet vv3

NASSCAR on Fox. ^I ^I^I^INASCAR^I^IY^I48$
afsdfsdfsdsfs$

the only problem i see is that last two lines are also merged.

please help me.

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 19:52:09 GMT

James Please do reply!!!!

Re: Urgent: Help with sed command

Vivek_3 — Fri, 08 Nov 2002 23:21:20 GMT

Hi,

Still trying to solve my problem.

at following link i find a solution but it is not working.

http://www.student.northpark.edu/pemente/sed/sed1line.txt

# if a line ends with a backslash, append the next line to it

sed -e :a -e '/\\$/N; s/\\\n//; ta'

in my case is a line end with ^M$ then remove the ^M$ and append the next line to it. so i tried following and its not working.

sed -e :a -e '/\\^M$; s/\\\n//; ta' filename

actually i know i didn't do anything to remove "^M$" in the above command. it is giving me following error message.

$; s/\\\n//; is not a recognized function.

Please help me!!!

Re: Urgent: Help with sed command

Gary Seibak — Sat, 09 Nov 2002 06:52:15 GMT

If u are just trying to remove ^M's from the file then just vi the file and do
"%s/^V^M//g" the control key is held down while pressing "V" then "M" and then let go of the control key

Re: Urgent: Help with sed command

Vivek_3 — Mon, 11 Nov 2002 17:23:24 GMT

Thanks to all.