Urgent: Help with sed command

[Vivek_3]

i need help with sed command. i have to release this ASAP. this is because some user did cut and paste from word.

i have a file with following format.

field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4

All the fields are text field. Sometime some of the fields of a line contains "\r \n". this is where problem start.

if any filed contains "\r \n", cause that line to be devided into two lines.

for example field2 of first line contains
"\r \n" then it looks like that
we have five lines instead of 4 lines.

field1 field2
field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4
field1 field2 field3 field4

please help me. i need help to remove "\r \n" from a line. i tried following
comand but it didn't remove "\n".

sed 's/^V^M//g' filename > newfilename

please help me.

thanks

[Sudhakar.J]

Hi Vivek,

You can try to remove these characters thru vi editor.
Open the file in vi (vi ),press colon(:) at colon prompt u enter this string to replace all the occurences of "\r\n" (1,$s/\\r\\n//g).It does global replacement of \r\n to null.

regds
sudhakar

[Sudhakar.J]

Hi Vivek,

You can try to remove these characters thru vi editor.
Open the file in vi (vi ),press colon(:) at colon prompt u enter this string to replace all the occurences of "\r\n" to null(:1,$s/\\r\\n//g).It does global replacement of \r\n to null.

regds
sudhakar

[Vivek_3]

Thanks Sudhakar,

sorry for any confusion. actually in the file the is no occurrence of "\r \n" itself. actually what I meant was, there is a carriage return and a line feed character in some of the fields.

if I run following commands

#1
> cat filename
field1 adgak field3

#2
> cat -vet filename
field1 adgak^m$ field3

Because of this it looks like that there are two lines as following.

field1 adgak
field3


if I run following commands it does not solve the problem.

Sed ???s/^M$//g??? filename >newfilename

Or

Sed ???s/^M//g??? filename >newfilename


Any help would be appreciated.

[Vivek_3]

There could be one solution to my problem. i just thought of it. if we can replace every ocuurence of ^M$ with a special character which will cause current line and next line to be merge.

botom line , is there any way to merge tow line using unix script?

please help me.

[Darrell Allen]

Try using the tr command. I'm not at a system right now but I think it would be something like:
tr -d "\015" file2

That should get rid of the ^M characters.

Darrell

[Chia-Wei]

sed '
# look for "^M" at end of line
/^M$/{
#if found, read the next line
N
# merge line
s/^Mn//
}' filename

[curt larson_1]

to merge your two lines together try something like this

awk '{if (NF == 4 ) print $0;
else {
printf("%s",$0);
getline
print $0;
}
}'

[Christian Gebhardt]

Hi
- to replace a ^M yor are right to use ^V^M (typing Cntr-V Cntr-M)
- to replace \n you can use standard unix-command cat | tr "\012" "x"

Chris

[Jean-Luc Oudart]

tr will do the job :
oudartj:/home/oudartj/k1 $ more aa
asasdsdsa
sdsfdsfdsfds

oudartj:/home/oudartj/k1 $ od -xc aa
0000000 6173 6173 6473 6473 610d 0a73 6473 6664
a s a s d s d s a \r \n s d s f d
0000020 7366 6473 6664 730d 0a0a
s f d s f d s \r \n \n
0000032

oudartj:/home/oudartj/k1 $ cat aa | tr -d "\r" | od -xc
0000000 6173 6173 6473 6473 610a 7364 7366 6473
a s a s d s d s a \n s d s f d s
0000020 6664 7366 6473 0a0a
f d s f d s \n \n
0000030


Jean-Luc

[Vivek_3]

Thanks,

Nothing helped me.

in my case i have to remove any occurance of "^M$" or "\r\n" from the middle of a line, not from the end of a line.

for example:

> cat -vet file

abcd efgh^M$
ijklmn ghkj 48

i have to remove "^M$" from the middle of line and output should be

abcd efgh ijklmn ghkj 48

so basically we have to merge next line with current line at each occurance of "^M$".

i don't have to delete line feed from the end of file.

please help me.

thanks

[John Meissner]

you are getting the ^M at the end of your lines because of the way the file was transfered with ftp to your system. When you launch ftp type:
ascii
to change the transfer mode to text. This will fix the ^M and the end of each line

as for the /r or \n :
sed 's/\\r//g' file > file2
sed 's/\\n//g' file2 > file

[John Meissner]

ok - ignore my previous post - i was skimming when i saw the ^M$ comment...

[James R. Ferguson]

Hi:

Try:

# dos2ux myfile|awk '{if (NR%2==0) print X$0;X=$0};END{if (NR%2==1) print X}'

Regards!

...JRF...

[John Meissner]

James - how do you get that good :)

[Vivek_3]

Thanks James,

It seems that your solution may work but it still need some change.

let me explain again:

i have file name "vv2". if run following commands

#1

> cat -vet vv2

NASSCAR on Fox. ^M$
^I ^I^I^INASCAR^I^IY^I48$
afsdfsdfsd$
sfs$

explination: it looks like that there are four lines. in fact there are only three line originaly. at the time of data entry user did cut & paste. because of it there is a carriage ruturn after " Fox. " because of this first line is broken into two lines.

what i want is , just remove any occurance "^M$" from the middle of the line so that it will look line as follows.

> cat -vet vv2

NASSCAR on Fox. NASCAR^I^IY^I48$
afsdfsdfsd$
sfs$

so as per your suggestion i ran following command
> dos2ux vv2 |awk '{if (NR%2==0) print X$0;X=$0};END{if (NR%2==1) print X}' > vv3

> cat -vet vv3

NASSCAR on Fox. ^I ^I^I^INASCAR^I^IY^I48$
afsdfsdfsdsfs$



the only problem i see is that last two lines are also merged.

please help me.

[Vivek_3]

James Please do reply!!!!

[Vivek_3]

Hi,

Still trying to solve my problem.

at following link i find a solution but it is not working.

http://www.student.northpark.edu/pemente/sed/sed1line.txt


# if a line ends with a backslash, append the next line to it

sed -e :a -e '/\\$/N; s/\\\n//; ta'


in my case is a line end with ^M$ then remove the ^M$ and append the next line to it. so i tried following and its not working.

sed -e :a -e '/\\^M$; s/\\\n//; ta' filename

actually i know i didn't do anything to remove "^M$" in the above command. it is giving me following error message.

$; s/\\\n//; is not a recognized function.


Please help me!!!

[Gary Seibak]

If u are just trying to remove ^M's from the file then just vi the file and do
"%s/^V^M//g" the control key is held down while pressing "V" then "M" and then let go of the control key

[Vivek_3]

Thanks to all.