topic Re: Sed?? in Operating System - Linux

Sed??

Rinky — Thu, 30 Aug 2007 02:21:08 GMT

Hi,

I have a comma seperated file. The second field in it is a description field.
But has new line character within the field.
I want the new line character to be removed so that the description field comes in the same line.

For eg: In the file if it is:
10000,"Today
is a holiday"

I want it to be
10000,"Today is a holiday"

Please help..

Re: Sed??

Dennis Handly — Thu, 30 Aug 2007 03:22:20 GMT

You'll have to explain in more detail. Are ALL lines broken up? Or only some? Or do you need to look for a mismatched "?
Or any line with a " is mismatched?

If the lines are always broken in pairs, then you could possibly use sed. But if you have to check, you need awk:
awk '
{
inx = index($0, "\"")
if (inx != 0) {
save = $0
getline
print save, $0
next
}
print
}' file

Re: Sed??

Rinky — Thu, 30 Aug 2007 03:35:33 GMT

Ok..
Only some lines have this problem. Probably a search for mismathced " will do in a line..
how do we accomplish tat?

Re: Sed??

Marcin O. — Thu, 30 Aug 2007 03:52:12 GMT

Hello

If all lines are broken use edit file with vi editor, enter :g/./join

Regards
Marcin

Re: Sed??

Dennis Handly — Thu, 30 Aug 2007 04:05:41 GMT

>Probably a search for mismathced " will do in a line.. how do we accomplish that?

This assumes that the second field has quotes, not the first nor any other.
awk '
{
inx = index($0, "\"")
if (inx != 0) {
rest = substr($0, inx + 1) # after quote
inx = index(rest, "\"")
if (inx == 0) { # mismatched
save = $0
getline
print save, $0 #join lines
next
}
}
print
}' file

I suppose if you had an arbitrarily complex case you could set the delimiter to ", then if the number of fields was even, then join lines.

Re: Sed??

Marcin O. — Thu, 30 Aug 2007 04:11:48 GMT

If all lines start from numeric and only broken line have to be processed you may use in vi

:v/^[0-9]/-1j

and only broken line will be connected

Regards
Marcin

Re: Sed??

Dennis Handly — Thu, 30 Aug 2007 04:57:18 GMT

>Marcin may use in vi
:v/^[0-9]/-1j
>and only broken line will be connected

I'm at a loss on how to get that to work and why it would work.
It seems you are executing the ex(1) visual command from the vi ":" prompt??

Or did you mean "g" instead of "v"?
Ah, you're using a foreign devil vim instead of vi. And "v" did work but I still don't know why.

Re: Sed??

Marcin O. — Thu, 30 Aug 2007 05:17:07 GMT

Hi Dennis

I using standard hp-ux vi.
And there is no mistake with "v", "v" - inverts the search, ex. ":v/./d" = ":g/^$/d" (delete all empty lines)

Regards
Marcin

Re: Sed??

Dennis Handly — Thu, 30 Aug 2007 06:01:08 GMT

>Marcin: I using standard hp-ux vi.

Hmm, it works now. I don't know what I did to get it to fail. ??

>And there is no mistake with "v", "v" - inverts the search,

You can only find it in ex(1) if you know v is related to g or global. They don't list it alphabetically under v. And don't have flashing red lights. :-(
It is in the Abbreviations box.

Thanks for the info. Though if you used g!, it would have been easier to figure it out.

Re: Sed??

Marcin O. — Thu, 30 Aug 2007 06:18:48 GMT

Rinky, please assign points.

Re: Sed??

Hein van den Heuvel — Thu, 30 Aug 2007 07:10:51 GMT

If not resolved yet, then this perl should do it. It just checks field 2 as written:

perl -pe 'if (!/^.*?,".*?"/ && !eof) { chomp; $_ .=<> }' old > new

To check any field (including 2), leave the begin-of-line anchor of:

perl -pe 'if (!/,".*?"/ && !eof) { chomp; $_ .=<> }' old > new

fwiw,
Hein.

Re: Sed??

James R. Ferguson — Thu, 30 Aug 2007 07:50:53 GMT

Hi Rinky:

Another Perl:

# perl -ni.old -e 'if (!m/"$/) {chomp;print $_." "} else {print}' file

This will also update your file in situ (in place) leaving an unmodified copy as "file.old".

Regards!

...JRF...

Re: Sed??

Sandman! — Thu, 30 Aug 2007 08:47:55 GMT

Try the sed construct below:

# sed 'N;s/\n/ /p' file

Re: Sed??

Dennis Handly — Thu, 30 Aug 2007 16:40:48 GMT

>Sandman: Try the sed construct below:
# sed 'N;s/\n/ /p' file

This was rejected as not every line had the problem.

Re: Sed??

Sandman! — Thu, 30 Aug 2007 16:52:40 GMT

>Only some lines have this problem. Probably a search for mismathced " will
>do in a line..

Good point Dennis...hadn't read the entire thread until now and based on requirements try the construct below which might be what Rinky's looking for:

# sed '/"$/!N;s/\n/ /p' file

Re: Sed??

Rinky — Thu, 30 Aug 2007 22:14:34 GMT

Thanks everybody:)

Re: Sed??

Rinky — Thu, 30 Aug 2007 22:15:23 GMT

Got wat i wanted!