Delete rogue carriage return (blank space)

[Declan Heerey]

I have a file (test.txt) with the following data in it

test.txt

here
there
everywhere






I use the following sed command to delete the blank lines

sed -i -e :a -e '/^\n*$/{$d;N;ba' -e '}' test.txt

but i am left with

here
there
everywhere


Can anyone advise how to get rid of the final blank line or last carriage return (i've tried loads of methods to no avail)

Thanks in advance

Declan

[spex]

Why not use grep instead of sed?

$ grep -v '^$' test.txt
here
there
everywhere

PCS

[Declan Heerey]

Thanks but that still leaves a blank line in the file or a new file (if created) i.e.

grep -v '^$' test.txt > test2.txt

still leaves me with

here
there
everywhere


:o(

[James R. Ferguson]

Hi Declan:

# perl -ni.old -e 'print unless /^\s*$/' file

This will update "file" in-place keeping a backup copy as "file.old".

Regards!

...JRF...

[spex]

$ cat test.txt
here
there
everywhere





$ grep -v '^$' test.txt > test2.txt
$ cat test2.txt
here
there
everywhere
$ xd -x test2.txt
0000000 6865 7265 0a74 6865 7265 0a65 7665 7279
0000010 7768 6572 650a
0000016

There is no extra newline.

[Hein van den Heuvel]

Declan,


Looks like the is really a !
That would explain all.

James addresses that with the \s* in the reg expr.

Spex addresses that with the dump, to 'see' the tab(s) (x08) or space(s) (x20) if there are any.

I would use #xd -b test.txt for that.

You can also try to make the blanks visible with something silly like:

awk '{print $0 "*"}' x.awk test.txt


fwiw,
Hein.

[Peter Nikitka · Accepted solution]

Hi,

... and if you wnt to use 'awk' for skipping empty lines:

awk NF test.txt

mfG Peter

[James R. Ferguson]

Hi Peter:

Wow, your 'awk' solution deserves a 10+ !!!

I *really* like that. Elegant, to say the very least. Thanks!

Regards!

...JRF...

[Sandman!]

As Hein said the file contains lines with blanks not to be confused with empty lines. Verify by displaying the invisible characters in it as:

# cat -vet file

Based on the above output change the sed cmd so that it matches the regexp to a tee or you can try the one below to see if it works:

# sed -n '/^ *$/!p' file

>Peter: awesome awk construct very compact and elegant.

~cheers

[Bill Hassell]

And for yet a another answer:

man rmnl
man ssp

but as with previous solutions, an empty line (rmnl and ssp work well) is not the same as a line of blanks or white space. Hats off to the awk solution.

[Dennis Handly]

>Peter: awk NF test.txt
>JRF: your 'awk' solution deserves a 10+
>Bill: Hats off to the awk solution.

The only problem will be that you need to have a comment that tells maintainers what it really does. :-)

[Peter Nikitka]

OK, Dennis (and others),
seeing the request, I will comment
awk NF test.txt

1) a single awk commandline consists of
statement {action}
which are both optional. If 'statement' evaluates to TRUE 'action' is performed.
Missing statement means: TRUE
Missing action means: print the line

2) the internal awk variable NF contains always the number of fields of the current line (no matter if we are in a statement or an action)

3) we have now no action in the above awk-commandline => print line

4) lines containing only whitespace do not contain any fields, so NF is zero => the statement evaluates to FALSE

1)-4) ==> the awk commandline
NF
selects all non-empty lines only.

mfG Peter

PS: JRF, I was really proud when I saw your comment!

[Dennis Handly]

>Peter: Dennis (and others), seeing the request, I will comment

I figured it out. :-)

>1) a single awk commandline consists of
statement {action}

Actually awk(1) says: pattern {action}
http://docs.hp.com/en/B2355-60130/awk.1.html

[leelangco_1]

tr -s ["\n"] newtwst.txt

[Dennis Handly]

>leelangco: tr -s ["\n"] < test.txt

This fails to delete lines with actual spaces.
Using character classes, we can use the following:
$ sed -e '/^[[:space:]]*$/d' test.txt
$ grep -v '^[[:space:]]*$' test.txt