sed help

[slydmin]

I can use some help using sed. I have a file that has a lot of trailing spaces.
I use vi to edit the file, and search and replace all the trailing space like so
:%s/[ ]*$//

Although, when I try to use sed, I am not able to edit the file at all. Here is how I am trying to do this

cat filename | sed -e 's/[ ]*$//'

This does not edit the file at all. What could I be doing wrong? Any help is greatly appreciated.
-S

[slydmin]

I think I may have a solution using tr

cat fielname | tr -d '\032* $'

Although, I need to confirm it with the QA ppl to make sure the file works for them.

[Alexander Chuzhoy]

You need to append g (to allow sed to fix all the matched cases and not just the first one). So it's:
sed -e 's/[ ]*$//g' filename

[Eric SAUBIGNAC]

Bonsoir -S,

It should work asis, that is without 'g' flag. Under HP-UX it works well.

As far as I know, regular expressions are satisfied with the largest number of characters that can match the expression. If it was not the case, and IMHO, '*' in a regular expression would have no sens. And '[ ]*$' clearly means : 0 or more spaces before end of line ! So it shoul work.

Don't know Linux as well as HP-UX but maybe there is a limitation on implementation of sed. Maybe there is a special option to tell 'sed' to use extended regular expressions ? I have this beahavior under HP-UX with 'grep' : if you don't use -E option, 'grep' doesn't treat correctly all regular expressions.

An other idea : '[]' is used to match a list of characters. As you have only one character, space, may be '[ ]' is not correctly interpreted ? So try this :

cat filename | sed -e 's/ *$//'

Hope this will help

Regards

Eric

[Stuart Browne]

As far as I'm aware, none of these commands 'edit' the file in place, but alter an output stream in the pipe.

If you want to edit a file in place, you can use:

perl -pi -e 's/\s+$//' filename

[Alexander Chuzhoy]

Good point Stuart.

slydmin, in order to actually update the content of the file you can redirect the output:
sed -e 's/[ ]*$//g' > newfile; mv newfile origfile;


make sure you have a backup of the original file.

[slydmin]

Alexander,
I was actually doing that, but did not mention in my original email.

sed -e 's/[ ]*$//'outputfile

I tried everyone's suggestion but none gave desired output.

I have a csv file, with say about 14 comma separated fields. The last field has a lot of spaces in it.
vim works like a charm (:%s![ ]*$!), although we are going to have a large number of files to process and manually doing it does not make sense.

So this attempt to use same regex with sed does not work. tr -d \032 works, although it removes white space from everywhere in the file.
I will copy 3 lines from the file and attache it herewith, although the data is changed.

I am stumped at this point.

[Eric SAUBIGNAC]

Bonsoir,

Just found this at http://pwet.fr/man/linux/commandes/sed :

-r, --regexp-extended
use extended regular expressions in the script

It sounds like like -E option for grep with HP-UX, no ?

Could you try to add -r option and tell us ?

Regards

Eric

[slydmin]

Eric,
I thought I had tried that (sed -r). But I gave it a try and it did not work.

What is interesting though, the file I uploaded , if that were to be manipulated with sed -r 's/[ ]*$//g' , I get the desired output but not on the original file.

The original file has a line count of about 25000. I tried splitting it up in to 5000 line files, but still am not having success.

I am going to try and use perl to see if I can use the expression posted to my OP (it removed all spaces, just like tr -d\032).

Thanks,
-S

[slydmin]

Actually I was wrong about the perl expression, it does not remove all white spaces like tr does.

It removes the white space(s) at the end of the line, but not all new lines are lost.

2nd line is not a continuation of the 1st.

Will try a few more things, but I stand corrected on the tr /perl comparison.