Insert an incremented number via VI or sed

[OFC_EDM]

I have a text file over 1 million lines long.
I'll spare the long details and give a normalized view of the problem.

I want to use VI to edit the file.
I want to insert text on every line.
As part of that text I want an incremented number.

To illustrate I'll use a very simplistic scenario using 3 lines of text.

There are some sheep
They left their leavings on my lawn
We'll be having lamb stew tonight

Within VI I invoke the CMD prompt using :
Then I could insert text on each line
: 1,$s/^/TESTING //g

Would result in
TESTING There are some sheep
TESTING They left their leavings on my lawn
TESTING We'll be having lamb stew tonight

But I want to modify the search and replace string to add an incremented number. Maybe so the results look like this

100 TESTING There are some sheep
101 TESTING They left their leavings on my lawn
102 TESTING We'll be having lamb stew tonight

How do I get that incremented number into the replace string?

I could do this via a script...really easily. But there are reasons why this can't be used as an option. Either via VI or sed command line using
"sed -e 'command'"

Cheers

[Pete Randall]

sed 's/^/TESTING /' yourfilename |sed '/./=' | sed '/./N; s/\n/ /'


Pete

[James R. Ferguson]

Hi:

One way:

# perl -pe '$i++;s/^(.*)/$i TESTING \1/g' file

...to update the file "inplace" do:

# perl -pi.old -e '$i++;s/^(.*)/$i TESTING \1/g' file

This preserves a backup copy as "*.old" too.

Regards!

...JRF...

[Hein van den Heuvel]

>> I could do this via a script...really easily. But there are reasons why this can't be used as an option.

It would help to explain those reasons.
Do you consider an AWK or PERL solution a script?

>> As part of that text I want an incremented number.

What is the base of the number? Any specicial formatting reaquirements?
Most importantly... will the line number itself suffice?

JRF already shows one of many Perl solutions.

Here is an AWK based solution sticking close to your example

$ awk -v x=100 '{print x++, "TESTING", $0}' old > new
$cat new
100 TESTING There are some sheep
101 TESTING They left their leavings on my lawn
102 TESTING We'll be having lamb stew tonight

You may want to use printf for more output formatting control.
And you may need to define whether the text to be inserted will always go at the front of the line or not.

Good luck!
Hein.

[Eric SAUBIGNAC · Accepted solution]

Bonjour,

"I could do this via a script...really easily." So I will not reply with a new script and there are already 3 answers with 3 valuable scripts.

I want to focus on your "But there are reasons why this can't be used as an option. Either via VI ...".

Have you considered "!" command in vi ? "!" is something like a pipe that will take all text between cursor line position and a marked line, send it to an external command, then replace it with output of command.

To illustrate it, take Peter's answer (the one from wich I learnt the more about sed ;-) and modify it slightly to sqeeze file name in input.

vi your file
Go to line 1 --> 1G
Press --> !
Press --> G
Introcuce Peters's modified command --> sed 's/^/TESTING /' |sed '/./=' | sed '/./N; s/\n/ /'
Then validate. Good job, no ? With this method you can do many things like sorting, merging, ... directly from vi

Anyway, as previously said : "It would help to explain those reasons". For example why do you need line numbering ? Just for editing puposes ? So consider vi command ":set number"

Eric

[OFC_EDM]

Eric that's awesome!!
But that example only does the one line.

How do I get it to do lines 1 to say 1000?

I can feel 10 pts heading your way :)

[Eric SAUBIGNAC]

I don't understand what you mean by "only does the one line".

Do you mean that it works only on the first line ? It should not since 'G' after the '!' means end of file.

Do you mean it works only on the last line ? You must stand on the first line before issuing '!' command. '1G' should bring you to the first line. Are you sure you were on line 1 when you pressed '!' ?

Now in a more general way if you want to work only between specified lines, for example line 25 to 1292.

Go to line 1292 --> 1292G
Mark this line --> ma
Go to line 25 --> 25G
press --> !
press --> 'a (I really mean quote then a)
type command --> sed 's/^/TESTING /' |sed '/./=' | sed '/./N; s/\n/ /'
then validate

Regards

Eric

[Pete Randall]

I just realized that I forgot to attach my SED document that I borrowed from. There are a few different options for numbering, like left aligned vs right aligned, skipping blank lines, etc., etc.


Pete

[OFC_EDM]

I figured it out (Thanks to Eric)

Put a 1,$ in front of the !

:1,2!sed 's/^/TESTING /' |sed '/./=' | sed '/./N; s/\n/ /'

This command transformed
This is a test
just a sillty test
only a test

Into
1 TESTING This is a test
2 TESTING just a sillty test
only a test

If I replace the 2 with $ it does it for all the lines to the end of the file.

Thank you for giving me what I needed to solve this.

Cheers

[OFC_EDM]

Thanks Peter for the handy one liners.

I have that on my website. I must have passed by the numbering area...in too much of a rush these days!

[Eric SAUBIGNAC]

particulary thanks to Pete (my apologizes for Peter instead of Pete :-( who gave us something like a "gold mine" with his sed document.

Eric

[OFC_EDM]

I meant to thank everyone not just Eric.

Assigned points. Plus 2 extra for Eric and Pete.

Cheers

[Hein van den Heuvel]

These Unix folsk will never cease to amaze me:


VI... :1,2!sed 's/^/TESTING /'

So you are in VI, a perfectly good editor and then you fork SED to do something VI was build to do ?! Why? Because you can?!

This works just fine in vi itself...

:1,$s/^/TESTING /


I still would like to understand why a 'script' is not acceptable, but a script within vi might be.

I would still like to understand the numbering requirements details.

Cheers,
Hein.

[OFC_EDM]

Hein,

I'll be taking large text files and embedding commands in them so they will themselves become scripts.

The numbering will be part of the embedding of commands.

But will also be used for echoing progress as the script executes.

[OFC_EDM]

I didn't answer why VI and not a script.
Or even why VI and fork to a script is acceptable.

Besides the official reasons which I won't cover.

Beyond that for me personally it's faster to open VI and manually enter a one liner with one or two modifications. Than to modify a script fr each file (and there are numerous) I have to change.

I can also run it and simply do :u to undo any change. Then just retype or cut paste.

Its a matter of function from a human perspective rather than a technical script perspective.

[James R. Ferguson]

Hi (again):

> Beyond that for me personally it's faster to open VI and manually enter a one liner with one or two modifications. Than to modify a script fr each file (and there are numerous) I have to change.

In answer, I would point you back to the Perl solution I suggested. It's a one-liner typed on the commandline; creates an automatic backup copy of the file(s) that you can 'diff', keep, discard, whatever; AND, you can stack as many filenames as arguments to it as you want.

Regards!

...JRF...

[Dennis Handly]

If you want to number the lines you can also use nl(1).

[Dennis Handly]

>Pete: sed 's/^/TESTING /' yourfilename | sed '/./=' | sed '/./N; s/\n/ /'

I finally figured out why this works. I was confused by the redundant "/./".
You could use:
... | sed '=' | sed 'N; s/\n/ /'

[Pete Randall]

Dennis,

I never said I understood it! ;^)

I just combined a couple of the examples from Handy One-Liners.


Pete