How can I delete entire lines based on a search expression in vi?

[Eric Antunes]

I need to remove the following lines from a file:

<AccountType>Some stuffs in here also</AccountType>

How can I do this with vi or another hpux program?

Thank you,

Eric Antunes

[H.Merijn Brand (procura · Accepted solution]

Assuming the (XML like) lines are all on a single line each:

In vi

  :g/<AccountType>/d

With perl:

  $ perl -ni -e'm/ <AccountType>/ or print' file.txt

But that is only safe if there is onlo what you show on a single line. If text folds and the closing tag is on the next line, or if other text is on the same line, this is going to fail.

Is the whole file valid HTML? or XML? 

[Eric Antunes]

Hi Merijn,

It worked! This is a valid XML file but I need to remove that part in the entire file.

Thank you,

Eric

[Dennis Handly]

><AccountType>Some stuff in here also</AccountType>

You can be more pedantic:

:g:<AccountType>.*</AccountType>:d

If you need to anchor the lines to beginning and end:

:g:^<AccountType>.*</AccountType>$:d

And you can use these in sed.

[H.Merijn Brand (procura]

If you want to be more pedantic, please escape the inside slash :)

:g/<AccountType>.*<\/AccountType>/d

If you need to anchor the lines to beginning and end:

Assuming \s is whitespace in your editor, and it supports * for greedy and ? non-greedy modifiers

:g/^\s*<AccountType>.*?<\/AccountType>\s*$/d

Otherwise greedy .* might get way to much :)

And you can (maybe) use these in sed (depending on what version od sed you have).

[Dennis Handly]

>If you want to be more pedantic, please escape the inside slash :)

Oops, done, replaced by a colon ":".  :-(

[Eric Antunes]

I don't need to be pedantic. :-)

Thank you guys.

[James R. Ferguson]

---

@Eric Antunes wrote:

I don't need to be pedantic. :-)

---

Learning precision when deploying regular expressions will keep you from matching when you *don't* want to match as well as allow you to match when you should.  Form the good habits early and save yourself countless hours of pain.

Regards!

...JRF...

[Eric Antunes]

Hi again,

I didn't need to be pedantic since this is a XML file with serious parse errors. So, things like <AccountType>,<EMail/>, <Contact/>... could safely be removed. But now it gets harder (at least for me, a DBA): I need to add stuffs to the file.

For example, I would like to add the line "<SelfBillingIndicator>0</SelfBillingIndicator>" before each line "</Customer>"

How can I do that in vi?

Thank you,

Eric

[James R. Ferguson]

---

@Eric Antunes wrote:

For example, I would like to add the line "<SelfBillingIndicator>0</SelfBillingIndicator>" before each line "</Customer>"

How can I do that in vi?

---

Using 'vi' search for the first line with "Customer":

Insert your "SelfbillingIndicator" line above it by (O)pening a new line above it.

After you have inserted the new line, yank (yy) a copy of it.

Hit 'n' to continue to search forward for "Customer" and when you reach a matching line, use 'P' to put the yanked buffer above the "Customer" line.

Regards!

...JRF...

[H.Merijn Brand (procura]

vi

  :g,</Customer>,s,^,<SelfBillingIndicator>0</SelfBillingIndicator>\n,

(if your version of vi does not support \n, use Ctrl-V - Enter)

perl

 $ perl -pe'm{ </Customer>} and print "<SelfBillingIndicator>0</SelfBillingIndicator>\n"' file.xml

[Eric Antunes]

Hi again,

The vi command worked perfectly with Ctrl-V-Enter.

Now, I have this pattern:

    <BillingAddress>
      <AddressDetail>Stuffs a</AddressDetail>
      <City>Stuffs b</City>
      <PostalCode>Stuffs c</PostalCode>
      <Country>Stuffs d</Country>
    </BillingAddress>
    <ShipFromAddress>
      <AddressDetail/>
      <City/>
      <PostalCode/>
      <Country/>
    </ShipFromAddress>

And would like to correct/fiil the ShipFromAddress part with the BillingAddress data, to look like this:

    <BillingAddress>
      <AddressDetail>Stuffs a</AddressDetail>
      <City>Stuffs b</City>
      <PostalCode>Stuffs c</PostalCode>
      <Country>Stuffs d</Country>
    </BillingAddress>
    <ShipFromAddress>
      <AddressDetail>Stuffs a</AddressDetail>
      <City>Stuffs b</City>
      <PostalCode>Stuffs c</PostalCode>
      <Country>Stuffs d</Country>

    </ShipFromAddress>

This is really getting harder...

Thank you,

Eric

[Eric Antunes]

I'm trying things like this in vi:

:g,<AddressDetail/>,6^Uyy6^D,s,p

:g,<AddressDetail/>,!6^Uyy6^D,s,p

:g,<AddressDetail/>,6^U,yy,6^D,s,p

For this last one, vi says "U: Not an editor command"

But no success for now...

[H.Merijn Brand (procura]

By the time your requirements get as complicated as this, you should not want to "solve" that in vi

I'd suggest perl, but I am clearly biased

Take the suggestions from this thread and create yourself a simple perl script that modifies the original file and outputs the result to *another* file, so you can then alter the script as long and often as you want till you get where you want to be.

Of course we can help with the specifics, but in order to believe in this new approach you will have to follow your own signature: learn something new every day. Don't be affraid this new approach will take too long. Its time lost is regained manyfold very very soon after. When I see the time span between your posts, the time you would have needed to learn the basics would already have been regained. 

[James R. Ferguson]

HI Eric:

A Perl script that meets your requirements might be:

#!/usr/bin/perl
use strict;
use warnings;
my $insert = 0;
my @a = ();
while (<>) {
    if (m{<BillingAddress>}..m{</BillingAddress>}) {
        $insert = 1;
        if (m{</?BillingAddress>}) {
            print;
            next;
        }
        push @a, $_;
    }
    if (m{<ShipFromAddress>}..m{</ShipFromAddress>}) {
        print if m{</?ShipFromAddress>};
        if ($insert) {
            $insert = 0;
            print @a;
            @a = ();
        }
        next;
    }
    print;
}
1;

Assuming you called the script 'reformat', run it as:

# ./reformat file

Regards!

...JRF...

[H.Merijn Brand (procura]

JRF, s/\A/#/ :)

(first line of a script starts with #!/path/to/program (the # was missing)

More serious, once you get here, it might be worth looking at XML parser modules and modify the content in a more structural way. When done, use the module's methods to write valid XML to disk 

[Eric Antunes]

Ok, Merijn, I'm not so unscriptable ;)

The script ran wonderfuly with just a little change in the file and script from <BillingAddress> to <FBillingAddress> because there were other <BillingAddress> I didn't want to be found:

./MyFirstPerlScrit.pl file-to-format.xml > file-formated.xml

Thanks,

Eric

[James R. Ferguson]

---

@H.Merijn Brand (procura wrote:

JRF, s/\A/#/ :)

(first line of a script starts with #!/path/to/program (the # was missing)

More serious, once you get here, it might be worth looking at XML parser modules and modify the content in a more structural way. When done, use the module's methods to write valid XML to disk 

---

Yes, thanks for pointing out the absent interpreter line :-)  I editted my post to reflect that change.

I know you are correct when you note that the XML parser modules are far better than the roll-your-own method I chose.  The decent into HTML or XML hell can be steep and slippery :-)  Thanks, Merijn!

Regards!

...JRF...

[James R. Ferguson]

---

@Eric Antunes wrote:

The script ran wonderfuly with just a little change in the file and script from <BillingAddress> to <FBillingAddress> because there were other <BillingAddress> I didn't want to be found:

---

Hi (again) Eric:

If it was only the first stanza that you wanted to change, a slight code change would have enabled you to leave your input file alone.

Change these lines:

if (m{<BillingAddress>}..m{</BillingAddress>}) {
       
if (m{<ShipFromAddress>}..m{</ShipFromAddress>}) {

 TO:

if (m?<BillingAddress>?..m?</BillingAddress>?) {
        
if (m?<ShipFromAddress>?..m?</ShipFromAddress>?) {

That is, alter the match delimiter from "{" and "}" to "?" and "?".

This will match one pattern range, only.

Regards!

...JRF...

[H.Merijn Brand (procura]

Or use the triple-dot

from 'man perlop'

   Range Operators
       Binary ".." is the range operator, which is really two different
       operators depending on the context.  In list context, it returns a list
       of values counting (up by ones) from the left value to the right value.
       If the left value is greater than the right value then it returns the
       empty list.  The range operator is useful for writing "foreach (1..10)"
       loops and for doing slice operations on arrays. In the current
       implementation, no temporary array is created when the range operator
       is used as the expression in "foreach" loops, but older versions of
       Perl might burn a lot of memory when you write something like this:

           for (1 .. 1_000_000) {
               # code
           }

       The range operator also works on strings, using the magical auto-
       increment, see below.

       In scalar context, ".." returns a boolean value.  The operator is
       bistable, like a flip-flop, and emulates the line-range (comma)
       operator of sed, awk, and various editors.  Each ".." operator
       maintains its own boolean state.  It is false as long as its left
       operand is false.  Once the left operand is true, the range operator
       stays true until the right operand is true, AFTER which the range
       operator becomes false again.  It doesn't become false till the next
       time the range operator is evaluated.  It can test the right operand
       and become false on the same evaluation it became true (as in awk), but
       it still returns true once.  If you don't want it to test the right
       operand till the next evaluation, as in sed, just use three dots
       ("...") instead of two.  In all other regards, "..." behaves just like
       ".." does.

 Personally I have never ever felt the need for ??

YMMV TIMTOWTDI 

[Eric Antunes]

Hi James,

Given that the xml file contains several patterns like:

<BillingAddress>..</BillingAddress>

<ShipToAddress>..</ShipToAddress>

..

<BillingAddress>..</BillingAddress>

<ShipFromAddress>..</ShipFromAddress>

..

<BillingAddress>..</BillingAddress>

<ShipToFromAddress>..</ShipToFromAddress>

..

And I just wanted to format the 2º pattern:

<BillingAddress>..</BillingAddress>

<ShipFromAddress>..</ShipFromAddress>

I coud use the following at the upper level:

if (m{<BillingAddress>}..m{</ShipFromAddress>}) {

Couldn't I?

Thanks,

Eric

[H.Merijn Brand (procura]

No. The .. operator is a range operator

If you want to match a single line where both the opening and the closing tag have *simple* data in between the tags (e.g. no < or other nested stuff),

if (m{<(BillingAdress)>[^<]*</\1>}) {
    # Do something on this line

    # Keep the current indent
    my ($indent) = (m/^(\s*)/);

    # Insert a new line before (same indent)
    s{^}{$indent<newTag>content</newTag>\n};

    # Add a new line after this (same indent)
    $_ .= "\n$indent<newTag>content</newTag>"; # TIMTOWTDI
    }

is what you'd want