topic Re: Need help with scripting mass file edits.. in Operating System - Linux

Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 15:59:23 GMT

Hello,

I am wanting to know a way to shell (ksh)script-edit a file by having a script that searches for a specific string, and then inputs lines after that specific string. Please help, as I will be up all night if I can't figure this out.

Re: Need help with scripting mass file edits..

James R. Ferguson — Tue, 14 Aug 2007 16:46:49 GMT

Hi Patrick:

I am going to assume that you want to preserve the entire contents of the line on which your match occurs, and simply add some number of lines thereafter.

This script will do that for any number of files *and* update them "in-place":

By example, using 'localhost' as a string to be matched (e.g. as with '/etc/hosts'):

# perl -pi.old -e 's/(localhost.*)/$1\nnew\nlines\nhere/' file1 file2 file3

The original file will be preserved with an ".old" suffix and the file updated in-place.

Regards!

...JRF...

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 17:41:20 GMT

Perhaps I should have stated that I wanted to input lines of text after the string that is being searched out.

Re: Need help with scripting mass file edits..

James R. Ferguson — Tue, 14 Aug 2007 17:52:23 GMT

Hi (again):

> Perhaps I should have stated that I wanted to input lines of text after the string that is being searched out.

OK, and do what with any text that follows the match on the line?

This discards any text on the line on which the pattern matches and inserts new test lines thereafter:

# perl -pi.old -e 's/(localhost)/$1\nnew\nlines\nhere/' file1 file2 file3 ...

Change the pattern in parentheses to anything you want.

Regards!

...JRF...

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 18:23:51 GMT

I would want to keep the text on that line, but I guess a solution would be to let that text be discarded, and re-add it in the replacement text.

Re: Need help with scripting mass file edits..

James R. Ferguson — Tue, 14 Aug 2007 18:37:40 GMT

Hi:

> I would want to keep the text on that line, but I guess a solution would be to let that text be discarded, and re-add it in the replacement text.

OK, solution-1 retains it. Solution-2 discards it and this one (solution-3) pushes text that trails the matched pattern to its own line and then inserts the line(s) you specify:

# perl -pi.old -e 's/(localhost)(.*)/$1\n$2\nnew\nlines\nhere/' file1 file2 file3 ...

Regards!

...JRF...

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 18:58:58 GMT

Nice! I just tested this out, and it works pretty dang good!! Thanks so much for your responses. Solution 3 was the one.

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 19:06:18 GMT

Wait, what if I wanted to enter the followng after the string:

Cmnd_Alias EPROV_ADMIN_CMDS = /usr/sbin/groupadd, /usr/sbin/groupdel, /usr/sbin/groupmod, \
/usr/bin/last, /usr/bin/listusers, /usr/sbin/logins, \
/usr/sbin/usermod, /usr/sbin/useradd, /usr/sbin/userdel, \
/usr/bin/passwd, /usr/bin/ypmatch, /usr/bin/yppasswd, \
/usr/sbin/ypcat, /usr/bin/login

Is there a way to do this?

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Tue, 14 Aug 2007 19:19:57 GMT

I will be up all night if I can't get this figured out. I just got tasked (last minute) to change the sudoers file on 398 hosts by by tomorrow morning.

Re: Need help with scripting mass file edits..

Steven Schweda — Wed, 15 Aug 2007 01:03:23 GMT

I have an aversion to doing someone else's
job for him when he's getting paid for it
and I'm not, but if you expect someone else
to do it for you, you might consider
precisely and accurately specifying the work
you want done. In this case, one way to do
that might be to show an example of what the
file would look like before and after the
change(s) you'd like to make.

When a reply begins, "I am going to assume",
then _you_ can assume that you didn't specify
the problem clearly enough.

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Wed, 15 Aug 2007 06:12:44 GMT

1. It's called asking for help, because I don't have the answer. Because I still have to do the job anyway, I ask those with experience how to do it more efficiently. When a good example is given, I give thanks, take the example, and modify it for what I need to get done, and test it. I wouldn't say that's doing my work for me. That's called asking for help!

2. Here's a good example of before and after:

BEFORE:
# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification

AFTER:
# Host alias specification
Host_Alias = hostname

# User alias specification
User_Alias ADMINS = someuser

# Cmnd alias specification
Cmnd_Alias ADMIN_CMDS = /usr/sbin/groupadd, /usr/sbin/groupdel, /usr/sbin/groupmod, /usr/bin/last, /usr/bin/listusers, /usr/sbin/logins,
/usr/sbin/usermod, /usr/sbin/useradd, /usr/sbin/userdel, /usr/bin/passwd, /usr/bin/ypmatch, /usr/bin/yppasswd,/usr/sbin/ypcat, /usr/bin/login

Cmnd_Alias PROTECTED_CMDS = !/usr/bin/passwd root, !/usr/bin/rm -rf /*, !/usr/bin/chown root
Cmnd_Alias MISC_CMDS = /usr/bin/make, /usr/bin/chmod, /usr/bin/awk, /usr/bin/cat,/usr/bin/chown, /usr/bin/cp, /usr/bin/cut, /usr/bin/di
ff,/usr/bin/grep, /usr/bin/ls, /usr/bin/mv, /usr/bin/rm,/usr/bin/sed, /usr/bin/sleep, /usr/bin/sort, /usr/bin/tail,/bin/echo, /bin/touch, /usr
/bin/which

# User privilege specification
IDM_ADMINS LOCALHOST = MISC_CMDS,ADMIN_CMDS

If there is already text in each section, the text is moved down, and the new text is iserted.

Re: Need help with scripting mass file edits..

Dennis Handly — Wed, 15 Aug 2007 06:33:14 GMT

Assuming you wanted to add complete lines after the string you found, you could use sed. The 3 backslashes are one to quote the next and then one to continue the sed append.

This inserts the text after the string "firstname1".

(Note: TMP in the script below is removed just be tidy. You don't want to remove your data file. ;-)
###########################################################################################
#!/usr/bin/ksh

TMP=/var/tmp/itrc.in.$$

cat < $TMP
firstname1 lastname1
firstname2 lastname2
EOF

sed -e '
/firstname1/a\
Cmnd_Alias EPROV_ADMIN_CMDS = /usr/sbin/groupadd, /usr/sbin/groupdel, /usr/sbin/groupmod, \\\
/usr/bin/last, /usr/bin/listusers, /usr/sbin/logins, \\\
/usr/sbin/usermod, /usr/sbin/useradd, /usr/sbin/userdel, \\\
/usr/bin/passwd, /usr/bin/ypmatch, /usr/bin/yppasswd, \\\
/usr/sbin/ypcat, /usr/bin/login
' $TMP

rm -f $TMP

If you have this data in a file, you can use the following instead of "a":
/firstname1/r file-to-insert

Re: Need help with scripting mass file edits..

Dennis Handly — Wed, 15 Aug 2007 06:43:30 GMT

>2. Here's a good example of before and after:

You have 4 strings to look for, you need 4 sed "a" functions.

>If there is already text in each section, the text is moved down, and the new text is inserted.

That's what the "a" function does.

Re: Need help with scripting mass file edits..

James R. Ferguson — Wed, 15 Aug 2007 07:36:33 GMT

Hi (again) Patrick:

Please don't forget to evaluate the answers you received, either:

https://forums1.itrc.hp.com/service/forums/helptips.do?#28

Regards!

...JRF...

Re: Need help with scripting mass file edits..

Patrick Ware_1 — Wed, 15 Aug 2007 07:39:00 GMT

I won't forget.

Re: Need help with scripting mass file edits..

Sandman! — Wed, 15 Aug 2007 10:17:22 GMT

Based on your sample input here is an ex(1) construct you could try:

ex -s infile </^# Cmnd/r !echo "Cmnd_Alias EPROV_ADMIN_CMDS = /usr/bin/last, \\\"
/^# Cmnd/+r !echo "/usr/sbin/groupmod,/usr/sbin/useradd,/usr/bin/listusers, \\\"
/^# Cmnd/++r !echo "/usr/bin/ypmatch,/usr/bin/yppasswd,/usr/sbin/ypcat"
wq
EOF