VMS Script - Search for String on a Line

[Bushytea]

Hi,

I am working on a VMS script that was written by a former employee. The script works like I need it to besides with the addition of one If/Then statement. I am trying to get the statement to search for a particular string on a line that could have up to 5 or so other strings on the same line. However, I just need it to locate the one certain string.

An example of what the line could look like is below but it can vary and I am just needing to figure out how to have the script search this line, find the DisUser flag and if present move onto the next record.

Flags: DisCtlY Restricted DisUser DisNewMail DisReport Captive

Below is a piece of the script that I am working with as well. As you can see in the middle there is a statement that is trying to search for the DisUser string. However, instead of skipping the records with this string it still includes them in the file it writes.

$ GETREC:
$ read/end=quit infile rec
$ if (f$locate("Username:",rec) .eq. 0) .and. (f$locate(":",rec) .ne. 0)
$ then usr = f$extract(10,9,rec)
$ endif
$ if (f$locate("Flags",rec) .eqs. "DisUser")
$ then goto getrec
$ endif
$ if (f$locate("Last Login:",rec) .eq. 0) .and. (f$locate(":",rec) .ne. 0)
$ then date1 = f$extract(12,11,rec)
$ date2 = f$extract(45,11,rec)
$ else goto getrec
$ endif

Thank you in advance for any help or assistance.

[Hoff]

Here's the basic logic to check for the existence of a substring within the specified string:

$ if f$locate("DisUser",rec) .ne. f$length(rec) then write sys$output "Found DisUser"

I'd probably change that to the following to force the search string to single-spaced and trimmed uppercase text, and tweak the search as follows:

$ rec = f$edit(rec,"UPCASE,COMPRESS,TRIM")   ! or analogous

$ if f$locate("DISUSER",rec) .ne. f$length(rec) then write sys$output "Found DisUser"

And FWIW, parsing the output of commands and utilities is generally considered unsupported and fragile.  What you have probably adapts, but changes to the command or utility output can cause weird run-time errors.

As for alternatives, there is the GETUAI tool.   This uses documented interfaces to access SYSUAF.

I'd also suggest reading the OpenVMS User's Manual in the OpenVMS documentation shelf, if you're unfamiliar with DCL programming and DCL command procedures.

[Mike Kier]

The author of the script is confused on the use of f$Locate.  The return value is an integer, either a byte offset beginning with zero if the substring is found, or the length of the string being searched if the substring is not found.  See $ HELP LEXICAL F$LOCATE.

That said, the first IF statement's second clause is superfluous since if "Username:" is found at byte offset zero then there cannot also be a ":" at byte offset zero.

The IF statement for the "DisUser" is completely wrong.

I would suggest something like

$ IF f$Locate("Flags:", rec) .ne. f$Length(rec) ! This is the Flags line

$ Then

$   IF f$Locate("DisUser") .ne. f$Length(rec) Then GoTo getrec

$ EndIf

[Hein van den Heuvel]

>> ind the DisUser flag and if present move onto the next record.

 I don't think that is really what you want, unless you mean 'next sysuaf record' versus next record in listing.

I strongly suspect that the script should not just skip that line, but skip everything else for that username.

For that the script needs to maintain a flag to remember the event.

Personally I often prefer  to use a multi-purpose 'skip' flag versus a specific 'disuser_seen' flag.

The code might then ROUGHLY look like:

$ close/nolog infile
$ open /read infile sysuaf.lis
$
$ skip = 0
$ GETREC:
$ read/end=quit infile rec
$ len = f$length(rec)
$ if f$extract (0,9,rec) .EQS. "Username:"
$ then
$   skip = 0
$   usr  = f$extract(10,9,rec)
$ endif
$ IF (f$extract (0,6,rec).EQS."Flags:") .AND. (len .NE. f$locate("DisUser",rec)) THEN skip = 1
$ IF skip THEN GOTO getrec
$ if (f$locate("Last Login:",rec) .eq. 0) .and. (f$locate(":",rec) .ne. 0)
$ then
$   date1 = f$extract(12,11,rec)
$   date2 = f$extract(45,11,rec)
$   WRITE SYS$OUTPUT usr, " ", date1, " ", date2
$   skip = 1  ! All done untill next "Username:"
$ endif
$  goto getrec
$quit:
$ CLOSE infile

 Hein.

[Bushytea]

Thanks for the quick replies.

Yes, I do mean the next sysauf record. What the script is doing is writing users to a file but I am needing to omit the users who have this DisUser flag from the file.

Thanks,

Mike

[The Brit]

Spoiler

If all you are trying to do is separate "DisUser'd" accounts from the rest, you might want to consider using the "/Brief" output from the SYSUAF.     This way you get 1 line per account, and the the DISUSER flag will be the last item on the line, (if the account is disusered)

$ mc authorize show /brief   smithd

Owner            Username         UIC         Account      Privs     Pri       Directory

David Smith  SMITHD       [120,4127]      912            All         4         Disuser

The obvious advantage is that there is less garbage to churn through.

Having said that, you should look in to using GETUAI utility, as suggested by Hoff.

Dave.

[Craig A Berry]

The problem with the GETUAI utility that a couple of people have suggested is that it doesn't really fit the problem being presented, which appears to be to query for all the usernames that have not been disusered.  As far as I can tell, with GETUAI you have to already know the username you are interested in.  You'd still have loop over a listing or SYSUAF.DAT itself to find out what all the usernames are and then test each one for whether it's been disusered.

Joe Meadows' UAF utility, on the other hand, makes querying easy.  Just do:

$ uaf/select=flags=nodisuser/display=username

It's available from:

http://code.google.com/p/jmuaf/

[x2084]

You may want to use search to reduce and select the wanted information. Something like

$ pipe define/user sysuaf sys$system:sysuaf ; mc authorize show * |search sys$input Username:,Flags: |search sys$input DisUser/match=nand/out=x.lis
$ pipe search x.lis Flags:/wind=(1,0) |search sys$input Username:

[Hein van den Heuvel]

So are you all set now with the various suggestions?

btw 1... careful with the hardcoded 9 character username.

OpenVMS itself supports 12.

Btw 2... I 'happened' to have a bit of DCL which possibly does pretty much what you want.

It can go straight for SYSUAF (local copy in example below).

And yeah, it could flip the disuser bit and update as well, but I prefer to go through a file.

$!
$! uaf_lastlogin.com    Hein van den Heuvel,August 2007.
$
$! List records from SYSUAF for which the Last Interactive Login is more
$! than 'p1' days ago (default 90).
$
$IF p1.eqs."" THEN p1 = "90"
$cutoff_date = f$cvtime("TODAY -''p1'-")
$!libr/extr=$uafdef/out=uafdef.tmp sys$library:lib.mlb
$!sea uafdef.tmp flag...
$!EQU    UAF$Q_LASTLOGIN_I       396
$!EQU    UAF$L_FLAGS     468
$!EQU    UAF$V_DISACNT   4
$
$define sysuaf sys$disk:[]sysuaf.dat  ! Local copy for testting
$sysuaf = f$parse("SYSUAF","SYS$SYSTEM:.DAT",,,"SYNTAX_ONLY")
$open /read/share=write uaf 'sysuaf'
$loop:
$ read/end=done uaf rec
$ lastlogin_bin = F$EXTR(396,8,rec)
$ lastlogin_asc = F$FAO("!%D",f$cvui(32,32,f$fao("!AD",8,lastlogin_bin)))
$ IF f$cvtime(lastlogin_asc) .GTS. cutoff_date THEN GOTO loop
$ IF f$cvsi(468*8+4,1,rec) THEN GOTO loop ! disuser already?
$ username = F$EXTRACT(4,12,rec)
$ WRITE SYS$OUTPUT "MODIFY ''username' /FLAG=DISUSEER ! Last Login: ", lastlogin_asc
$ GOTO loop
$done:
$close uaf

Also, a similar script using perl....

while (<>) {
  if (/^Username:(.{9})/) {
    $skip = 0;
    $user = $1;
  }
  next if $skip;
  $skip = 1 if /^Flags.*DisUser/;
  if (/^Last Login:.(.{11}).{22}(.{11})/) {
    print qq($user $1 $2\n);
    $skip = 1;
  }
}

[Bushytea]

Yeah, the suggestions help give a bit of a clearer picture as to how some of this works. It is always difficult to work with a new language plus try to figure out what the previous person was doing.

The way this was setup was to compare dates to see who has not logged on in 90 days, the user does not have a DisUser flag already and then write the Username plus dates to a .dat file. Then, you run another script that actually goes through the .dat  file created and DisUser's all the usernames. I believe we write to the file with one script do allow us to proof the usernames before going ahead and setting the DisUser flag.

Just so you guys can see here is the entire script that searches through the sysuaf.lis file for the users.

$ !
$ set noverify
$ set noon
$ !
$ compdt = "28-FEB-2008"
$ defdt = "01-JUL-2008"
$ !
$ if p1 .nes. ""
$ then compdt = p1
$ write sys$output "Compare date is " + compdt
$ else
$ write sys$output "No comparison date included."
$ goto quit
$ endif
$ !
$ set def sys$system
$ uaf := "$authorize"
$ uaf list * /full
$ set def mvx
$ open/read infile sys$system:sysuaf.lis
$ open/write outfile uaf_old_accts.dat
$ GETREC:
$ read/end=quit infile rec
$ if (f$locate("Username:",rec) .eq. 0) .and. (f$locate(":",rec) .ne. 0)
$ then usr = f$extract(10,9,rec)
$ endif
$ if (f$locate("Last Login:",rec) .eq. 0) .and. (f$locate(":",rec) .ne. 0)
$ then date1 = f$extract(12,11,rec)
$ date2 = f$extract(45,11,rec)
$ else goto getrec
$ endif
$ if date1 .eqs. "           " then date1 = defdt
$ if date2 .eqs. "           " then date2 = defdt
$ if (f$cvtime(date1,,) .lts. f$cvtime(compdt,,)) .and. -
 (f$cvtime(date2,,) .lts. f$cvtime(compdt,,)) then goto writerec
$ goto getrec
$ WRITEREC:
$ wrtrec = usr+"   "+date1+" - "+date2
$ write outfile wrtrec
$ goto getrec
$ QUIT:
$ close infile
$ close outfile
$ exit
[End of file]

 Thanks!

[Craig A Berry]

Of course there are various ways to do this and fixing up your current approach can work, but with UAF you can get all the users (who are not disusered) and have not had an interactive login within the last 90 days and display the username and last login time with a single query like the following:

$ t_minus_90 = f$cvtime("-90-0", "absolute")
$ uaf/select=(flags=nodisuser,interactive=(17-nov-1858,"''t_minus_90'")) - 
$_ /match=and/display=(username,interactive)

[John McL]

It might be that this new system is messing with the formatting but just in case it's not, I think you're doing yourself no favors with your code format.

1 - You need comments in it.  There's nothing more embarrassing than returning to your own code in 6 months time and not being able to quickly understand what it does.

2 - Using indenting makes it far easier to read.  My system is to indent labels by one space, other statements by one tab character and anything within a block (eg. from an IF ... THEN ... ENDIF) idented by another tab.

Eg.

$ <tab>if p1 .nes. ""
$ <tab>then

$ <tab><tab> compdt = p1
$ <tab><tab>write sys$output "Compare date is " + compdt
$ <tab>else
$ <tab><tab>write sys$output "No comparison date included."
$ <tab><tab>goto quit
$ <tab>endif

3 - Putting IF conditions in parentheses makes them much clearer and they'll look the same as with many languages.

You might think I'm being picky but starting out with good habits is easier than trying to change over at a later time.  Also if you are new to VMS then clarity and ease of understanding are important.

PS.  Your F$LOCATE is still wrong.  If the substring doesn't exist in the target string F$LOCATE returns the length of the string, i..e. F$LENGTH(STRING).   Think of it as a offset from the start of the string (first pos = 0).  If F$LOCATE returns 0 it's the start of the string, and having searched the whole string the pointer has moved past the end where the "offset" (although it's now past the end) is now equivalent to the length of the string. 

[Bushytea]

John McL, I agree 100% on your suggestions I just have not yet worked on cleaning up the code to fix some of the issues with the f$Locates as well as making it easier to read. This is not my code and nor did I write it this way. It was a past programmer who set this code up.

I have tried a few of the suggestions here to get the code to not write the users who have the disuser flag to the .dat file but still no luck. I will keep plugging away at it today and hopefully get the code to process those users correctly.

Thanks,

Mike

[Hein van den Heuvel]

>> (f$locate("Last Login:",rec) .eq. 0)

That checks whether the specified string found at the begin of the file: offset 0

If NOT found, then the string length is returned which also might be 0.

>>  .and. (f$locate(":",rec) .ne. 0)

That addition basically protects against empty lines.

I find it much better to make that check immediately after the READ, for once and for all:

$ GETREC:

$ read/end=quit infile rec

$ IF 0.eq.f$len(rec) THEN GOTO GETREC

:

And for testing for a piece of string in a known location, such as the begin of the line, I prefer a simple extract and compare as per my earlier example. That removes any doubt and is likely faster.

Did you check the code I posted from 2007? (or google: +hein +uaf_lastlogin.com  )

I think that does _exactly_ what you want and more efficiently so.

It also creates a temp file, but makes it a useable file,  exectuable as input to authorize.

Hein

[Bushytea]

Yeah, I did see check your code that you previously posted and it does look like it will work better than what is currently setup however I hate to ask but would you be able to explain the code a bit more? With the help so far it has cleared up a ton for me how some of this works. I may not be reading your code correctly but it looks as though it will go search through the users for certain criteria but then it enters the loop which then sets the disuser flag on the users who meet that criteria.

I believe they like this old code because it would produce a file that they could go a double check the results/users before going through and setting the disuser flag. So once this code was run it would produce a .dat file that could be double checked then with another piece of code that list would be run through, setting the disuser flag.  

I may have to try that route though because no matter what variation of code I put in this current code it just will not omit the users who already have the disuser flag set from the file.

Thanks again for everyone's help and patience.

[Hein van den Heuvel]

>> Yeah, I did see check your code that you previously posted and it does look like it will work better

Just try it?!

>>  I hate to ask but would you be able to explain the code a bit more? 

Sure...

>>  it will go search through the users for certain criteria but then it enters the loop which then sets the disuser flag on the users who meet that criteria.

The only criteria it tests for is the last interactive login date.

It TESTS the DisUser bit, but only generated file with suggested changes for review before executing, just like the doctor ordered.

>> it just will not omit the users who already have the disuser flag set from the file.

Maybe post your final code again, or Email if you feel better about that.

Also, just set it again? No harm done.

It is seems there is  more time wasted finding out whether to do it or not, than jut doing it regardless.

$IF p1.eqs."" THEN p1 = "90"
$cutoff_date = f$cvtime("TODAY -''p1'-")

 Accept an optional parameter with a number of days for a cut-off, and convert to a comparison time.

$!libr/extr=$uafdef/out=uafdef.tmp sys$library:lib.mlb
$!sea uafdef.tmp flag...
$!EQU    UAF$Q_LASTLOGIN_I       396
$!EQU    UAF$L_FLAGS     468
$!EQU    UAF$V_DISACNT   4

 Those comment lines show how one can find the magic offsets and bit numbers for the 'binary' sysuaf records. You may need to add a section for non-interactive logins using UAF$Q_LASTLOGIN_N  ( 404 )

$define sysuaf sys$disk:[]sysuaf.dat  ! Local copy for testting
$sysuaf = f$parse("SYSUAF","SYS$SYSTEM:.DAT",,,"SYNTAX_ONLY")

 You need the second line to be ready for SYSUAF either being a logical, or a file in SYS$SYSTEM.

That first line is just for  TESTING against a private copy. To be removed for production usage.

$open /read/share=write uaf 'sysuaf'
$loop:
$ read/end=done uaf rec
:
$ GOTO loop
$done:

 Core loop, after opening the file by name as established from the parse call.

$ lastlogin_bin = F$EXTR(396,8,rec)
$ lastlogin_asc = F$FAO("!%D",f$cvui(32,32,f$fao("!AD",8,lastlogin_bin)))
$ IF f$cvtime(lastlogin_asc) .GTS. cutoff_date THEN GOTO loop

 Now that's a tricky bit, a hack if you like.

DCL unfortunately does NOT have a native method to convert the 8-byte binary OpenVMS dates, such as stored in  SYSUAF records to a textual representation or visa versa. This trick using F$FAO does the job.

An alternative trick is to calculate the binary equivalent for the cutoff date, and just compare the fmost significant longword. That get's you withing 7 minutes. See below.

(My favourite : run any program/debug, set radix hex, depo/date 10000="1-jan", exam .)

$ IF f$cvsi(468*8+4,1,rec) THEN GOTO loop ! disuser already?
$ username = F$EXTRACT(4,12,rec)
$ WRITE SYS$OUTPUT "MODIFY ''username' /FLAG=DISUSEER ! Last Login: ", lastlogin_asc

 Now to the final test, for disuser, by extracting just one specific bit from the record @ byte 468, bit #4

If we the bit was not set, fall through, pick up the username, and generate a command FOR REVIEW.

Hein

Proof that the top 4 bytes in an 8 byte date is 7  minutes...

$ run tmp/debu
%         OpenVMS Alpha Debug64 Version V8.3-009
DBG> set rad hex
DBG> dep /long 10000=0
DBG> dep /long 10004=-1
DBG> ex/date 10000
TMP$MAIN\TMP$MAIN:         0 00:07:09.49

[Bushytea]

Thanks for all your help Hein. I apologize for my delay but just got multiple projects going so this one was put off for a bit.

I am starting to understand some of this vms programming language after the help, research and reading but I know there is a ton to learn. I just never saw vms programming but I have worked with other languages.

Below is the current code in the original script. I have tried most variations that I can come up with and I still have not been able to get the script to omit the users with the DisUser flag. They want it to do this so that they have a file to go back to for reference of how many new users got this DisUser flag set.

$ set noverify
$ set noon
$ !
$ compdt = "17-MAR-2011"
$ defdt = "15-JUN-2011"
$ !
$ if p1 .nes. ""
$ then compdt = p1
$ write sys$output "Compare date is " + compdt
$ else
$ write sys$output "No comparison date included."
$ goto quit
$ endif
$ !
$ set def sys$system
$ uaf := "$authorize"
$ uaf list * /full
$ set def mvx
$ open/read infile sys$system:sysuaf.lis
$ open/write outfile uaf_old_accts.dat
$ GETREC:
$ read/end=quit infile rec
$ IF f$Locate("Username:",rec) .ne. f$Length(rec)
$ then usr = f$extract(10,9,rec)
$ endif
$ IF f$Locate("Flags:", rec) .ne. f$Length(rec)
$ Then
$   IF f$Locate("DisUser",rec) .ne. f$Length(rec) Then GoTo getrec
$ EndIf
$ IF (f$Locate("Last Login:",rec) .eq. 0) .and. (f$Locate(":",rec) .ne. 0)
$ then date1 = f$extract(12,11,rec)
$ date2 = f$extract(45,11,rec)
$ else goto getrec
$ endif
$ if date1 .eqs. "           " then date1 = defdt
$ if date2 .eqs. "           " then date2 = defdt
$ if (f$cvtime(date1,,) .lts. f$cvtime(compdt,,)) .and. -
 (f$cvtime(date2,,) .lts. f$cvtime(compdt,,)) then goto writerec
$ goto getrec
$ WRITEREC:
$ wrtrec = usr+"   "+date1+" - "+date2
$ write outfile wrtrec
$ goto getrec
$ QUIT:
$ close infile
$ close outfile
$ exit

Then once the file is made from the script above it is then run in this script:

$ set verify
$ set noon
$ !
$ uaf := "$authorize"
$ !
$ open/read infile uaf_old_accts.dat
$ !
$ GETREC:
$ read/end=quit infile rec
$ usr = f$extract(0,11,rec)
$ sys
$ uaf modi 'usr' /flag=disuser
$ mvx
$ goto getrec
$ QUIT:
$ close infile
$ exit

I have been working with your code as well but when I run it I get the errors you see below. This is just a snipet of them as it runs through them pretty fast. I did not change anything with your code.

%DCL-W-UNDSYM, undefined symbol - check validity and spelling
 \REC\
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
 \LASTLOGIN_ASC\
%DCL-W-UNDFIL, file has not been opened by DCL - check logical name
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
 \REC\
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
 \LASTLOGIN_B\
%DCL-W-UNDSYM, undefined symbol - check validity and spelling
 \LASTLOGIN_ASC\

Thanks!

[Hein van den Heuvel]

Hmmm, you seem to have trouble reading no progress was made. Please re-read, slowly.

When you loop through a full sysuaf  listing the 'disuser' flag is on a line  seperate from the username.

So if the code, as you have it, just reads the next record after 'seeing' the disuser, nothing happens.

The code somehow has to remember that is has seen disuser and use that to skip the write.

For example you could clear 'usr' to indicate it is no longer intersting. Change

$ IF f$Locate("DisUser",rec) .ne. f$Length(rec) Then GoTo getrec

to become

$ IF f$Locate("DisUser",rec) .ne. f$Length(rec) Then usr = "" 

and further down, it currently reads a (ugly)

$ IF (f$Locate("Last Login:",rec) .eq. 0) .and. (f$Locate(":",rec) .ne. 0)
$ then date1 = f$extract(12,11,rec)
$ date2 = f$extract(45,11,rec)
$ else goto getrec
$ endif 

Change all that to :

$ IF (f$extr(0,11,rec) .NE. "Last Login:") .OR. (usr .EQ. "") goto getrec
$ date1 = f$extract(12,11,rec)
$ date2 = f$extract(45,11,rec)

That should do it.

Now my code reads SYSUAF records directly, so it can go to the next record when idsuser is detected.

My code also outputs an intermediate file, as you require, but if formats it as valid input for AUTHORIZE such that no second script is needed, of course is could readily be changed to output just the names.

No you have trouble with that code.

Typically you would have to go to the FIRST error lines to see the root cause.

It probably just failed to open the file, and the code as posted is not robust enough to detect that. Sorry.

It needs to have an /ERROR=xxx on the OPEN and READ lines. Oh well.

>> I did not change anything with your code.

Did you see that for testing purposed it does NOT go to the real file but rather to a local copy?

$define sysuaf sys$disk:[]sysuaf.dat  ! Local copy for testting

You need to remove that line, or provision a copy as intended, and I would use a 'small' sysuaf file to test.

For investigate you may want to issue : $ SET VERIFY

To reduce thepotential  volume of data you may want to change that simple $LOOP:

to become

$ i = 0

$loop:

$ i = i + 1

$ if i .gt. 2 then exit

Hein