How to get the NUmber of lines in a file into a variable

[Prem Mohan]

I want to get the number of lines in a file into a variable. I do get the number of lines in a files using this command
sear "" /log /nooutput.
But I have trouble assigning this output to a variable. In Unix I do wc -l . Any help is greatly appreciated.
Prem.

[Kris Clippeleyr]

Hi,

You might try the lexical function F$FILE_ATTRIBUTES, with FILE_LENGTH_HINT as item-code. Something like:
$ flh = f$file_attributes("TEST.TXT","FILE_LENGTH_HINT")
$ show symbol flh
See also HELP LEXICALS F$FILE_ATTRIBUTES

Greetz,

Kris (aka Qkcl)

[Volker Halle]

Prem,

you can also achiev this via PIPE (see examples in OpenVMS FAQ):

$ pipe search login.com ""/log/noout| ( read sys$input line ; wc = F$ELEMENT(3," ",line) ; define/job wc &wc)
$ x=F$TRNLNM("wc")
$ sho sym x
X = "79"
$ DEASS/JOB wc

FILE_LENGTH_HINT only works on ODS-5 volumes !

$ write sys$output f$file_attributes("login.com","file_length_hint")
(-1,-1)

Volker.

[Ken Robinson · Accepted solution]

The new way is to use the PIPE command:

$ pipe sear/noout/stat "" | sear sys$pipe "Records sear" | ( read sys$pipe line ; num = f$ele(2," ",f$edi(line,"compress")) ; def/job numl &num )
$ numl = f$trn("numl")
$ sho sym numl

Ken

[Ian Miller.]

note that the file length hint may be invalid. The command ANAL/RMS/UPDATE attempts to update the file length hint. The help text for ANAL/RMS/UPDATE explains the file and record formats to which the file lenght hint applies.
You also need a recent version of VMS (its present in VMS V7.3-1 but no in V7.1).

Parhaps something ingenious could be done with the SEARCH command in a PIPE.

[Craig A Berry]

Prem,

If you have GNV installed, just do it the same way you do it on Unix:

$ bash -c wc -l

If not, a few lines of DCL should do the trick:

$ open/read a 'p1
$ count = 0
$ loop:
$ read/end=all_done a line
$ count = count + 1
$ goto loop
$ all_done:
$ close a
$ write sys$output "There are ",count," records in ", f$parse("''p1'")
$ exit

[Jan van den Ende]

Craig,

I am with you on the GNV solution.

However, for the DCL script, if the input file has some size, it tends to be VERY MUCH slower than the PIPE | search solution by Ken.

However, the DCL works on any version as far as I remember, the PIPE only in 7.2 & up.

(and am __I__ writing this? The DCL aficionado?? Then again, PIPE should now also be considered native DCL, so... )


Proost.

Have one on me.

Jan

[Craig A Berry]

Jan,

No doubt you're right about the speed of DCL reads, but it's something that has always worked and I think it always will work. Parsing the output of SEARCH will probably continue to work, but it's less certain.

Just for something completely different, here's how to get the number of lines in a file into a DCL symbol using a one-line Perl command:

$ perl -"MVMS::DCLsym" -ne "$i++; VMS::DCLsym->setsym('rcnt',$i,'GLOBAL');" test.txt
$ sh sym rcnt
RCNT == "8"

[John Gillings]

SORT is very fast. I've attached a small MACRO32 program that does nothing more than count the records. I'd thought that bypassing the string matching might speed things up a bit, but my program isn't significantly faster than SORT, even for large files.

However, it doesn't need PIPE to catch the record count, so you save yourself some process creations. File is LC.TXT. Rename to .MAR, then MACRO and LINK. Should work on all versions of OpenVMS, VAX, Alpha and IA64.

The interface is foreign command, so define a symbol, use MCR or define DCL$PATH to include the directory containing the image.

Return value is in R0, so it's in $STATUS. To prevent a spurious error, it's left shifted and the low bit is set (so it looks like a success status). To get the count, just divide $STATUS by 2. For example:

$ LC MYFILE.TXT
$ stat=$STATUS
$ IF stat
$ THEN
$ count=stat/2
$ ELSE
$ ! error
$ ENDIF

[Hein van den Heuvel]

An other standard VMS program that is willing to spit out the number of records in a sequential file is copy:

$ copy /log tmp.com nl:
%COPY-S-COPIED, xxxx copied to NL: (5 records)

This eliminates the need for a search or repeated read in the pipe line:

$ pipe copy login.com nl:/log | (read sys$pipe line ; x= f$elem(1,"(",line) - " records)" ; defi/job lines &x )


An other way for to solve this in perl with a logical name is:

$ perl -e "{} while <>; $ENV{'lines'}=$." < tmp.idx
$ show log lines
"LINES" = "2000" (LNM$PROCESS_TABLE)

Pretty clean and flexible. Cheaper than piping.

Or with perl and the return status:

$ perl -e "{} while <>; exit $.*10 + 1" < tmp.idx
$ show symb $status
$STATUS == "%X00004E21"
$ lines =$status/10
$ show symb lines
LINES = 2000 Hex = 000007D0 Octal = 00000003720


Enjoy,
Hein.

[Aleksander Sinigoj]

If you have large files to check, and you consider to use John's solution (macro program) you can speed up it a litle. Replace the following line:
$GET RAB=InRAB
by
$FIND RAB=InRAB.

[Prem Mohan]

Hi All,

First I am overwhelmed with the fast response I got from all of you. Thanks all of you I appreciate it. I would like to test which one of the procedures suggested is not resource intensive and use that

Hi Greetz,

I tried file_length_attributes before I put this question into the forum and it did not work properly.

Hi Volker,

It is working, But I am using the option given by Ken.

Hi Craig/Jan,

GNV is not installed, But I do not want to open the file and do the DCL commands the reason being the files are 200/300 MB and it will take long time to execute.

Hi Hein,

I would not use copy for the simple reason of the file size and I do not have perl installed in the system.

[Ian Miller.]

read a head with multiple multiblock buffers may also speed up the program.

[Hein van den Heuvel]

>> I would not use copy for the simple reason of the file size

Well, my suggestion does not acutally copy the file. The output goes to the null device (NL:). Try it, Time it.

>> 200+ MB, at least we'll know that any pipe overhead will be neglectable.

>> and I do not have perl installed in the system.

Well, you should. QED!
What problem are you _really_ trying to solve?
A number of lines is often interesting, but not the ultimate goal.
You may find that the subsequent processing, is more readily done in perl than anything else.

And if you were to special case a program for this, then indeed the RMS call $FIND will be faster than $GET, and use MBF=120, MBC=4, ROP=RAH, SHR=UPI. It return the byte count.

Be sure to try native C - IO or even COBOL or SORT to read it, because they will NOT use RMS for simple, unshared sequential files. The CPU overhead for going into RMS (Executive inner mode) and back is simply too high when all you do is count records and bytes.

Cheers,
Hein.

[Ian Miller.]

Hien, are you saying some C IO routines will not use RMS sometimes (do they use RMS to open then ACP QIO READ virtual block)?
I thought C RTL always used RMS record I/O but parhaps things have changed.

[Hein van den Heuvel]

Ian,

Yes indeed. I thought it was an opportune moment to point out that it has been recognized that RMS Record IO is some times TOO CPU expensive, due to the CMEXEC, argument probing, and so on. This is all there to support safe file sharing, and multiple file organization. If all you are doing is a simple sequential file read (variable, stream, fixed), then rolling your own using SYS$READ or QIO is more expedient.

The C RTL has recognized this first, next COBRTL (same folks more or less), and of course SORT.

If one really want to do this well, you would want to build in double (or more) buffering for max IO speed, and you even might consider IO_PERFORM for repeated reads fo very large simple files. And ideally you might consider sys$getjpi / getsyi to read default rms buffer settings, in case they are larger then the program defaults (C RTL does this, I happen to have coded the dcl support for this best I remember.).

Hein.

[Ian Miller.]

Hein, which version was this implemented for C and does it apply to read and fgets ?

[Hein van den Heuvel]

Oh, best I know this has been the case 'forever' in DECC and since 1990 or so for VAXC.

Hein.

[Prem Mohan]

Hi,

$ pipe sear/noout/stat "" | sear sys$pipe "Records sear" | -
( read sys$pipe line ; num = f$ele(2," ", f$edi(line,"compress")) ; def /job numl &num )
$ numl = f$trn("numl")
$ sh sym numl


works fine from the command line, But when I use the same into a com file to be executed the output shows 0.

Please tell me what am I doing wrong.
The com file is enclosed for your review.

Best Regards
Prem.

[Jan van den Ende]

Prem,

could it be that the default directory when you run the comfile is NOT the directory where your .PP file is located?
All through the script you speel out the locations of various files (which I HIGHLY endorse! It prevents so many glitches!), EXCEPT for the line:

pipe sear/noout/stat 0103_WAT002616_00.PP "" | sear sys$pipe "Records sear" | -

Check this first, please.

Proost.

Have one on me.

Jan

[Antoniov.]

Prem,
I'm with Jan about default directory.
Just for talking I don't use f$elem function to extract filename, extension and version of file; using of f$parse it's safer than f$elem.

Antonio Vigliotti

[Jan van den Ende]

... and on F$PARSE I have to agree with Antonio!

Proost.

Have one on me.

Jan

[Hein van den Heuvel]

What would happen if a file was coming in during the run? It would be locked no, and give 0 records.

You carefully parse out file names and version numbers, but then search 'the latest' version from a simple name in the current default directory. Ooops!
Surely that is due to using the context from a prior scripts.

Moreover, please consider replacing the convoluted DIR/OUT with temp file work with F$SEARCH and F$PARSE solution. Once you get the hang of that you'll never look back to parsing file names.

Regards,
Hein.

[Prem Mohan]

Hi Jan,Antonio,

Thanks a lot the default directory was the issue, I overlooked this ( too much stress). I added another parameter in the define (def /job/nolog numl &num) command as I was getting this error

%DCL-I-SUPERSEDE, previous value of NUML has been superseded.

thanks a lot.
prem.