topic Re: Obtaining only filenames from a wildcard search in Operating System - OpenVMS

Obtaining only filenames from a wildcard search

John D McLean — Mon, 19 Dec 2005 22:17:28 GMT

I would like to obtain the names of files which contain a specified character string so that I can process only those files. (eg. find the names of all .DAT files containing a specific string so each of those files can then be updated. Okay, maybe not the most brilliant example but it gives you the general picture.)

It really requires finding only the first instance of the string within the file and then saving the filename and terminating the search.

I'd prefer not to use SEARCH/OUTPUT=OUT.TXT *.C because the output file might be huge and that's not attractive when I'm working in DCL.

I'd also prefer not to use F$SEARCH("*.*") and feed each file to SEARCH/OUTPUT=NL: (etc.) then examine the $SEVERITY because in my case this would mean about 850 activations of the SEARCH.EXE image.

Ideally I want a SEARCH/INSTANCES=1/OUTPUT=... etc. (which stops after finding the first instance) and then I could use the contents of the output file for the subsequent processing. This would minimise the use of cpu, I/O and disk-space.

Any suggestions ?

Thanks i a,

John

Re: Obtaining only filenames from a wildcard search

Steven Schweda — Mon, 19 Dec 2005 22:39:12 GMT

I'd use a loop with F$SEARCH(), and not worry
about activating SEARCH.EXE. (If it's not
already INSTALLed, I'd change that.)

_If_ that were too slow, then I'd write my own
program(s).

Re: Obtaining only filenames from a wildcard search

John D McLean — Mon, 19 Dec 2005 23:40:43 GMT

Steven - I said that I would prefer not to use that method. Apart from the activations, image if the desired string was located in record 1 of a 10 000 record file? SEARCH would rather inefficiently (for my needs) continue to examine the file even after finding the string in record 1.

Maybe I could write a program and there are ways to make it fast (eg. use $QIO to read logical blocks because I don't need to know the exact location) but my preference is to use some existing and efficient tool if one can be found.

John

Re: Obtaining only filenames from a wildcard search

Steven Schweda — Mon, 19 Dec 2005 23:59:05 GMT

> I said that I would prefer not to use that
> method.

I was trying to overlook that part.

So far as I can see, unlike DIFFERENCES, with
its /MAXIMUM_DIFFERENCES=n option, SEARCH
appears too diligent, and won't quit early.
Overcoming that feature shortage would appear
to require a fresh program. (I don't know of
an existing one, but that's not a
nonexistence proof by any stretch.)

I tend toward an incremental approach on
these sorts of tasks, writing something
which works, and then trying to speed up the
pieces where needed, as needed. Thus, I'd
start with a rather simple search-like
program, and make the I/O fancier later, as
needed. With all the multi-buffer,
read-ahead, and so on stuff accessible with
some ease through the C RTL, I'd tend to
avoid excessive complexity (read: "work")
until it was forced upon me.

If you want to do a client-server system with
one process looking up the file names while
another does the searches (with some fancy
inter-process comm scheme), be my guest. I'm
sure it could save a lot of image
activations, but I don't know how much that
would actually buy you. (And I know that
it'd be more work.)

Re: Obtaining only filenames from a wildcard search

Hein van den Heuvel — Tue, 20 Dec 2005 01:20:45 GMT

Uh... why not just use SEARC/OUT=OUT.TXT/WIND=0 *.c

only when you are dealing with Gigiabyte sized files woudl I worry about a little extra time spend reading the whole file. a few thousand line more or less surely go unnoticed as long as your have a reasonable RMS default block size. 16KN new default, or chose your own: SET RMS/BUF=4/BLO=120

But if you feel you need to stop after the first match (not unreasonable I might add), how about perl? :

$ perl -e "foreach $f (<*.c>) { open(F,""<$f""); foreach () { if (/string/) {print ""$f\n""; last;}}}"

hth,
Hein.

Re: Obtaining only filenames from a wildcard search

Volker Halle — Tue, 20 Dec 2005 01:45:39 GMT

John,

there now (as of OpenVMS V8.2) is a SEARCH/LIMIT=n qualifier, which
limits the number of matches displayed to the number specified by n.

Volker.

Re: Obtaining only filenames from a wildcard search

Wim Van den Wyngaert — Tue, 20 Dec 2005 01:59:47 GMT

I thought of proposing freeware grep but a test revealed that grep -l did read the complete file.

Wim

Re: Obtaining only filenames from a wildcard search

Jan van den Ende — Tue, 20 Dec 2005 05:35:22 GMT

John,

Let me start with:

WELCOME to the VMS forum!

From Volker's answer:

there now (as of OpenVMS V8.2) is a SEARCH/LIMIT=n qualifier, which
limits the number of matches displayed to the number specified by n.

Please be informed that this functionality also exists in V7.3-2, since one of its earlier patches.

using /LIMIT=1 pretty much seems to do what you are asking.

So, it becomes of interest which VMS version you are using...

fwiw

Proost.

Have one on me.

jpe

Re: Obtaining only filenames from a wildcard search

Volker Halle — Tue, 20 Dec 2005 06:10:22 GMT

Jan,

there never was an OpenVMS V7.3-2 patch including SEARCH.EXE, which I would have expected, if the /LIMIT qualifier would be supposed to work. I also tried SEARCH/LIMIT on a V7.3-2 system including VMS732_UPDATE-V0400 and it didn't work (%DCL-W-IVQUAL).

John,

time to upgrade to V8.2

Volker.

Re: Obtaining only filenames from a wildcard search

Jan van den Ende — Tue, 20 Dec 2005 08:28:33 GMT

Volker,

just tested it, and you are right.

This is strange, though. I vividly remember Guy Peleg announcing it, both in Nashua at the bootcamp, and again during the Dutch TUD, in the session on DCL enhancements.
And and if my memory is not REALLY derailed, this, and several other SEARCH additions, were also in 7.3-2.
--- but I clearly never had any specific uses for it yet ---

So, this DOES make a point for backporting this to 7.3-2 ASAP. After all, it IS still supported, and will stay on PVS for some time to come.
HP, listening in anyone?

Proost.

Have one on me.

jpe

Re: Obtaining only filenames from a wildcard search

Hein van den Heuvel — Tue, 20 Dec 2005 08:58:11 GMT

I don't have an 8.2 system handy to verify this, but I suspect that this /LIMIT discussion is a moot point.

The manual indicates it only limits the display.

I expect the /STAT to indicate that the whole file was read and counts the actual matched, not just the /limit, much like /window=0 behaves.

John's desire, reasonable or not, was to stop reading after a match. My perl solution will do just that.

Cheers,
Hein.

Re: Obtaining only filenames from a wildcard search

Volker Halle — Tue, 20 Dec 2005 09:04:11 GMT

Hein,

SEARCH/LIMIT=1/STAT indicates (as compared to the same SEARCH/STAT without /LIMIT), that Records searched is actually much smaller, as well as CPU time used and Elapsed Time.

Guy seems to have done a good job ;-)

Volker.

Re: Obtaining only filenames from a wildcard search

Hein van den Heuvel — Tue, 20 Dec 2005 09:07:27 GMT

Thanks for verifying Volker!
So the documenation could have been better.
It was not wrong, it just could have been better.
Hein.

Re: Obtaining only filenames from a wildcard search

John D McLean — Tue, 20 Dec 2005 18:05:01 GMT

Thanks Volker, Jan & Hein

It's nice to know that someone else has had the same problem and addressed it at some stage. The qualifier /LIMIT might not be commonly used but it will be perfect for some operations.

This system is on 7.3-2 with 19 patches with name prefix of "VMS732_" but SEARCH/LIMIT still fails.

If anyone can determine which (if any) 7.3-2 patch the change was in I'd like to be told.

Otherwise since I'm only a contractor here I'll need to start pushing for a VMS upgrade. That might not be easy because there's a plan - you know the story - but if they want the functionality here it seems like the upgrade is necessary.

Thanks again,

John

Re: Obtaining only filenames from a wildcard search

Karl Rohwedder — Wed, 21 Dec 2005 01:17:51 GMT

John,

if an upgrade is not possible at the moment, pls. have a look at the FIND utility. You can find it on the freeare CD, it is a kit from HP and is installed ising PCSI.

It is a combination of search and optional replace and has a /LIMIT qualifier.

regards Kalle

Re: Obtaining only filenames from a wildcard search

John D McLean — Wed, 21 Dec 2005 17:29:14 GMT

Thanks Kalle, I'll check out the Freeware with the software distribution kits or, as a last resort, I'll download the ZIPped CD.

I'll leave this thread open for another 24 hours for further comments. I think everything has been covered but maybe someone can surprise me.

John

Re: Obtaining only filenames from a wildcard search

Lawrence Czlapinski — Thu, 22 Dec 2005 13:06:07 GMT

I have Alpha 7.3-2 Update 5 installed plus latest patches. SEARCH/LIMIT gives:
CVTS1Â» sea/limit=1 "login" *.com
%DCL-W-IVQUAL, unrecognized qualifier - check validity, spelling, and placement
\LIMIT\
Lawrence

Re: Obtaining only filenames from a wildcard search

John D McLean — Thu, 22 Dec 2005 16:35:18 GMT

Thanks all.

To summarize for any future reader, SEARCH/LIMIT=1 looks the best way to go but it only became available with VMS 8.2. I'm on a 7.3-2 system so I need to either get VMS upgraded or use the FIND program on the Freeware.

John