topic Re: GNV: make hitting command line maximum length in Operating System - OpenVMS

GNV: make hitting command line maximum length

Maxy2 — Fri, 24 Jul 2009 04:02:59 GMT

My makefiles (make, not MMS) seem to be exceeding the maximum command line length. For example:

OBJECTS = file1.obj file2.obj file3.obj (etc.)

.PHONY: clean
clean:
$(RM) $(OBJECTS)

Once OBJECTS gets to be a certain length (a few lines long, around 1000 characters or so), clean won't work. I've tried invoking make from bash and from DCL, and the same thing happens. Using line continuation characters inside the makefile doesn't help, although if the command is written out to a .com script with line continuation characters and then invoked, it works fine.

The release notes for the VMS port of make seem to imply that make is supposed to do something like that (i.e. create a temporary script and invoke it) automatically to circumvent the maximum line length limit.

Any other suggestions?

Re: GNV: make hitting command line maximum length

John Gillings — Fri, 24 Jul 2009 04:23:39 GMT

Maxy2,

Yes DCL is showing its age with absurdly low limits on things like command lines and tokens. Drives me nuts having to code complex workarounds

If you're a high enough version of OpenVMS (at least V7.3) you can try extending your limits with:

$ SET PROCESS/TOKEN=EXTEND

Unfortunately this doesn't work for everything. For example, parameters to batch jobs and RSH commands are still stuck at 255 characters maximum :-(

Even the increased limits only takes it up to 4000 bytes, which is still rather low for generated commands.

If the extra long lines are a result of expanded file specification, you may be able to reduce them using (short) concealed logical names.

Re: GNV: make hitting command line maximum length

H.Becker — Fri, 24 Jul 2009 07:30:16 GMT

>>>
although if the command is written out to a .com script with line continuation characters and then invoked, it works fine.

The release notes for the VMS port of make ...
<<<

GNV make is old. It indeed creates and uses a temproary command file.

If you use it from DCL (the .obj indicate you are) then you may want to replace GNV make with the current version of GNU make, which is V3.81. There are some fixes in handling long action lines. They are split and written to command procedures and then executed under the control of make.

Also, newer versions keep the temporary command file, if you use the -d switch. But I would check for CMD.COM in sys$scratch or the default directory, anyway. They may be left in case of an error and they may give you some hints why this command failed.

I never tried GNU make V3.81 within GNV/bash: I have no idea whether it works or not.

Re: GNV: make hitting command line maximum length

Maxy2 — Fri, 24 Jul 2009 12:01:40 GMT

I downloaded make 3.81 and followed the build instructions. It seems to run for trivial examples, but for anything real I get a bus error:

$ run make.exe
makefile.:1: common.mak: no such file or directory
makefile.:54: handledepends.mak: no such file or directory
gmake.exe;1: *** No rule to make target `handledepends.mak'. Stop.
%NONAME-E-NOMSG, Message number 00000002

$ bash

bash$ gmake.exe
makefile.:1: common.mak: no such file or directory
makefile.:54: handledepends.mak: no such file or directory
gmake.exe: *** No rule to make target `handledepends.mak'. Stop.
%NONAME-E-NOMSG, Message number 00000002

bash$ make -f zlib.mak all
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=0000000000000064, PC=0000000000067E70, PS=0000001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
MAKE IMPLICIT pattern_search 13157 0000000000002BE0 0000000000067E70
MAKE IMPLICIT try_implicit_rule 13112 00000000000000D2 0000000000065362
MAKE REMAKE update_file_1 27566 0000000000001AA2 000000000005BEE2
MAKE REMAKE update_file 27420 0000000000000D72 000000000005B1B2
MAKE REMAKE update_goal_chain 27267 0000000000000432 000000000005A872
MAKE MAIN main 15155 00000000000038F2 0000000000048682
MAKE MAIN __main 14172 0000000000000212 0000000000044FA2
0 FFFFFFFF80BA8192 FFFFFFFF80BA8192
DCL 0 000000000006BD22 000000007AB8FD22
%TRACE-I-END, end of TRACE stack dump
Bus error

Re: GNV: make hitting command line maximum length

H.Becker — Fri, 24 Jul 2009 14:54:57 GMT

>>>
$ run make.exe
makefile.:1: common.mak: no such file or directory
makefile.:54: handledepends.mak: no such file or directory
gmake.exe;1: *** No rule to make target `handledepends.mak'. Stop.
%NONAME-E-NOMSG, Message number 00000002
<<<

I assume that is a run of GNU make, but I do not understand, why the started exe is make and the error message is reported by gmake.
Anyway, you seem to get the same error when running gmake in bash.

What's on line 1 and 54 and where are these *.mak files?

Where can I download your zlib kit?

Re: GNV: make hitting command line maximum length

Maxy2 — Fri, 24 Jul 2009 16:54:01 GMT

The error messages from make are correct. I was running make on a makefile in a directory where the required include files do not exist. This is just to show that make is starting and parsing the makefile. But run make in the proper directory and all I get is a bus error.

Zlib is available from zlib.net and the VMS make for it was written by Martin P.J. Zinser.

You can see "gmake" in one place because I forgot to change it. In order to not conflict with the GNV make, I renamed my build of make to gmake.

Re: GNV: make hitting command line maximum length

Maxy2 — Fri, 24 Jul 2009 17:18:40 GMT

From make's README.VMS:

New for 3.77:

Command lines of excessive length are correctly broken and written to a
batch file in sys$scratch for later execution. There's no limit to the
lengths of commands (and no need for .opt files :-) any more.

So, how come there does seem to be a limit?

Re: GNV: make hitting command line maximum length

H.Becker — Sat, 25 Jul 2009 15:40:16 GMT

I'm slow, I don't get it.
>>>
OBJECTS = file1.obj file2.obj file3.obj (etc.)

.PHONY: clean
clean:
$(RM) $(OBJECTS)
<<<
illustrates the error for exceeding the command line, it's not really an extract of any makefile you tried.

>>>
bash$ make -f zlib.mak all
<<<
Seems to be a makefile for zlib, but that is not in the zlib123.zip I fetched from zlib.net. However, there is a Makefile and GNV make seems to have no problem to build from that. GNU make for VMS reads it but has no implicit rules for .o, as it is built for VMS: it only has implicit rules .obj.

In my zlib kit there is also a make_vms.com which generates a descrip.mms for MMS or MMK. GNV make or GNU make can't read it, there are spaces where make expects tabs. Once these are changed (and after doubling $ in sys$output and creating the libz.olb) GNU make can process this input file with no problem other than that a link fails due to a missing a missing x11vms:xvmsutils.olb.

What/where is this zlib.mak and what should it show regarding to the base problem?

>>>
New for 3.77:

Command lines of excessive length are correctly broken and written to a
batch file in sys$scratch for later execution. There's no limit to the
lengths of commands (and no need for .opt files :-) any more.

So, how come there does seem to be a limit?
<<<
There was at least one report in 2003 or 2004 that command line splitting was broken for some action lines. Without knowing and seeing the failing action line it is difficult to say what the problem is and how to avoid it or work around it.

For a test, I created a very long action line which was correctly handled by GNU 3.81 but was not by GNV make.

Re: GNV: make hitting command line maximum length

Maxy2 — Mon, 27 Jul 2009 11:56:47 GMT

> I'm slow, I don't get it.

I'm sorry. I'm leaving out too much information.

>>>>
>OBJECTS = file1.obj file2.obj file3.obj (etc.)
>
>.PHONY: clean
>clean:
>$(RM) $(OBJECTS)
><<<
>illustrates the error for exceeding
>the command line, it's not really an
>extract of any makefile you tried.

No. This IS an extract of a makefile that I tried. It isn't the only instance where I found this problem, but it is the cleanest most self-contained example from among the ones I encountered.

> Seems to be a makefile for zlib, but that is not...

The makefile that I was trying to run for zlib is not the one that comes with zlib. I have run the MMS file for zlib. I have made a my own makefile from the MMS file that DOES run with GNV make (i.e. make version 3.78.1 that comes with GNV). I downloaded the source for make 3.81 and built it myself following the VMS building instructions. However, using this build, this same makefile that works with make 3.78.1 throws a bus error. There must be something not quite right with this build of make 3.81 that I built myself. I would love to get a copy of a built version of make 3.81, because I'm obviously having some trouble buildng one myself.

> For a test, I created a very long action
> line which was correctly handled by GNU
> 3.81 but was not by GNV make.

This is great news, and all the more reason why I'd love to get my hands on a working make 3.81 build.

Re: GNV: make hitting command line maximum length

Joseph Huber_1 — Mon, 27 Jul 2009 12:40:01 GMT

If Your system is an Alpha VMS 7.3-1 or later, get my executable make 3.81
http://mpiw10/vms$common/exe/make.exe

Re: GNV: make hitting command line maximum length

Maxy2 — Mon, 27 Jul 2009 12:51:33 GMT

Thanks, but our system is Itanium OpenVMS 8.3.

Re: GNV: make hitting command line maximum length

Maxy2 — Mon, 27 Jul 2009 15:25:01 GMT

It looks like I left out a line that is relevant. Here is a complete makefile that shows two problems:

SHELL = /gnu/bin/bash
MYFILES = file01.txt file02.txt file03.txt file04.txt file05.txt \
file06.txt file07.txt file08.txt file09.txt file10.txt \
file11.txt file12.txt file13.txt file14.txt file15.txt \
file16.txt file17.txt file18.txt file19.txt file20.txt \
file21.txt file22.txt file23.txt file24.txt file25.txt

.PHONY: test1 test2
test1:
touch $(MYFILES)

test2:
DCL show time

I had the SHELL line in my example before, but I didn't think it was relevant. Turns out it is. In this makefile, test1 fails and test2 works. If you take out the SHELL line, test1 works and test2 fails, returning an error that stops make from proceeding:

bash$ make -f support4.mak test2
DCL show time
27-JUL-2009 12:21:04
/gnu/bin/make: *** [test2] Error 1
bash$

I had no idea that my fix for test2 actually caused of the problem with test1.

Re: GNV: make hitting command line maximum length

Maxy2 — Mon, 27 Jul 2009 15:38:37 GMT

Sorry, one more thing. The reason why there is an Exit 1 error from make is because the DCL command (all of them) are returning an error code 1.

bash$ ls hello.c
hello.c
bash$ echo $?
0
bash$ dcl show time
27-JUL-2009 12:35:31
bash$ echo $?
1
bash$

Re: GNV: make hitting command line maximum length

Hoff — Mon, 27 Jul 2009 15:44:36 GMT

Um, OpenVMS I64 V8.3 *is* newer than V7.3-1.
You'll want to look at the update.

Re: GNV: make hitting command line maximum length

Maxy2 — Mon, 27 Jul 2009 16:35:30 GMT

I was just assuming that any executable built for Alpha would not run on Itanium.

Re: GNV: make hitting command line maximum length

Hoff — Mon, 27 Jul 2009 17:02:24 GMT

That wasn't my reading of that note. You're correct that you do need a platform-specific executable, but I'd confirm that the port of make isn't applicable in your environment. I'd expect the intent of the comment was not that Alpha was central, but that the port required some feature of OpenVMS that first arrived with V7.3-1.

If you're using gnv, you're going to want to rummage around for (newer) pieces. What's in gnv is often a little stale, and there can be updates and fixes around. John Malmberg posted various updates to gnv over at the Eisner host, for instance. Though the gnv bits have moved (and there are references at the following), start reading here:

http://eisner.encompasserve.org/~malmberg/GNV/

There's been an associated and running discussion about gnv and about other open source on OpenVMS in general, too. (It'd be nice to see the latest gnv bits available out at sourceforge, for instance.)

Re: GNV: make hitting command line maximum length

H.Becker — Tue, 28 Jul 2009 09:38:06 GMT

As said in the other thread, the dcl utility has a bug, it should not return a VMS success status.

Using GNV make from bash ...

If you have the SHELL specified, your action line for test2 really is $(SHELL) -c 'dcl show time'. So you get the 0 returned from shell and make sees success.

The same applies to your action line for test1, now you have $(SHELL) -c 'touch file01.txt file02.txt file03.txt file04.txt file05.txt ...'

That fails in my environment with
make: execv 2: /gnu/bin/bash: no such file or directory
make: execv 2: -c: no such file or directory
make: execv 2: touch file01.txt file02.txt file03.txt

At the moment I can't explain the behavior of GNV make, here.

It looks like you need the SHELL only to work around the bug in the dcl utility. So you may want to use /gnu/bin/bash -c 'show time' for such actions. (However, fixing the bug in the dcl utility seems easier :-)

Using GNU make 3.81 ...

GNU make for VMS will not use the specified SHELL. In my environment it works for test1 And if you remove the dcl in the other action line, it works for test2, too. Both from bash and (the real) DCL.

I do have zipped GNU make 3.81, built on V8.3 on Integrity, about 400 disk blocks. Let me know where to copy it or if you want me to attach it, here.

Re: GNV: make hitting command line maximum length

Maxy2 — Tue, 28 Jul 2009 12:38:47 GMT

Some things are finally starting to become clear. The reason defining SHELL improved my situation was because of the issue with DCL return codes. This really only masked the situation, and created another problem of severely limiting the command line length. So, to summarize:

DON'T define makefile SHELL variable. Even if correctly defined, this will severely limit the maximum command line length.

DO define an alias for make with the full path (i.e. alias make=/gnu/bin/make). Otherwise make can't properly re-execute itself when needed.

DCL commands don't return zero on success. Therefore, if you use them in a makefile, either instruct make to ignore the return value with "-" or invoke something after the DCL command that will return a zero on success, like:

DCL DIR /FULL | grep MAKEFILE

or
DCL DIR MYFILE.TXT ; dcl_return $?

where dcl_return is:

#!/gnu/bin/bash
# Converts DCL return value into bash return code
# In DCL, odd value means success. In bash, 0 value means success.
# If DCL failure (even), return the DCL return value.
if [ $(( $1 & 1 )) -eq "1" ]
then
exit 0
else
exit $1
fi