Mapping to global sections with FORTRAN and C

[Scot Newton]

We have a legacy FORTRAN product that makes extensive use of global sections. We now have a process written in C that we want to be able to map to the existing global sections.
When we attempt to map, we get error 4042 - invalid argument. BTW, we are running on an Integrity server.
Anyone seen this problem before? Know of any examples of using the mgblsc system service from C to map to an existing global section created by a FORTRAN program via the crmpsc system service?

[Steven Schweda]

I know nothing, but I read that SS$_INVARG
implies this:

Invalid argument specified to service. Common
sources are the incorrect specification of
relpag or the values in the inadr array.


Not being able to see any of what you're
doing (in Fortran or C) makes it hard for a
non-psychic like me to discern much about
what you might be doing wrong.

[Hoff]

The usual cause of an "%SYSTEM-E-INVARG, invalid argument" invalid argument error is, well, an invalid argument. A bad parameter on the failing call.

See the discussion of the SS$_INVARG error in this context in the Programming Concepts documentation materials located here:

http://h71000.www7.hp.com/doc/82final/5841/5841pro_041.html

As for examples, those are (among other places) here:

http://h71000.www7.hp.com/wizard/swdev/gblsec.c

http://h71000.www7.hp.com/wizard/wiz_2486.html

http://www.eight-cubed.com/examples/framework.php?file=sys_gs64.c

[Robert Gezelter]

Scot,

As Steven mentioned, the only way to identify what is happening is to review the sources.

- Bob Gezelter, http://www.rlgsc.com

[John Gillings]

Scot,

Taking a wild guess, the most likely suspect is your inadr argument to $CRMPSC and/or $MGBLSC, defining the range of addresses to be mapped. On older systems the interpretation was fairly lax. Newer systems require the addresses to be down to the byte on the starting page and ending page.

$ help system $mgblsc arg

...

inadr

OpenVMS usage:address_range
type: longword (unsigned)
access: read only
mechanism: by reference
Starting and ending virtual addresses into which the section is to be mapped. The inadr argument is the address of a 2-longword array containing, in order, the starting and ending process virtual addresses. Only the virtual page number portion of each virtual address is used to specify which pages are to be mapped; the low-order byte-within-page bits are ignored for this purpose.

The interpretation of the inadr argument depends on the setting of SEC$M_EXPREG in the inadr argument and whether you are using an Alpha, an I64, or a VAX system. Refer to the HP OpenVMS System Services Reference Manual for a complete description of these differences.

...

Run your program under DEBUG control and STEP/INTO the system service, then examine your argument list to make sure you're sending exactly what you think you're sending. My bet it you aren't.

I'd also strongly recommend you build an access module around your global section, rather than access the naked bits. Be especially careful about data alignment. Examine compiler listings (use cross reference maps) to confirm that the compilers really have placed fields at the same offsets.

You also MUST (absolutely, positively, without exception) synchronize access to the data. I don't care if you believe you have only one writer and that makes it all OK, all data directly shared between processes MUST be synchronized correctly. The lock manager is simple to use and will protect you from having to debug weird, impossible to reproduce, timing bugs.

[John McL]

Please post, preferably as an attachment that retains the original formatting, the relevant extract of your code. It might also be worth mentioning the values of any parameters. (This could be as little as a few lines or if there's relevant code just before your system service call that might blow out to 15 or 20 lines. Don't attach what's not relevant to your problem.)

BTW, did you just cut and paste the Fortran code, then make a few tweaks before you tried running with it? You might be hitting pass-by-reference and pass-by-value conflicts.

[Scot Newton]

Hi,

Thanks to all. Your input proved useful. I fixed the arguments, but now encountered a different problem. I spawned both images (a Fortran and a C executable) in debug mode, and looked at the values. The RETADR match the INADR of the FORTRAN program, meaning $CRMPSC system call is successful.

The INADR of C match the one in Fortran, but RETADR is meaningless, and $MGBLSC returns exit code 12 (access violation).

FYI, the INADR arrays of both the Fortran and C contain the same values.

The condensed source codes of both programs are below. My apologies for code alignment issues due to window sizing in the reply window:


PROGRAM A (Fortran)

definitions.....

common /xyz/ abc,def,ghi

flgl = sec$m_wrt .OR.
sec$m_gbl .OR.
sec$m_dzro .OR.
sec$m_perm .OR.
sec$m_pagfil .OR. sec$m_wrt

maprange(1) = %loc(abc)
maprange(2) = %loc(ghi)
i = lib$getsyi (%loc(syi$_page_size),
CPU_PAGE_SIZEL,,
SIZEOF(CPU_PAGE_SIZEL),,)
templ = mod (maprange(2),
CPU_PAGE_SIZEL)
if (templ .ne. 0)
1 maprange(2) = maprange(2) +
1 (CPU_PAGE_SIZEL - templ - 1)
sizel = (maprange(2) - maprange
1 (1)) / 512 + 1

i= SYS$CRMPSC (maprange, retrange,
0, %val(flgl), 'BSCOM', ,
%val(0), ,%val(sizel), , ,)
abc=10
def=20

loop...do something...

end


PROGRAM B (C program)

#include
...other includes....

#pragma extern_model common_block
struct xs
{
unsigned long int abc;
unsigned long int def;
unsigned long int ghi;
};

extern struct xs xyz;

unsigned long int starange[2];
unsigned long int maprange[2];
char *gsdnam = "BSCOM";
unsigned long int flag = SEC$M_GBL |
SEC$M_WRT | SEC$M_PAGFIL |
SEC$M_DZRO | SEC$M_PERM;
...other definitions.....

starange[0] = (unsigned long int)
&xyz.abc;
starange[1] = (unsigned long int)
&xyz.ghi;

templ = (starange[1] % 8192);
if (templ != 0){
starange[1] = (starange[1] +
(8192 - templ - 1));
}
sizel = ((starange[1] - starange
[0]) / 512 + 1);

statl = SYS$MGBLSC (&starange,
&maprange, 0, flag, *gsdnam,
0, 0);

printf ("Status for MGBLSC is
%d\n",statl);
printf ("This value is from fortran
%d\n", xyz.abc);

return;

[Joseph Huber_1]

Without checking all details,
one obvious error in the C code:
gsdnam must be passed by descriptor, but the code passes a pointer to a C string.

Change it to:
#include descrip

$DESCRIPTOR(gsdnam, "BSCOM");

Then pass &gsdnam as parameter to $crmpsc.

[Hoff]

That C code looks vert much like Fortran code, coded in C. (Been there, done that, and have the battle scars.)

8192? Either hard-code 64K alignment, or grab the page size value SYI$_PAGE_SIZE of the running system from sys$getsyi. I don't know if there are currently plans to change the page size, but there were certainly various similar recommendations over the years; there were various recommendations to avoid hardcoding alignments to the 8 KiB page size.

And I'd probably use a #define to represent that 8192 value, or the 65536 value, regardless. Be kind to whomever will support this code.

from pagedef.h:

#define MMG$C_VAX_PAGE_SIZE 512 /* SIZE OF VAX PAGE */
#define MMG$C_ALPHA_MAX_PAGE_SIZE 65536 /* MAX. PAGE SIZE */
$

And as for alignment and rounding, something akin to:

size_rounded = ((value + 8191) & 8191);
addr_rounded = value & 8191;

also works. ( With 8192 or with MMG$C_ALPHA_MAX_PAGE_SIZE, or...)

Use pointers and position-independence to access the contents of the global section where you can. Fortran does those now, as does C. Using COMMON_BLOCK means you have to align virtual memory across processes, and maintaining code that does that gets interesting.

#include

#define __NEW_STARLET to 1 if you want to turn on better argument-passing and parameter diagnostics, too.

IThe other approach is to avoid the $creprc and (if you're using a COMMON), just link the COMMON reference stuff directly into the image. That's more or less what this does:

http://h71000.www7.hp.com/wizard/wiz_2486.html

And yes, you need a descriptor for the section name, and a $VMS_STATUS_SUCCESS( status_value) test or a direct low-bit test to verify that the $crmpsc call actually worked is recommended.

[SAMI AHMAD]

stupid question: Hoff the example you gave at http://h71000.www7.hp.com/wizard/swdev/gblsec.c

what is it achieving ? I am trying to understand why global sections are used in programming , where can i find some basic info about it ?

thanks
Sami

[Jon Pinkley]

Sami,

Hoff's code is an example program that isn't meant to do anything useful. The purpose is to show how to get the syntax correct for people that are having problems with the basics.

Global sections are used to allow multiple processes to access the same physical memory, in ether read-only mode or read write mode.

An example of what VMS uses global sections for are for programs that are installed /shared. This allows multiple processes to share the physical memory that is holding the code (read only) portion of the program. See $ help install add /shared

There are very few times I would recommend end users using global sections. The o/s needed them, the work was already done, and when VMS was written, memory was EXPENSIVE and in general the programmers were much more interested in the nitty-gritty details of the O/S. Many programmed in VAX assembly language. The source code for VMS was even shipped to customers on micro fiche. So the APIs were documented, and people were able to use them as well as the operating system.

Having shared writable memory and maintaining correct synchronization to it is not trivial. As others have said, RMS indexed files with global buffers work quite well and the hard work has already been done and debugged.

The HP OpenVMS Programming Concepts Manual is the place to start if you want some basic info.

http://h71000.www7.hp.com/doc/os83_index.html (index of O/S documentation)

HTML version of manual

http://h71000.www7.hp.com/doc/82FINAL/5841/5841PRO.HTML

PDF (two parts)

http://h71000.www7.hp.com/doc/82FINAL/5841/aa-rnshd-te.PDF (part 1)

http://h71000.www7.hp.com/doc/82final/5841/aa-pv67h-tk.PDF (part 2)

This stolen directly from chapter 3 of HP OpenVMS Programming Concepts Manual:

"Global sections: Global sections can be either disk files or page-file sections that contain shareable code or data. Through the use of memory management services, these files can be mapped to the virtual address space of more than one process. In the case of a data file on disk, cooperating processes can synchronize reading and writing the data in physical memory; as data is updated, system paging results in the updated data being written directly back into the disk file. Global page-file sections are useful for temporary storage of common data; they are not mapped to a disk file. Instead, they page only to the system default page file. Global sections are described in more detail in Chapter 13 and Chapter 12."

Jon

[Hoff]

Cynical view follows:

Global sections and shared memory and Fortran COMMON structures are used by various ancient or misguided folks and rarely by a few and skilled folks as an approach to share application data in memory, and sometimes in the usually false belief that the use of these shared memory structures will provide performance benefits over more modern designs as compared with the maintenance and support overhead.

These older models and structures are also used by those looking to sabotage a software project schedule or the software product stability, or to seek a form of perpetual employment.

Locally-written shared memory solutions often to provide a means to inject obscure and application-destabilizing and subtle run-time errors. This with matters of processor caching and proper memory synchronization. And a means to introduce limits to application scaling, too.

Better approaches include databases and RMS files with global buffers enabled. They're also more portable.

If you're working for me and you use a shared global section, you're going to expect be in my office having a design discussion; you're going to need justification for using what is usually budget- or schedule-busting programming "solution." Can this sharing design be implemented successfully? Sure. Are you willing to read and heed the architecture- and processor-level manuals for each box you're working with? There are alternatives here, and often vastly superior better alternatives. And the best part: alternatives that deal with the caching and careful writes and updates and such that somebody else is going to maintain and upgrade, too.

There are "libraries" around that allow you to share data far more effectively.

[John Gillings]

Having successfully implemented systems that use shared memory data structures, I can confirm most of what Hoff says. You can get a performance advantage, BUT only when there is some kind of specific property in your data that can be exploited. If you're not sacrificing generality to gain performance, you're re-implementing RMS and you won't win!

That said, the key to writing code that is solid and dependable is to package your shared data structure in a procedural interface that makes sense at application level. It should not reveal any detail of the lower level implementation. By forcing all accesses through a common interface, you can enforce synchronisation, and change internal details when and if necessary.

Start by instantiating the interface using the simplest mechanism. Most likely RMS. Once you have the application working, implement the same mechanism using (say) global sections and locks. This should not require any changes at application level. All you're doing is replacing the RMS implementation. If this is done correctly, it will be possible to plug in other mechanisms, like networked client/server, or move your portable application level code to other architectures or operating environments.

If you're exposing and directly accessing the "naked" data structures directly in your application code, you're doomed to failure.

[Hoff]

What John G. is referring to in his reply is known as Model View Controller (MVC) on other platforms.

Or as "premature optimization", depending on the particular antecedent.

[Robert Gezelter]

Scot,

I agree with Hoff and John.

Quite a while ago, a client "asked" me to use shared global sections to achieve "efficiency". For a variety of reasons, most of which had to do with integrity in the face of software component malfunction and time to completion, I used DECnet Task-to-Task logical links to implement what is now generally referred to as MVC.

Despite management concerns, the performance of the system was never a problem, even on processors far slower than todays (first generation Alphas), and at transaction volumes orders of magnitude larger than ever occurred.

The resulting product was also so stable that it was used for production as a Beta.

A good object lesson.

- Bob Gezelter, http://www.rlgsc.com

[Hein van den Heuvel]

Shared global sections are often used for message passing. Very nice, when done right, but as pointed out, a well designed RMS solution may be plenty fast. And maybe you can use a tool like SYS$ICC which has done the dirty work for you (John may dissagree :^).

That said, shared global sections are great for certain easy task where even a poor implementation is good enough. Like a 'shut down flag'. A data lookup table, or a 'please-re-read config file flag. Or perhaps application throughput and usage counters where a dirty write is acceptable. Yes 'locks' and ast's may be better for some of those, Logical name have been used, but a shared global section has its place. Such lookup table can be protected with a single simple CR lock, where an RMS indexed file would typically involve much more locking unless very careful (GBC, NQL, MSE,..)

Enjoy!
Hein.

[Hoff]

> Yes 'locks' and ast's may be better for some of those, ....

Or for basic process states and for flags-oriented stuff, a common event flag cluster and event flags can be used; the sys$ascefc system service call and related routines.

[John Gillings]

>And maybe you can use a tool like SYS$ICC
>which has done the dirty work for you (John
>may dissagree :^).

Now that OpenVMS engineering has fixed all the bugs I know of in ICC, I won't disagree! ICC is a magic mechanism. Cluster transparent and very fast. However, if you use it, make sure you have all the latest patches (including the ones which should hit ITRC in a couple of weeks...)

[Scot Newton]

Thanks to all for the assistance. Apologies for closing the thread so late!