Tuning parameter maxdsiz HP-UX 11.11

[Andrew_adm]

Hi,
I would like to know if it´s necessary to use ld -N command on application executable after tune the parameter maxdsiz with a value near to 1,9 GB. I´ve read this is necessary on some systems S700 with HP-UX 10.20, but i haven´t seen nothing similar about 11.11. One more thing, is any patch needed ?. I inform you i´m talking about a N4000 server.
Thx, and best regards.

[A. Clay Stephenson · Accepted solution]

This is one of those "it depends" answers. If the application is a 64-bit application then you need to increase maxdsiz_64bit. If the application is a 32-bit application then maxdsiz is the applicable tunable. 10.20 only supported 32-bit applications. Normally, a 32-bit applications were restricted to not crossing a 1GiB quadrant so that the -N compiler/loinker options changed this default behavior. You should also note that 32-bit applications allocate both data and stack from the same quadrant so often in an effort to make a program "bigger" maxssiz was increased -- which had the unforeseen consequence of limiting the dynamic memory allocation. You should set maxssiz to no more than 64MiB (and that is very generous).

In any event, the correct approach is to recompile your application as a 64-bit executable and all of these restrictions disappear.

[Bill Hassell]

Yes, you won't be able to use the first quadrant unless your executable is compiled as an EXEC_MAGIC program. This has always been true for all platforms, 10.xx and higher. For 32 bit programs, you'll only get about 960 megs from the data quadrant without changing to an EXEC_MAGIC binary. Compiling with -Wl-N, you'll get 1800 to 1900 (varies) because your program will be mapping malloc calls into quadrant 2 and then into quadrant 1, the text quadrant.

To get more than 1900 megs, you can take your existing EXEC_MAGIC executable and use chatr to map the 3rd quadrant into the data area for malloc, and with 11.11 (only), you can map into quadrant 4 for a maximum of about 3800 megs. The chatr options are +q3p and +q4p. The attached program will demo this capability as well as show the 11.23 MPAS option for easier mapping as well as 64bit capability.

[Kevin Wright]

I would say no, especially as I am assuming your app is 64 bit.

perhaps what you read on 10.20/S700 is regarding 32 bit apps?

[Steven E. Protter]

Shalom,

Don't forget maxdsiz_64 if the appliccation is 64 bit.

SEP

[Alzhy]

Bill, et. al,

Does this mean by changing the execution quadrant of my 32bit executable, it can now malloc(?) memory beyond the 1.9 or so GB that it can in its default quadarant? Even if maxdsiz stays at 1.9GB?

We're still grappling with issues in our NBU system's inability to restore as a single job - a filesystem with about 7 million files whose average file paths are well in excess of 1024 bytes (chars) and 7-9 levels deep...

[Bill Hassell]

> Does this mean by changing the execution quadrant of my 32bit executable, it can now malloc(?) memory beyond the 1.9 or so GB that it can in its default quadarant?

Yes.

> Even if maxdsiz stays at 1.9GB?

No. You do need to increase the fence to 2500 or even 3500 megs. BUT: maxdsiz_64 *must* be larger than maxdsiz or the lower of the two becomes the 32bit limit.

> We're still grappling with issues in our NBU system's inability to restore as a single job - a filesystem with about 7 million files whose average file paths are well in excess of 1024 bytes (chars) and 7-9 levels deep...

This may be a shared memory issue rather than local memory. fbackup finds all the filenames and stores the pathnames in shared memory so all the reader processes can get to the pathnames quickly. The same is true for frecover. If NBU is only a 32bit executable, then both the local memory (malloc and maxdsiz) as well as shared memory have limits. Shared memory is even more restrictive as it is, well, shared. That is, there is one map and every little use (semaphores, shared memory segments) can fragment the map.

The solution for local memory is as stated...use chatr to increase the available map for NBU. But if shared memory is the problem, now you'll need to adjust shmmax and possibly look at using a memory window to obtain your own local map for shared memory.

[Alzhy]

Thanks Bill,

I don't think Shared Memory is the problem... we have SHMMAX set to 12GB and this is a 24GB machine...

Changing the execution quadrant is a reversible process right?

Can you please state once more the chatr syntax please?

Andrew, apologies for hijacking your post...

[Bill Hassell]

No problem. The attachment a few posts back has all the combinations. The chatr options can be enabled or disabled as they are just modifying the front-end of the code (see man chatr for +q3p and +q4p)

chatr +q3p enable a.out
chatr +q3p disable a.out

and so on. Compile the mini program in the attachment to see the different behaviors.

[Andrew_adm]

Thank you very much, sorry the delay on the reply, i was out some days.Best Regards.