Huge data seg size and stack size

[Jordi Adame]

Hi!I need to change the values of data seg size and stack size to HUGE (GBs) values, currently I have the max allowed by SAM.newton:~$ ulimit -adata seg size (kbytes, -d) 3802048stack size (kbytes, -s) 392192In linux I had to recompile my kernel in order to have a 2.7 GB stack size, is there any way to achieve this in HPUX?

[Bill Hassell]

If your program is not written and compiled as a 64bit program, you have a lot of reading to do about the severe limits of 32bit programs (64bit programs have terabyte limits). 32bit programs in HP-UX have 4 quadrants of 1Gb each. For the data area, you are limited to about 950megs before you overflow the second (data) quadrant. With compile options, you can the first quadrant (text or program code) together with the second quadrant to address about 1.75Gb. Beyond 1.75Gb, the answer depends on your version of HP-UX and your patches.

Get a copy of the memory management and process management white papers in /usr/share/doc (10.20 and 11.0 systems). It will be virtually impossible to get a 2.7b stack. To which I ask: why would any sane program require more than a few megs for a stack?

[Jordi Adame]

Its actually compiled as 64bit. So I dont have the 32 bit limit.I really need the 2.7 GB stack, this programs loads LOTS of entries from a DB into memory.I'll read the white papers, thanx

[Jeff Schussele]

Hi Jordi,

You need to adjust the following kernel parms

maxdsiz_64bit
maxtsiz_64bit

they will support the sizes you need. The ones w/o the _64bit will not.

Rgds,
Jeff

[Jordi Adame]

already tried that.
maxdsiz_64bit 4396972769279
maxssiz_64bit 1073741824
maxtsiz_64bit 107374182
Thos params are at their top, but i need even more :(RegardsJordi

[Jeff Schussele · Accepted solution]

Hi Jordi,

The absolute maximums are roughly
maxdsiz_64bit => 4 Terabytes
maxtsiz_64bit => 439 Gigabytes

I'd hope you don't need more than that ;~))

So, you're probably hitting a SAM problem.
Try the latest:

11.11 SAM cumulative - PHCO_25870

http://www2.itrc.hp.com/service/patch/patchDetail.do?BC=patch.breadcrumb.search|&patchid=PHCO_25870&context=hpux:800:11:11

or

11.0 SAM cumulative - PHCO_25902

http://www2.itrc.hp.com/service/patch/patchDetail.do?BC=patch.breadcrumb.search|&patchid=PHCO_25902&context=hpux:800:11:00

whichever OS you're running.
Or set it/relink the kernel manually.

HTH,
Jeff

[Jordi Adame]

I relinked my kernel manually editing /stad/build/conf.c Its actually like thismaxssiz 401604608\n\maxssiz_64bit 1073741824\n\If I change maxssiz_64 to anything bigger, mk_kernel will change it to 1073741824.Yet, ulimit -s gives me 392192Is this a HPUX limitation??? (hpux 11.11)thanx by the way, how do i add new lines to a post?

[Don Morris_1]

Are you running a 64-bit shell? If not - I would imagine you'd see the 32-bit limit from ulimit, not the 64-bit one. Alternately, the shell profile may be setting a lower limit.

Try the following program to be sure:

# cat rlim.c
#include
#include

int
main(int argc, char *argv[] )
{
struct rlimit rl;

if ( getrlimit(RLIMIT_STACK, &rl) ) {
perror("rlimit stack get ");
} else {
printf("Stack max limit %ld Mb.\n",
rl.rlim_max / 1024 / 1024);
}

exit(0);
}

# cc +DD64 -o rlim64 rlim.c
# ./rlim64
Stack max limit 256 Mb.
# kctune -u -s maxssiz_64bit=0x40000000
WARNING: The automatic 'backup' configuration currently contains the
configuration that was in use before the last reboot of this
system.
==> Do you wish to update it to contain the current configuration
before making the requested change? no
NOTE: The backup will not be updated.
* The requested changes have been applied to the currently
running system.
Tunable Value Expression Changes
maxssiz_64bit (before) 268435456 Default Immed
(now) 0x40000000 0x40000000
# ./rlim64
Stack max limit 1024 Mb.
# kctune -u -s maxssiz_64bit=0x10000000
WARNING: The automatic 'backup' configuration currently contains the
configuration that was in use before the last reboot of this
system.
==> Do you wish to update it to contain the current configuration
before making the requested change? no
NOTE: The backup will not be updated.
* The requested changes have been applied to the currently
running system.
Tunable Value Expression Changes
maxssiz_64bit (before) 0x40000000 0x40000000 Immed
(now) 0x10000000 0x10000000
# ./rlim64
Stack max limit 256 Mb.

[maxssiz_64bit isn't dynamic on 11.11, I just wanted to show you that this should be correct.]

Regarding getting the limit up to 2.7Gb - you should be able to do this if you modify /usr/conf/master.d/core-hpux [change the line with *range maxssiz_64bit<=1024*1024*1024 to *range maxssiz_64bit<=3*1024*1024*1024 ] and /usr/conf/space.h.d/system_space.h [change 0x40000000 in the maxssiz_64bit tunedef to 0xC0000000]. NOTE: This is *not* an official endorsement of doing this from HP, no promise of support, etc. (Speaking just for myself here). After you do this you can use SAM/mk_kernel/whatnot. If you apply a patch that touches these files, you'll lose these changes -- so you're warned.

Getting to the root of the problem though -- hasn't the author of this program heard of dynamically allocating memory instead of using stack storage? I've heard of Fortran apps needing this much (due to Fortran assumptions) and Java goes to town sometimes... but if this is C code the program should really malloc the storage it copies these DB entries into. You'd then just be limited by data -- which should be doable.

# kmtune -q maxssiz_64bit
Parameter Current Dyn Planned Module Version
===============================================================================
maxssiz_64bit 0xb0000000 - 0XB0000000
# uname -a
HP-UX vm59mhk B.11.11 U 9000/889 237824391 unlimited-user license
# ./rlim64
Stack max limit 2816 Mb.

[Jordi Adame]

Thanx for all the information!!!I was in fact using a 32-bit shell, how can i use a 64-bit shell instead??thank yourgds, Jordi

[Don Morris_1]

You'd have to recompile the shell itself to get a 64-bit one. I don't know of any widely available (mainly because shells don't tend to need so much memory, etc.) to require recompilation to 64-bits. The main thing is that as long as you don't modify the ulimit within the shell first, the 64-bit program should get the 64-bit limit... not what ulimit tells you for the 32-bit shell. The rule (very generally) is that the limits are always inherited from the parent if the parent (or an ancestor of the parent) changed them. Otherwise, they use the system defaults... which for a new 64-bit process with a 32-bit parent means the 64-bit tunable.