HP UX PA2.0 64 Bit / malloc failed to allocate 4 GB memory

[scoolj]

As a part of debugging a customer's problem, I wrote a simple program to allocate 4GB of memory, but it failed to allocate.

It is allocating(consistently) only upto 3730.9 MB(3.64 GB).

To overcome the problem, I even tried using -N flg as below, but ld warns and rejects -N flag.

bash-3.00$ cc memtest.c +DA2.0W -N -o ./memtest
ld: (Warning) Option "-N" ignored. Non-shared text is not supported.
1 warnings.

I still remember, I did use -N option on 32 bit binary and benefited with an increased memory allocation.

Can anybody suggest, if I am missing any important process/system/kernel related configuration or what improvement should be done to enable malloc() to allocate >=4 GB memory.

Thanks in advance.
Srini

PS: Some info, may help...

bash-3.00$ getconf KERNEL_BITS
64

bash-3.00$ uname -a
HP-UX unknown B.11.23 U 9000/800 299594728 unlimited-user license

bash-3.00$ ulimit -a
core file size (blocks, -c) 2097151
data seg size (kbytes, -d) 1048576
file size (blocks, -f) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 2048
pipe size (512 bytes, -p) 16
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 257
virtual memory (kbytes, -v) unlimited
bash-3.00$

bash-3.00$ kctune | egrep "max|64"
aio_listio_max 256 Default Immed
aio_max_ops 2048 Default Immed
aio_prio_delta_max 20 Default Immed
dbc_max_pct 50 Default Immed
fr_statemax 800000 Default Immed
fr_tcpidletimeout 86400 Default Immed
ftable_hash_locks 64 Default
io_ports_hash_locks 64 Default
ksi_alloc_max 33600 Default Immed
ksi_send_max 32 Default
max_acct_file_size 2560000 Default Immed
max_async_ports 50 Default
max_mem_window 0 Default
max_thread_proc 256 Default Immed
maxdsiz 1073741824 Default Immed
maxdsiz_64bit 4294967296 Default Immed
maxfiles 2048 Default
maxfiles_lim 4096 Default Immed
maxssiz 8388608 Default Immed
maxssiz_64bit 268435456 Default Immed
maxtsiz 100663296 Default Immed
maxtsiz_64bit 1073741824 Default Immed
maxuprc 256 Default Immed
maxvgs 10 Default
msgmax 65535 65535 Immed
nsysmap64 8400 Default
scsi_max_qdepth 8 Default Immed
scsi_maxphys 1048576 Default
sendfile_max 0 Default
shmmax 1073741824 Default Immed
vol_max_bchain 32 Default
vol_max_nconfigs 20 Default
vol_max_nlogs 20 Default
vol_max_nmpool_sz 4194304 Default Immed
vol_max_prm_dgs 1024 Default
vol_max_rdback_sz 4194304 Default Immed
vol_max_vol 8388608 Default
vol_max_wrspool_sz 4194304 Default Immed
vol_maxio 256 Default
vol_maxioctl 32768 Default
vol_maxkiocount 2048 Default
vol_maxparallelio 256 Default
vol_maxspecialio 256 Default
vol_maxstablebufsize 256 Default
vol_mvr_maxround 256 Default
vol_rvio_maxpool_sz 4194304 Default Immed
voldrl_max_drtregs 2048 Default
voliomem_maxpool_sz 4194304 Default
voliot_iobuf_max 65536 Default
voliot_max_open 32 Default
volpagemod_max_memsz 6144 Default Immed
volraid_rsrtransmax 1 Default
vx_maxlink 32767 Default
vxtask_max_monitors 32 Default
bash-3.00$


bash-3.00$ swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 182 3914 4% 0 - 1 /dev/vg00/lvol2
reserve - 155 -155
memory 2046 964 1082 47%
total 6142 1301 4841 21% - 0 -
bash-3.00$


bash-3.00$ vmstat
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
1 0 0 79428 242083 135 18 10 49 2 0 75 494 2474 270 4 1 95
bash-3.00$ iostat

device bps sps msps

c2t0d0 0 0.0 1.0
c4t0d1 0 0.0 1.0

bash-3.00$

[Don Morris_1]

You're almost hitting maxdsiz_64bit -- but are enough lower that I don't think that's the problem (it is 4Gb exactly in your output).

Look at:

bash-3.00$ swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 182 3914 4% 0 - 1 /dev/vg00/lvol2
reserve - 155 -155
memory 2046 964 1082 47%
total 6142 1301 4841 21% - 0 -

That malloc requires swap reservations -- and at the time you ran swapinfo you have very little remaining and only 6Gb across the system in any event. I can easily imagine this big malloc hits swap reservation failure -- and hence malloc() fails. Either add more swap or look at the chatr man page -- see if "+dz" is there and how to use it to enable Lazy Swap for the malloc allocation. Note: That would allow you to do the malloc -- but if you actually go to _use_ the space, you'll reserve the swap then.. and if that fails you get killed. So that may not solve your problem.

Longer term and more robustly, you need more memory or more swap (or both).

[scoolj]

Thanks Morris. I will try your suggestions.

I am curious about why -N should be ignored, it worked well with 32 bit programs earlier.

Is it because a 64 bit program has more memory access than 32 bit, MAGIC etc. are applicable only to 32 bit ?
Or is it because I am using -N for a wrong type of program.

bash-3.00$ cc memtest.c +DA2.0W -N -o ./memtest
ld: (Warning) Option "-N" ignored. Non-shared text is not supported.
1 warnings.

Rgds,
Srini

[Don Morris_1]

Yes, that's exactly why -N and other address space layout flags are 32-bit only. 64-bit has one and only one address space layout -- which give you 4Tb on PA to work with for data/stack/private objects. As far as I know, no one has needed 8Tb (which is what a 64-bit EXEC_MAGIC equivalent would be), and I'd seriously expect if they asked for it the answer would be to move to IPF anyway (there's even more architecturally on 64-bit IPF..).

ld_pa(1) covers this with the section of options that are for PA-RISC 32-bit only [which is where -N is listed].

[Bill Hassell]

I have attached a simple program that can demonstrate all the 32bit and 64bit features. On a system with 24GB of swap (and maxdsiz_64 = 40GB), I compiled it with the 64bit option and it easily malloc'ed more than 23GB of virtual memory -- oh, the system had 512 megs of RAM. As you'll see from the comments, 32bit compiler options are quite involved to reach more than 3GB but 64bit is easy.

[Bill Hassell]

Oops, here's the source code attached...

[scoolj]

Hi Morris, Hassel

Thanks much.

Br,
Srini

[scoolj]

Thanks.