topic Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800 in Operating System - HP-UX

Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Mon, 24 Nov 2008 23:49:33 GMT

Hi,

My application is trying to allocate an array of size 953773 where each array element is 104 bytes each. However, the new operator throws an error.

The output of memprog on this machine is:
Memory Stat total used avail %used
physical 36288.0 13705.0 22583.0 38%
active virtual 12030.6 882.7 11147.8 7%
active real 2617.2 730.6 1886.5 28%
memory swap 28019.2 7420.9 20598.2 26%
device swap 32768.0 5982.5 26785.5 18%

Can anybody please advise how to investigate this problem further?

Thanks in advance.

-Shiv

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Mon, 24 Nov 2008 23:53:04 GMT

What is maxdsiz?
What does "swapinfo -tam" show?

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

James R. Ferguson — Tue, 25 Nov 2008 00:20:44 GMT

Hi Shiv:

> However, the new operator throws an error.

It's always nice to specify an error number ('errno') though in this case your failure is likely to be either insufficient swap space or too small a 'maxdsiz'. If swap space is sufficient look at 'maxdsiz' if your process is a 32-bit one. Otherwise, if your process is a 64-bit one, look at the value of 'maxdsiz_64bit'.

Regards!

...JRF...

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Tue, 25 Nov 2008 00:35:40 GMT

Hi,

Here's the output of swapinfo -tam:
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 38 8154 0% 0 - 1 /dev/vg00/lvol11
dev 8192 38 8154 0% 0 - 1 /dev/vg01/lvol12
dev 16384 0 16384 0% 0 - 2 /dev/vg01/lvol17
reserve - 5993 -5993
memory 28019 7421 20598 26%
total 60787 13490 47297 22% - 0 -

How do I find the maxdsize for 64 bit?

-Shiv

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Tue, 25 Nov 2008 00:37:16 GMT

>JRF: It's always nice to specify an error number

The error thrown from the default operator new, bad_alloc, which means malloc returned NULL.
But that doesn't distinguish between the maxdsiz limit or out of swap space.

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Tue, 25 Nov 2008 00:40:03 GMT

>total 60787 13490 47297 22%

You have plenty of swap. Did you run this at the same time as your application abort?

>How do I find the maxdsiz for 64 bit?

kmtune | fgrep maxdsiz

If your process is 64 bit and you are running out of space, you have major problems.

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

James R. Ferguson — Tue, 25 Nov 2008 00:46:38 GMT

Hi:

> Dennis: But that doesn't distinguish between the maxdsiz limit or out of swap space.

Yup, I certainly didn't say that well --- only that in this case the errno is probably ENOMEM and that indicates "either".

I was trying to point out that descriptions like "throws an error" are too general when good coding techniques can report the exact error which, in turn, then can be uplifted to good problem descriptions.

Regards!

...JRF...

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Tue, 25 Nov 2008 00:53:36 GMT

The swapinfo -tam output when my application aborted:

Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 38 8154 0% 0 - 1 /dev/vg00/lvol11
dev 8192 38 8154 0% 0 - 1 /dev/vg01/lvol12
dev 16384 0 16384 0% 0 - 2 /dev/vg01/lvol17
reserve - 5991 -5991
memory 28019 7421 20598 26%
total 60787 13488 47299 22% - 0 -

kmtune output is:
$ kmtune | grep maxdsiz
maxdsiz 1073741824 - 1073741824
maxdsiz_64bit 4294967296 - 4294967296

-Shiv

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

James R. Ferguson — Tue, 25 Nov 2008 00:58:25 GMT

Hi (again) Shiv:

Is your process a 32-bit one?

Regards!

...JRF...

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Tue, 25 Nov 2008 00:59:06 GMT

Its a 64 bit process.

-Shiv

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Tue, 25 Nov 2008 01:12:36 GMT

>JRF: I certainly didn't say that well

You said it well, I'm saying that libc and the aC++ runtime only tells you "black".

>I was trying to point out that descriptions like "throws an error" are too general

In this case "throws an error" is EXACTLY what the aC++ runtime does. But it only throws the color "black", bad_alloc.

> ShivS: The swapinfo -tam output when my application aborted:
>total 60787 13488 47299 22%
>maxdsiz 1 Gb
>maxdsiz_64bit 4 Gb

This says you are not out of swap, so the only answer is that you are very greedy. :-)
You are using more than 1 Gb or 4 Gb for a 64 bit executable.

You may want to look at the top(1) output as your application runs.

If you need more than 1 Gb, you can increase maxdsiz to 4 Gb and link with -N, to get to 2 Gb.

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Tue, 25 Nov 2008 01:15:04 GMT

>It's a 64 bit process.

(You have compiled with +DD64?)
Then you need to increase maxdsiz_64bit. Do you have any idea how much memory you need? You could try 8 Gb.

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Tue, 25 Nov 2008 16:27:20 GMT

>Dennis Handly: If you need more than 1 Gb, you can increase maxdsiz to 4 Gb and link with -N, to get to 2 Gb.

Can you please tell me all the commands?

>JRF: (You have compiled with +DD64?)
>JRF: Then you need to increase maxdsiz_64bit. Do you have any idea how much memory you need? You could try 8 Gb.

Yes, I have compiled with +DD64. I can try 8 GB.

Regards,
Shiv

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

Dennis Handly — Tue, 25 Nov 2008 19:39:10 GMT

>Can you please tell me all the commands?
>I have compiled with +DD64. I can try 8 GB.

You go into SAM and increase maxdsiz_64bit and it will reboot.

Re: Unable to allocate memory, HP-UX B.11.11 U 9000/800

ShivS — Tue, 25 Nov 2008 20:01:55 GMT

Yes, I increased the maxdsize_64bit to 8 GB and this has solved my problem.

Thanks a lot for all your help.

Regards,
Shiv.