Can't allocate enough memory

[Greg White]

Hi:

I am using the ansi c compiler on 32-bit HP-UX 11.0 on a K580 with 4GB of memory and 12GB of swapspace. I do not want to use memory windows and I want to remain in a 32-bit OS. My problem is that no matter what I do, I cannot malloc more than about 700MB. I really need 800MB. There is plenty of memory, maxdsiz is 1GB, and I'm not swapping. Glance shows that there is plenty of memory available and even with almost no processes running on this system, I still can only allocate about 700MB. I've installed the March 2003 Quality Pack but that didn't help.

Any ideas?

Regards, Greg

[A. Clay Stephenson · Accepted solution]

Well, I won't go into why you think you need 3x swap space. My first suggestion would be to go to 64-bit land and forget about these trivial little memory problems or at least go to memory windows and forget (almost) about these trivial memory problems.

If you were running 10.20 then I would say that you've done everything however in 11.x another rule applies that will limit the amount of available dynamic memory - maxssiz. As you increase maxssiz (taken from Q2) you reduce the available Q2 space. My pschic, Miss Cleo, tells me that your maxssiz setting is somewhere around 300MB. 1GB - 300MB = 700MB. Curiously near your limit. If Miss Cleo is right then drop maxssiz to about 64MB (and that is generous) and your problems will disappear.

[Greg White]

Hi Clay:

Miss Cleo was right. Maxssiz is set to 315MB. Our programmers disagree with you and say that maxssiz has nothing to do with the available memory pool for malloc. They also say that 64MB is too small to be safe in highly recursive situations. Do you have any reference that explains why maxssiz limits malloc?


Thanks, Greg

[A. Clay Stephenson]

I very seldom do searches and today ain't one of those times. I'll leave that to others. However the stack and malloc()'s space - the heap - are allocated from Quadrant 2.

If I were you, I would write a baby program and attempt to allocate memory then decrease maxssiz and try again. If the 2nd attempt works then there's your answer - reference or no reference.

As to 64MB not being a big enough stack: It's your code; you can do with it what you like but those guys had better have a VERY GOOD reason to request a stack bigger than 64MB. My normal limit is 32MB and with a lot of persuasion , I'll go to 64MB. Anything above that and I can almost guarantee that you'do doing it wrong.

The exceptions to this are having to link with FORTRAN or Pascal libraries that might pass large amounts of data by value rather than by reference. If this is pure C then data structures need to be rethought. If your recursive functions need large auto data structures then, again, you probably ain't thinking about the problem the right way.

[A. Clay Stephenson]

One more thing Greg,

I should add that even if all the conditions are met, you still may not be able to allocate 800MB because of memory fragmentation. There may simply not be a big enough contigious chunk. In the 32-bit world, I typically only positively, absolutely count on being able to allocate about 500MB though if you play exactly by the rules something approaching 1GB is possible - if you grab the memory when it's available - like soon after a boot. Don't be surprised if you find yourself in the situation that sometimes it works and sometimes it doesn't. I can assure you that it will never work in 32-bit land w/o memory windows, if you don't reduce maxssiz.

You really should think hard about 64-bit land so that you no longer have to think about these problems.

Food for thought, Clay

[Steven E. Protter]

here is a document that explains some of the issues in tuning the kernel.

It provides background to the issues Clay has brought up.

It may prove him correct and your programmers wrong.

Performance Tuning doc

http://www1.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&mode=id&admit=-682735245+1053636168960+28353475&searchCrit=allwords&printable=true


SEP

[Greg White]

I tried the test that Clay suggested and it worked just as he said. It looks like we will need to reduce maxssiz. Our programmers now realize that all they needed to do was pass a pointer to a big struct (about 200KB each) instead of the struct itself so the stack can be much smaller.

Thanks, Greg

[A. Clay Stephenson]

Greg,

You've got to be kidding me. A 200KB struct? The only thing I can think of that would do that is a huge array (or many Mama Bear size arrays) inside the struct. You should really think about dynamic arrays because I can't help but believe that much of the space in each struct is being wasted by planning for the "worst case". The goods news is that changing to pass by reference on 200KB structs should result in very obvious performance boosts.

If I am right about the arrays, I would really, really think about dynamically allocating them.

[Mike Stroyan]

You can use much more data in a 32-bit process. The first step you
should use is to link with cc -Wl,-N or ld -N to create your program as
EXEC_MAGIC. That means that the data area starts in the first quadrant,
just above the a.out text area. That will give you almost 1GB more data
area.

If you need to get larger than that you can use "chatr +q3p enable a.out"
to make the data area extend into the third quadrant. That will mean
that the program will not be able to map in shared memory segments that
were created by other programs in the third quadrant. That limitation
is not usually a problem. (You do need some patches to enable the +q3p
feature. There are plenty of other reasons that you should already have
a new enough quality pack installed.)

The 11.11 release added another switch, "chatr +q4p enable a.out",
which can allow almost 3.75GB of data area in a 32-bit process.

[Bill Hassell]

Your programmers need to read the white papers on memory management and process management found in /usr/share/doc on your 11.0 system. 32bit programs are painfully limited after a few hundred megs. This has always been the case even when the opsystem is 64bits. As mentioned, maxssiz affects multi-quadrant mapping such as EXEC_MAGIC and you'll find similar limits with shared memory allocations. If the programmers insist on staying with 32bit programs, they will have to plan their memory requirements carefully and become familiar with the chatr options for quadrant manipulation.