Is there any limitation on memory a process can use?

[Prathima]

I have enabled the 3rd quadrant on my program using the chatr options, enabling 2700 GBs of memory.

But still the program fails after allocating 1900 GB.

maxdsiz limit is at 4GB.
maxdsiz_64 is at 8 GB.

Is there anything else that I need to look at to provide the entire 2700 GB of memory?

[Steven E. Protter]

Shalom,

I think these figures are wrong.

You should not be able to exceed 8 GB on a 64 bit program and 4 GB on a 32 bit.

Lets say you are actually failing at 1900 MB, thats pretty close to the limit for a 32 bit program based on compiler memory windows.

Is it a 32 bit program? Details please.

SEP

[A. Clay Stephenson]

Since you have a 64-bit OS version, the obvious fix to to compile/link your program as a 64-bit executable and these barriers disappear. If you are restricted to a 32-bit environment then you have to compile and link with -N and also use the +q3p or +q4p chatr options. There may still be a "gotcha" in that your maxssiz is too large. Because data and stack segments are allocated from a common quadrant, increasing maxssiz instantly decreases the maximum data size by maxssiz -- even if the run-time stack never approaches maxssiz. Only in extremely rare cases does maxssiz ever need to exceed 32MiB. There is really nothing special about child processes because all processes except init are child processes. You might be limited by your current ulimit value. In any event, the real answer to your question is to convert to 64-bit and all of this nonsense disappears.

[Prathima]

Steve - Yes my Program is compiled as 32-bit program.

Stephensen,
My ulimit also shows 4GB for the data. I was also thinking whether to compile the program as 64-bit, but wanted to check, whether there could be any other parameter at the kernel level limiting the memory to 2 GB.

Why is it failing at 1900 GB when it can use 2700? So I thought probably some other parameter is restricting the memory to 2GB.

I used the +q4p option the program is still running, I could see that it has crossed the point where it was failing initially.

[Prathima]

Stephenson,

Value is 0X800000 for both maxssiz and maxsiz_64. Is this the reason why thought 2700 GB is available for data, only 1900 is available to the program?

[Patrick Wallek]

No.

0x800000 = 8,388,608 bytes = 8 MiB

[A. Clay Stephenson]

Your maxssiz id set to 8MiB which is about the smallest reasonable value so that is not your problem. The +q4p chatr option should suffice but unless you have an extremely compelling reason (such as no 64-bit application library is available) then you are far better served by creating a 64-bit executable.

[Prathima]

With +q4p, the program failed at 2.75GB.

I still dont get, why my program isnt using the available 3700 GB to it.

[A. Clay Stephenson]

I don't know why your code cannot exceed the 2.75 GiB barrier but then again, I didn't write your code.

Try this code, mem.c:

---------------------------------------------
#include
#include
#include
#include

#define assign_errno(x) ((errno != 0) ? errno : (x))

static void problem(char *msg, int err)
{
(void) fprintf(stderr,"\n%s (%d)\n",msg,err);
(void) fflush(stderr);
return;
} /* problem */

int main(int argc, char *argv[])
{
int cc = 0;
long m = 0;
size_t sz = (10 * 1024 * 1024);
char *p = (char *) (NULL + 1);

while ((cc == 0) && (p != NULL))
{
p = malloc(sz);
if (p != NULL)
{
m += 10L;
(void) printf("%7ld MiB\n",m);
}
else
{
cc = assign_errno(-1);
problem("malloc failed",cc);
}
}
return(cc);
}
-------------------------------------------

Compile, link it like this:

cc -N +DAportable mem.c -o mem

Now execute it (and I assume that your maxdsiz is set to 4GiB, your ulimit -m value is unlimited, and that your maxssiz is set to 32MiB or less). I also assume that you have sufficient virtual memory space (memory + swap) to allow for a 4GiB address space.

You should get a memory allocation somewhere near 1900 MiB.

Now execute chatr on the executable file.
chatr +q4p enable mem

Execute mem again and you should get a value very near 3800 MiB.

[Dennis Handly]

If you look at the fine print in both of Clay's replies, you need -N to get quadrant 1. And +q3p and +q4p for quadrants 3 and 4. And of course you get quadrant 2 for free.