1833760 Members
2341 Online
110063 Solutions
New Discussion

RLIMIT_AS vs. VSS

 
jmgallag
Occasional Contributor

RLIMIT_AS vs. VSS

Hi,

We use a 64 bit COTS product on HP-UX 11.31. On occasion, we have found bugs where the vendor's code gets stuck in a loop mmap()'ing memory. Since this is 64 bit, a single process can easily exhaust virtual memory on the server. Because the problem was with mmap()'ed memory, the regular process limits in the kernel (maxssiz_64bit, maxdsiz_64bit and maxtsiz_64bit) do not apply.

In order to limit the problem process, I built a shared library that calls setrlimit(). The shared lib is loaded via LD_PRELOAD and sets RLIMIT_AS. RLIMIT_AS was the only limit I found which could limit mmap()'ed memory.

I have discovered that the RLIMIT_AS set via setrlimit() can be hit, even though the "Total VSS" as reported by glance is substantially lower. For example, I picked 3GB as my RLIMIT_AS. But a VSS of close to 1GB can cause the process to run into my limit and crash.

I am looking for an explanation of how the RLIMIT_AS limit is implemented and how to reconcile the difference between that limit and what glance reports as VSS. I think they should be the same, but that is not the case.

Thanks,
Jim Gallagher

1 REPLY 1
Dennis Handly
Acclaimed Contributor

Re: RLIMIT_AS vs. VSS

>I have discovered that the RLIMIT_AS set via setrlimit() can be hit, even though the "Total VSS" as reported by glance is substantially lower.

 

Perhaps RLIMIT_AS also includes mmaps for shared files or text?

If shared, you or glance would think there is 0 cost.

 

You can write a program to sum up the VM usage of your process by calling pstat_getprocvm(2).