how to limit resident set size of a process

[blackwater]

Hi,

I wonder if there is a way to limit resident set size of a process on HP-UX 11.23 or 11.31?
My 64 bit program allocates for its caches lots of memory using malloc(). As a result its size is about 10 Gb according to the top command. I think it is useful to set a certain limit on the size of resident set of the process (for example 5GB) so that other applications also have enough free memory.

So my questions are:
1) If limiting the resident set size is necessary for archiving this goal?
2) How can I restrict the resident set size of the process?

[Steven E. Protter]

Shalom,

If I understand your question, limitation on process memory use can be done with the kernel settings on the system.

http://docs.hp.com/en/939/KCParms/KCparams.OverviewAll.html

See maxdsiz and maxsiz

I don't see anything that permits an adjustable hard limit on allocation, but with some work and research you might be able to get the job done.

If this involves shared memory, some manipulation may work there as well. Note, though this can have a negative impact on other applications.

Specifics:
1) No, I would set the limits within your application code.
2) This is in my opinion an issue best dealt with in the application. I would see the documentation on malloc and see if you can code in a limit.

SEP

[blackwater]

Steven, thank you for your reply.

Let me clarify my situation.

First, it doesnâ t involve shared memory.

Second, I have already thought of setting maxdsiz_64bit but as far as I understand it is not what I want. If I set it to let say 5 GB then I will not be able to allocate in total more memory than 5 GB. But it is not that I want. I want to let the application allocate as much memory as it needs (but no more than maxdsiz_64bit bytes) but also I want its resident set size (measured using the top command) to be lower than a certain level so that other applications have enough memory. So, for instance if the size of the process is 12 GB and the limit of resident set size is 5 GB then 7GB of the process is in the swap.

As for your suggestion about limiting within the application code it is really feasible as it will require rewriting a lot of the application and changing the way its caches operates. Moreover I believe is should be sort of OS feature.

[blackwater]

This sentence has not been posted properly:

First, it does not involve shared memory.

[Don Morris_1]

The only way to do something like this (physical limits below [especially well below] the virtual limits for a process / process group) would be to use Process Resource Manager (PRM) or functionality that sits on top of it (I believe WLM does, for one). This should enable you to set a physical memory cap for all private data of the process [there is no interface that lets you say "I want only X Gb applied only to malloc"] and/or a cap on shared data as well.

[blackwater]

Don, thanks a lot.

Seems like PRM memory resource groups might be useful. I need to read more about them to be certain about it.

[Dennis Handly]

>1) If limiting the resident set size is necessary for archiving this goal?
>2) How can I restrict the resident set size of the process?

Just let the kernel handle this. HP-UX will dynamically adjust the resident set size depending on the memory pressure.

[Venkatesh BL]

Yes, as Dennis mentioned, kernel will take care of it when there are more users of physical memory. Over time, the RSS would come down if the process is not actively touching those pages.

[blackwater]

Dennis, Venkatesh thank for your response.

Just would like to ask about that memory pressure. Ho do I measure when it is high and when it is low? My current understanding is that I should use the vmstat command. If the column po is low then the memory pressure is low. If there are a lot of page outs during work of my application and other applications and what even worse their number keep rising then probably I should think of increasing amount of RAM. Am I right?

[Dennis Handly]

>My current understanding is that I should use the vmstat command. If the column po is low then the memory pressure is low.

Yes, or use glance.

>If there are a lot of page outs during work of my application and other applications ... then probably I should think of increasing amount of RAM. Am I right?

Yes.
Don is the VM guru. I'm surprised he misunderstood your question. He assumed you were asking about mafia hit men, instead of traffic cops. ;-)

[blackwater]

OK, thanks!

[blackwater]

I have closed the thread since a solution has been suggested. I mean using vmstat.

[Dennis Handly]

You can also call pstat_getproc(2) to get:
pst_rssize resident set size for process (private pages)
pst_maxrss highwater mark for proc resident set size