HPUX 11.31 automatic process killing

[eran borovik]

Hi,
Does anyone knows whether HPUX 11.31 kernel has an "enhancement" which tries to kill processes when the memory pressure is high (Similar to the notorious Linux killer daemon)?
I'v seen some strange issues in my lab where in the kernel stack I saw the function:process_deact_action

Thanks,
Eran.

[Duncan Edmonstone]

Eran,

I'm no kernel engineer... but - AFAIK there's no significant change in behaviour in low memory situations in 11iv3... processes certainly aren't *killed* - in extreme situations where available memory falls below minfree they might be deactivated (prevented from running so vhand can mark pages for paging out and let more deserving processes get some memory/CPU time)

What specifically are you seeing?

HTH

Duncan

[eran borovik]

Hi,
Thanks for your response. Indeed, further examination showed that my process (And bunch of others) are being deactivated. Now, my process consumes alot of memory, but this memory is locked (Using mlock), so there isn't any point in deactivating it.
My questions:
1.Is it possible it is a kernel bug when it tries to deactivate a process with a lot of mlocked pages sinc there is no point in deactivating such a process.
2.Is there a possibility to mark a process as not eligible to deactivation by vhand.

Best regards,
Eran.

[Armin Kunaschik]

What do you expect the kernel do do?
This is how virtual memory works. And you see this only if your system is under heavy(!!) memory pressure!
Do you know what you could do to HP-UX if anyone could deactivate this feature? This system would simply thrash, that is paging without doing anything else. And your application is to blame! If I would be a customer I would no accept such behavior... any maybe buy another's product.
If you need such amounts of memory, make sure, it's available!


My 2 cents,
Armin

PS: Assign points if you find answers useful!

[eran borovik]

Hi,
Thanks for the response.
I would like to clarify. Although my process consumes locked memory, by default the amount of memory is not very big. We are just testing edge conditions in our lab. The issue that I see is that the HPUX kernel tries to deactivate my process when it won't do it any good!! The pages are mlocked therefore vhand cannot swap them.

Thanks,
Eran.

[Dennis Handly]

>Although my process consumes locked memory, by default the amount of memory is not very big.

Which memory have you mlocked? Your stack, data/heap and text?

>The pages are mlocked therefore vhand cannot swap them.

Well, hopefully nothing happens other than you don't run. Are you under CPU pressure too?

[eran borovik]

Hi,
I'v mlocked memory returned from shmat/shmget (Anon memory).
I am not under CPU pressure but under memory pressure. From some reason or another, my process is deactivated and never activated back. I have never seen such behaviour on previous HPUX versions. I am now trying to update some patches to see if they solved the problem.

Thanks,
Eran.

[Dennis Handly]

>I've mlocked memory returned from shmat/shmget (Anon memory).

(Without IPC_GLOBAL?)
If you don't mlock every stinkin' page you own, the kernel will likely deactivate you to get at them.

>I have never seen such behaviour on previous HP-UX versions.

Not even when memory pressure is reduced?
What behavior were you expecting or got on previous versions?

[eran borovik]

Hi,
I didn't specify IPC_GLOBAL.
Here is the call:
shmid = shmget(IPC_PRIVATE,sizeInBytes,IPC_CREAT | IPC_EXCL);

According to HP's white paper on memory management, it seems that processes that owns mlocked memory shouldn't be deactivated at all. Nevertheless, are you implying I need to mlock every "stinking" page?
The memory pressure is never reduced because my application is the biggest memory gobbler!! Other HPUX versions just left my process alone and didn't try to deactivate it at all. In 11.31, my process remains deactivated and was never activated.

Thanks,
Eran.

[Dennis Handly]

>According to HP's white paper on memory management, it seems that processes that owns mlocked memory shouldn't be deactivated at all.

Hmm, that's what it says. I'm not sure if it has changed for 11.31.

>are you implying I need to mlock every "stinking" page?

I was but the white paper contradicts me.

>In 11.31, my process remains deactivated and was never activated.

It looks like you need to contact the Response Center.

Have you looked at patches? PHKL_37185 mentions mlock and "may cause machine hangs during high memory pressure situations".

[eran borovik]

Thanks,
I now installed the patch you said, and plenty others (Including a VM patch). Hopefully, it will resolve the issue.
Will let you know.

Many thanks,
Eran.

[Armin Kunaschik]

Beeing not a HP-UX developer, I can only suspect a reason for this...
You lock your data, but do you lock the code too (if this is possible)? Code pages are subject of paging too...

Sorry to recommend that, but maybe you should raise a call to HP.

My 2 cents,
Armin

[Dennis Handly]

>Armin: You lock your data, but do you lock the code too (if this is possible)? Code pages are subject of paging too...

(Subject to paging in.) That was my thought. It still could happen but the process shouldn't be deactivated. It could be waiting for the needed code, stack, data/heap to be swapped in.

Anyone who is going to use mlock better make sure there is enough pages for the kernel, all other mlocked pages and text/data for itself.

[eran borovik]

Hi all,
Applying patches PHKL_37185 and PHKL_37184 seems to have solved the problem.
Thanks everyone for your help!!

Regards,
Eran.

[eran borovik]

Thread is closed as the patches solves my issues.