swapdevice does not appear to release - why ?

[Frank de Vries]

We still have a 10.20 box.
Going quite strong , very few incidents.

Apart that the swapdevice sometimes
does not appear to release.
It stays at 10% and slowly goes up until
it reaches 100% and then it you get messages
like
/usr/lib/dld.sl: Call to mmap() failed -
/usr/lib/dld.sl: Not enough space

This time it is not critical and may I through
some stats at you , for analysis ?
Vmstat end swapinfo output
For vmstat the device interrupts (in) and system calls (sy) are very high , is that normal ?

[root@oradb1:]/root<>>> vmstat 5
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
2 0 0 19236 41032 14 7 0 0 0 0 0 1051 427 174 28 6 67
0 0 0 20743 41002 8 0 1 0 0 0 0 1594 5541 1269 10 5 85
1 0 0 21229 41002 0 0 1 0 0 0 0 2827 10222 2761 18 10 72
1 0 0 21864 40986 0 0 0 0 0 0 0 3044 11218 3014 17 10 73
0 1 0 21419 40970 5 0 0 0 0 0 0 3050 11088 2791 18 7 74
0 1 0 21384 40970 0 0 0 0 0 0 0 2840 10026 2455 20 8 72
2 0 0 57550 40893 16 0 0 0 0 0 0 2122 8215 1662 17 5 78
1 0 0 56440 40966 20 0 0 0 0 0 0 1917 6346 1405 11 4 85
[root@oradb1:]/root<>>> swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 524288 63288 461000 12% 0 - 1 /dev/vg00/lvol2
dev 2048000 62368 1985632 3% 0 - 1 /dev/vg00/lvol9
reserve - 524848 -524848
memory 778036 349388 428648 45%
[root@oradb1:]/root<>>>

regards,

[Victor Fridyev]

Hi,

It seems to me, you have a problem with memory leak in one of your applications.
The message you recieve means that mmap - an operation for memory request ( mmap - map pages of memory) completed unsuccessfully. You can try to resolve the problem by updating the kernel (increasing of shmmax and shmmni)

HTH

[Antonio Cardoso_1 · Accepted solution]

In order to help determining which process(es) is forgetting to release memory, you can script something periodically using

export UNIX95=1
ps -ef -o pid,pcpu,sz,vsz,stime,comm,args

Notably check vsz evolution

[Frank de Vries]

Thanks I was able to find the donkey process
with the UNIX95 kipper.
Good advice, need more of that.