topic Re: Checking kernel memory usage in Operating System - Linux

Checking kernel memory usage

Paul Thomson_2 — Fri, 07 Mar 2008 12:02:38 GMT

Hi

I believe I may have a memory issue / leak on my machine.

I am trying to find out where this is, user memory areas or more specifically, kernel memory usage ?

/proc/meminfo / sar -r and vmstat don't really reveal this.

The machine is redhat 4u4
Kernel is 2.6.9-42.ELsmp

Re: Checking kernel memory usage

Paul Thomson_2 — Fri, 07 Mar 2008 13:49:04 GMT

Hello, I think the correct value needed is in /proc/meminfo, which totally contradicts what I wrote earlier.

Slab shows kernel usage.

Re: Checking kernel memory usage

Vitaly Karasik_1 — Tue, 11 Mar 2008 07:13:40 GMT

why do you think that you have a memory leak?

with "ps" or "top" you have monitor application's memory usage.

Re: Checking kernel memory usage

Paul Thomson_2 — Tue, 11 Mar 2008 08:27:10 GMT

top / ps aux for example don't really show an application process consuming lots of memory.

But when I monitor /proc/meminfo over time I see that the value of slab keeps growing, last week when I added the thread it was at around 500mb and now it shows as
Slab: 1056572 kB

Over the period of time (a few weeks) this increases until slab is using perhaps 6gb of memory (This machine) only has 8gb. The memory is slowly taken from the cache / buffers.

Re: Checking kernel memory usage

Vitaly Karasik_1 — Tue, 11 Mar 2008 08:58:44 GMT

it is very strange! do you use some 3rdparty kernel modules?

Re: Checking kernel memory usage

Paul Thomson_2 — Tue, 11 Mar 2008 09:14:19 GMT

Hello,
We do have some third party modules loaded into the kernel.

seos / eAC_mini perhaps
We also use VxVM on these servers too.

Here is my lsmod
Module Size Used by
iptable_nat 30437 0
ip_conntrack 53657 1 iptable_nat
iptable_mangle 4289 0
iptable_filter 4417 0
ip_tables 21569 3 iptable_nat,iptable_mangle,iptable_filter
8021q 23505 0
seos 2507744 13
eAC_mini 8636 2 seos
mptctl 34249 0
mptbase 66721 1 mptctl
sg 42617 0
cpqci 41704 4
ipmi_si 44760 2
ipmi_devintf 11024 4
ipmi_msghandler 34696 2 ipmi_si,ipmi_devintf
md5 5697 1
ipv6 282913 26
parport_pc 29185 0
lp 15089 0
parport 43981 2 parport_pc,lp
autofs4 24009 0
i2c_dev 13633 0
i2c_core 28609 1 i2c_dev
sunrpc 175545 1
ds 21449 0
yenta_socket 22977 0
pcmcia_core 69329 2 ds,yenta_socket
fdd 50192 1
vxportal 6528 0
vxfs 1727736 9 fdd,vxportal
dmpaa 6032 1
vxspec 10968 6
vxio 1768164 8
vxdmp 190816 22
dm_mirror 32465 0
joydev 11841 0
button 9057 0
battery 11209 0
ac 6729 0
ohci_hcd 24273 0
ehci_hcd 33605 0
uhci_hcd 34665 0
e1000 141456 0
bnx2 88372 0
bonding 68668 0
sata_nv 12357 0
libata 78473 1 sata_nv
ext3 137809 17
jbd 69104 1 ext3
dm_mod 68097 35 dm_mirror
qla2400 204800 0
qla2300 127360 0
qla2xxx 337440 44 qla2400,qla2300
qla2xxx_conf 303240 1
cciss 69809 5
sd_mod 19393 18
scsi_mod 141457 5 sg,libata,qla2xxx,cciss,sd_mod

Re: Checking kernel memory usage

Vitaly Karasik_1 — Tue, 11 Mar 2008 09:56:42 GMT

You can try to monitor modules memory usage with something like

lsmod |sort -rn -k2|head

But these sizes seem to relative small, in fact - up to 2.5MB for seos module...