What is M_TEMP and why it consumes so much memory?

[wm_shi]

Hi Guys,

I ran kmeminfo and got following result:
----------------------------------------------------------------------
Physical memory usage summary (in page/byte/percent):

Physmem = 13983744 53.3g 100% Physical memory
Freemem = 802148 3.1g 6% Free physical memory
Used = 13181596 50.3g 94% Used physical memory
System = 5448501 20.8g 39% By kernel:
Static = 633393 2.4g 5% for text/static data
Dynamic = 1989745 7.6g 14% for dynamic data
Bufcache = 2796748 10.7g 20% for buffer cache
Eqmem = 1237 4.8m 0% for equiv. mapped memory
SCmem = 27378 106.9m 0% for critical memory
User = 7733628 29.5g 55% By user processes:
Uarea = 25260 98.7m 0% for thread uareas
Disowned = 8 32.0k 0% Disowned pages
Bad = 1 4.0k 0% Bad

----------------------------------------------------------------------
Kernel dynamic memory usage (in page/byte/percent):

Dynamic = 1989745 7.6g 14% Kernel dynamic memory
Arenas = 1534703 5.9g 11% Kernel arenas
M_TEMP = 788913 3.0g 6%
M_IOSYS = 262205 1.0g 2%
M_SPINLOCK = 177811 694.6m 1%
M_VXVM = 90066 351.8m 1%
M_DYNAMIC = 69504 271.5m 0%
VFD_BT_NODE = 33809 132.1m 0%
KMEM_ALLOC = 20388 79.6m 0%
ALLOCB_MBLK_LM = 11626 45.4m 0%
M_SWAP = 10952 42.8m 0%
ALLOCB_MBLK_SM = 10138 39.6m 0%
LVM_PBUF = 8540 33.4m 0%
M_PREG = 7678 30.0m 0%
M_REG = 7377 28.8m 0%
VM MISC ARENA = 4129 16.1m 0%
M_KTHREAD = 3653 14.3m 0%
Other = 27914 109.0m 0% Other arenas...
Kalloc = 454711 1.7g 3% kalloc()
SuperPagePool = 134545 525.6m 1% Kernel superpage cache
BufcacheBufs = 257664 1006.5m 2% Buffer cache bufs
BufcacheHash = 40960 160.0m 0% Buffer cache hash heads
Other = 21542 84.1m 0% Other...
Eqalloc = 331 1.3m 0% eqalloc()

It shows that M_TEMP used 3g of memory.

After I ran kmeminfo -a to see more detail on arena, the M_TEMP part of details is as follows:
Variable kmem_arena_t 0xd8005780 "M_TEMP" owns 788980 pages.
Attributes: KMT_DEFAULT KT_DEFAULT KAS_ALIVE
index 0 (size 24) = 8302 pages
index 1 (size 56) = 33376 pages
index 2 (size 120) = 5834 pages
index 3 (size 184) = 50329 pages
index 4 (size 248) = 344364 pages
index 5 (size 312) = 62 pages
index 6 (size 376) = 41 pages
index 7 (size 440) = 32 pages
index 8 (size 568) = 8828 pages
index 9 (size 632) = 46 pages
index 10 (size 760) = 4 pages
index 11 (size 952) = 245 pages
index 12 (size 1336) = 325633 pages
index 13 (size 1976) = 18 pages
index 14 (size 4024) = 40 pages
index 15 (size 4096) = 179 pages
index 16 (size 8192) = 320 pages
index 17 (size 12288) = 21 pages
index 18 (size 16384) = 308 pages
index 19 (size 20480) = 160 pages
index 20 (size 24576) = 96 pages
index 22 (size 32768) = 128 pages
index 23 (size 36864) = 10614 pages

Who can tell me what the M_TEMP is and why it consumes so much memory?

Thanks,
William

[Senthil Kumar .A_1 · Accepted solution]

Hi,

Check this link for more details...

ftp://eh:spear9@hprc.external.hp.com/memory.htm#jfs_inode_cache

Regards,
Senthil Kumar .A

[Arunvijai_4]

Hi William,

Take a look at this thread,

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=949773
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=954245

-Arun

[wm_shi]

Hi, Senthil,

Thank you very much for your information!

I have idea on that now.

But another question is:
What is M_IOSYS? It used 1G of my memory while several M of memory in other system.

Thanks,
William

[Arunvijai_4]

Hi William,

What is M_IOSYS? It used 1G of my memory while several M of memory in other system.

M_IOSYS : Memory used for IO.

Do you feel that kernel is using most memory ? Probably, you need to double check your tunable parameters.

-Arun

[Senthil Kumar .A_1]

Hi,

Try this doc,...

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000079999303

Regards,
Senthil Kumar .A

[wm_shi]

Hi,

I also noticed that static data used 2.4G memory and used kmeminfo -static and got following:

Kernel TEXT pages not requested in crashconf
Will use an artificial mapping from a.out TEXT pages
kmeminfo (3.71)
unix: /stand/vmunix 11.11 64bit PA2.0
core: /dev/kmem live
link: Tue Feb 14 21:57:25 EAT 2006
boot: Tue Feb 14 22:09:57 2006
dump: Mon Mar 13 17:45:59 2006

----------------------------------------------------------------------
Pfdat processing:

Scanning 13671023 pfdat entries (be patient) ...

Warning: Found a P_BAD page (pfn 0xc92489, pfd_t 0xd1f20aa0)
Warning: P_BAD is set by the memory diagnostics (dmem scrubbing).

----------------------------------------------------------------------
Static kernel memory usage (in page/byte/percent):

Static = 633393 2.4g 5% Static memory
Text = 2574 10.1m 0% Text
Data = 444 1.7m 0% Data
Bss = 4238 16.6m 0% Bss
Tables = 626137 2.4g 4% System tables
pfdat = 320414 1.2g 2% pfdat

----------------------------------------------------------------------
Static system memory (size in bytes and pages):

Name Start- End Nent Size
text 0x00020000-0x00a2e000 1 10543104
data 0x00a2e000-0x00bea000 1 1818624
bss 0x00bea000-0x01c783a8 1 17359784
phys_mem_tbl 0x01c90710-0x01c9074c 5 60
sysmap_32bit 0x01d9fd80-0x01e15080 30000 480000
sysmap_64bit 0x01e15080-0x01e8a380 30000 480000
pgclasstab 0x023cb000-0x03121000 13983744 13983744
mpproc_info 0x0312f000-0x03196900 32 424192
pfn_to_virt_ptr 0x5aadb000-0x5aae87e0 3454 55264
pfn_to_virt 0x5aae9000-0x5fff9000 5574656 89194496
callout_info_array 0x5fff9000-0x5fffb200 32 8704
quad1map_64bit 0x5fffc000-0x5fffd910 401 6416
quad4map_64bit 0x5fffd940-0x5ffff250 401 6416
htbl2_0 0x60000000-0x80000000 16777216 536870912
pfn_to_virt 0x80000000-0x88050000 8409088 134545408
inode 0x88059000-0x882fb400 5568 2761728
file 0x882fb400-0x88f921f0 150010 13200880
ncache 0x88f92200-0x890d2c00 9120 1313280
nc_hash 0x890d2c00-0x89112c00 16384 262144
nc_lru 0x89112c00-0x8911cc00 512 40960
cfree 0x89315800-0x89354fa0 6500 260000
pfdat 0x8932d2c0-0x8938d2c0 4096 393216
tmp_save_states 0x89354fc0-0x8935dfc0 32 36864
memWindows 0x8935dfc0-0x8935e020 2 96
quad4map_32bit 0x89361300-0x89362c10 401 6416
pfdat_ptr 0x89364000-0x893717e0 3454 55264
pfdat 0x8937a900-0x8aaba900 253952 24379392
page_groups 0x89384440-0x89384680 18 576
pfdat 0x8aaa5720-0x8ab05720 4096 393216
pfdat 0x8ab05740-0xd7785740 13418496 1288175616
space_map 0xd7787000-0xd778f000 262144 32768
kmem_lobj_hdr_tbl 0xd8400000-0xd8684210 164897 2638352

Total accounted static memory = 522395 pages

So, what are pfdat, htbl2_0 and pfn_to_virt which consume the most of the static data memory?

Thanks,
William

[Senthil Kumar .A_1]

Hi William,


If you are really concerned about the fact that you memory is being hogged up unecessarily or there is a memory leak, Please analyze your system as per the data found in the following link,...

ftp://eh:spear9@hprc.external.hp.com/memory.htm

There might be many info with which we need not concern much as it is used internally by the kernel, whose data manipulation and intervention might be specific to the WTEC engineers at HP.

As far as a System admins are concerned the above link provides enough data to keep our memory utilization checked.

Regards,
Senthil Kumar .A
(P.S: If you need to know intricasies anyway, wait until ,SEP ,Clay , James and other experienced admins take a peek into this thread.)

[wm_shi]

Thank you Senthil very much!

I have learned a lot from memory.htm.
But there aren't anything about the system static data such as pfdat, htbl2_0 and pfn_to_virt which looked like something was mis-configured.
I want to get some links to documents describing these internal structures, what are these, what tunables impacting these and how to change something to save memory from kernel to my applications.

Thanks,
William

[Kent Ostby]

Willliam -- There were some patches for 11.11 memory leak in M_IOSYS.

For M_IOSYS, there are three patches or their predecessors that might assist you:

PHKL_29039, PHKL_30044, PHKL_29468 all introduced fixes for issues in this area.

Also, if you use CA Unicenter, you may want to talk to them to see if TF93187 would apply to your system since it addressed some extra memory usage by CA Unicenter in the M_TEMP area of index 12.

Best regards,

Kent M. Ostby

[Don Morris_1]

pfdat - physical frame descriptors. This is the main metadata (per physical page) for physical memory.

htbl2_0 - That's the page table (virtual to physical translations... i.e. where you go to figure out what to load when you take a TLB miss fault)

pfn_to_virt - Physical to virtual reverse lookup table. (The anti-htbl2_0).

These are core VM metadata tables. You can't shrink them -- and you certainly need them to be there. These are *always* going to take a certain percentage (exact percentage can vary by release and patch level) of memory because they scale directly with the number of physical pages.

Don

[wm_shi]

Thank you Senthil and Kent,

I have studied your suggestion About why M_IOSYS consumes so much memory.
And we found the most recent patch is:
PHKL_29039 --> PHKL_32090
PHKL_30044
PHKL_29468 --> PHKL_31227
And in our system:
$/usr/sbin/swlist -l product | grep -E 'PHKL_29039|PHKL_32090|PHKL_30044|PHKL_29468 |PHKL_31227'
PHKL_30044 1.0 Memory leak fix for Core Bus ioscan
PHKL_31227 1.0 PCI patch,boot-time HPMC,PCI-X,OLA/R,AGP
PHKL_32090 1.0 SCSI IO Cumulative Patch

The above shows that we have no memory leak problem in our system.

So is there any further reason for that 1G memory?

Thanks,
William