리눅스에서 메모리 사용률이 안떨어지는 현상

[박한진]

OS : RHEL AS4 설치하였습니다.



성능 테스트 중인데요



문제가 하나 있습니다.





oracle 설치해서 DB insert 하고 이런저런 테스트 해봤는데요





메모리 사용률이 안떨어집니다.



procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa

0 3 0 861216 21840 1077960 0 0 0 592 1640 88 2 0 48 50

2 0 0 755616 22108 1185072 0 0 4 12732 1835 2262 25 12 49 15

0 2 0 699168 22160 1237020 0 0 4 106956 1666 310 13 13 56 17

1 2 0 699744 22176 1237264 0 0 0 3340 1387 211 10 1 62 28

0 4 0 700512 22176 1237264 0 0 0 860 1355 72 2 0 63 35

1 1 0 700432 22196 1238544 0 0 4 6940 1816 1857 18 8 42 32

1 1 0 689680 22236 1248904 0 0 8 19228 2636 3307 22 4 49 24

1 0 0 684624 22248 1253832 0 0 12 12596 1920 1863 20 3 64 14

0 1 0 684112 22256 1254344 0 0 4 8832 1266 600 23 2 71 5

2 0 0 673880 22264 1264476 0 0 8 11932 2448 2892 3 2 74 21

1 1 0 673880 22280 1264460 0 0 0 8944 1221 516 24 2 70 3

1 1 0 673368 22316 1264424 0 0 4 8980 1252 764 24 3 69 4

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa

2 1 0 668376 22320 1269880 0 0 4 15244 2066 2195 24 4 49 23

1 0 0 667760 22328 1270392 0 0 0 9392 1319 669 24 2 67 6

1 0 0 661424 22336 1276884 0 0 8 10212 2009 2031 8 2 73 16

0 1 0 656176 22336 1281824 0 0 0 5836 1717 1420 1 1 75 23

1 0 0 651824 22348 1286232 0 0 4 9896 1715 1457 15 2 67 16

1 0 0 651872 22372 1286208 0 0 0 8660 1246 527 24 2 70 3

2 0 0 645984 22384 1292176 0 0 8 15544 2076 2200 24 4 54 18

1 1 0 508256 22648 1427372 0 0 8 11196 1631 1336 22 3 62 13

1 0 0 501216 22660 1434120 0 0 24 14796 2142 2315 21 3 55 21

1 0 0 501152 22676 1434104 0 0 8 9236 1234 539 24 3 70 4

1 1 0 495200 22692 1440328 0 0 4 11212 2077 2174 9 3 72 16

1 0 0 495200 22692 1440328 0 0 4 8840 1220 508 24 2 70 4

1 1 0 488864 22708 1446812 0 0 8 10836 1996 2018 11 3 65 22

1 1 0 317344 23076 1615444 0 0 0 11936 1674 1396 24 4 51 20

2 0 0 261600 23144 1670236 0 0 8 15500 2205 2528 23 12 44 21

0 4 0 202400 23204 1724776 0 0 0 106884 1595 289 13 13 56 17

1 3 0 203168 23212 1724768 0 0 0 1948 1491 146 6 1 65 28

0 4 0 203616 23220 1724760 0 0 0 640 1299 99 1 0 26 73

0 5 0 205856 23244 1724736 0 0 0 1172 1538 155 4 1 45 51

1 0 0 204192 23260 1726540 0 0 0 9712 1506 1056 19 2 65 13

0 2 0 201184 23268 1729392 0 0 8 7908 1573 1025 13 2 71 14

1 0 0 141032 23472 1783008 0 0 0 9108 1301 657 24 3 69 5

1 0 0 135056 23536 1789184 0 0 4 13768 1706 1294 16 4 66 14

0 1 0 132304 23552 1791768 0 0 24 11596 2145 2323 9 2 73 16

0 2 0 23632 23668 1894352 0 0 4 108624 1574 691 16 21 51 13

0 2 0 24080 23676 1894604 0 0 0 124 1359 58 0 0 69 31

0 2 0 25056 23676 1894604 0 0 0 208 1594 102 0 1 50 49

0 1 0 26976 23700 1895100 0 0 0 3784 1976 960 0 2 56 42

1 0 0 21024 23740 1901300 0 0 24 12288 1661 1184 11 3 71 15



이랫던 놈이.무거운 job을 돌리고 나니 위와같이 계속 free 메모리가 떨어지더군요



그리고 마지막에는





procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa

0 3 0 24352 18520 1897420 0 0 29 456 282 72 3 1 93 3



procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa

0 0 0 23840 19180 1897280 0 0 28 435 281 70 3 1 94 3



procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r b swpd free buff cache si so bi bo in cs us sy id wa

0 0 0 23584 20840 1895880 0 0 25 392 278 65 3 1 94 2





위와같이 freememory가 떨어졌습니다.





문제는 job 이 끝난다음에도 메모리가 반환이 안된다는 것입니다.



계속 위와같은 상태로 있는거죠..



메모리 사용률을 확인할때 freememory 만 가지고 평가한다는게



무리이긴 하지만. 그래도 값의 반환이 너무 안되는거 같아서요



시스템의 문제인지 아니면 원래 그런건지...

[신동인]

리눅스의 특성을 알고계시겠지만 Free메모리가 작다고 문제가 되는 것은 아닙니다. 리눅스의 특성상 메모리의 제어를 커널이 조정하기 때문에 cash영역에 할당되어 있는 메모리들이 실제 Free영역에 포함시켜 환상하는 것이 Unix와 다른 리눅스만의 특성입니다. 즉 시스템의 성능을 향상시키기 위해서 사용하는 리눅스만의 고유적인 메모리의 사용방법인 것입니다. 아래의 자료는 Linux의 메모리관련 자료입니다. 참조하시길 바랍니다.



※ Linux Memory 관련한 자료



ActiveAnon:

Active memory which is mapped into user processes, plus swap cache.

=> nr_active_anon_pages in swap





ActiveCache:

Active memory which is caching files and not mapped into user memory.



Inact_laundry:

Memory which has been tagged to be reclaimed, but whose contents have not yet

been written out/synced to disk.



SwapCached:

Memory that once was swapped out, is swapped back in but still also is in the

swapfile (if memory is needed it doesn't need to be swapped out AGAIN because

it is already in the swapfile. This saves I/O).

=> swapper_space.nrpages, i.e. # of total swap pages



...and soo, "ActiveAnon" is not the same as "SwapCached".





Buffers:

--------

Relatively temporary storage for raw disk blocks shouldn't get tremendously

large (20MB or so)



=> Memory used by buffer.





Cached:

-------

in-memory cache for files read from the disk (the pagecache). Doesn't include SwapCached.



Cached = (# of pages currently in the hash table) - (Memory used by buffers) - (# of total swapped pages)









Eighth line of /proc/meminfo shows:



Cached: 103280 kB = 103280 x 1024 = 105758720





# cat /proc/meminfo

total: used: free: shared: buffers: cached:

Mem: 3133325312 290783232 2842542080 0 32587776 105758720

Swap: 3144613888 0 3144613888 ^

MemTotal: 3059888 kB |

MemFree: 2775920 kB |

MemShared: 0 kB |

Buffers: 31824 kB |

Cached: 103280 kB <--------------------------------+

SwapCached: 0 kB

Active: 131264 kB

ActiveAnon: 42320 kB

ActiveCache: 88944 kB

Inact_dirty: 34560 kB

Inact_laundry: 11328 kB

Inact_clean: 0 kB

Inact_target: 35424 kB

HighTotal: 0 kB

HighFree: 0 kB

LowTotal: 3059888 kB

LowFree: 2775920 kB

SwapTotal: 3070912 kB

SwapFree: 3070912 kB

Committed_AS: 106768 kB

HugePages_Total: 0

HugePages_Free: 0

Hugepagesize: 262144 kB







vm.hugetlb_pool:



The hugetlb_pool file is responsible for recording the number of megabytes used

for huge pages. Huge pages are just like regular pages in the VM, only they are

an order of magnitude larger. Note also that huge pages are not swappable. Huge

pages are both beneficial and detrimental to a system. They are helpful in that

each huge page takes only one set of entries in the VM page tables, which allows

for a higher degree of virtual address caching in the TLB (Translation

Look-aside Buffer: A device which caches virtual address translations for faster

lookups) and a requisite performance improvement. On the downside, they are very

large and can be wasteful of memory resources for those applications which do

not need large amounts of memory. Some applications, however, do require large

amounts of memory and can make good use of huge pages if they are written to be

aware of them. If a system is running applications which require large amounts

of memory and is aware of this feature, then it is advantageous to increase this

value to an amount satisfactory to that application or set of applications.







# cat /proc/meminfo | grep -i huge

HugePages_Total: 0

HugePages_Free: 0

Hugepagesize: 262144 kB



# grep HUGETLB /boot/config-2.4.21-40.EL

CONFIG_HUGETLBFS=y

CONFIG_HUGETLB_PAGE_SIZE_256MB=y



=> Hugepagesize = 262144 kb / 1024 = 256 MB.

If you require 9000 MB, then I'd do:

9000 / 256 ~= 35 pages of size 256 MB.



You then use:

# echo 35 > /proc/sys/vm/hugetlb_pool



....this does not seem to work for me...I will be consulting with Red Hat.

PLEASE provide me a sysreport out form the customer system.





This information was provide to you earlier:

- /usr/src/linux-2.4/Documentation/vm/



The hugetlb_pool file is responsible for recording the number of megabytes used

for huge pages. Huge pages are just like regular pages in the VM, only they are

an order of magnitude larger. Note also that huge pages are not swappable. Huge

pages are both beneficial and detrimental to a system. They are helpful in that

each huge page takes only one set of entries in the VM page tables, which allows

for a higher degree of virtual address caching in the TLB (Translation

Look-aside Buffer: A device which caches virtual address translations for faster

lookups) and a requisite performance improvement. On the downside, they are very

large and can be wasteful of memory resources for those applications which do

not need large amounts of memory. Some applications, however, do require large

amounts of memory and can make good use of huge pages if they are written to be

aware of them. If a system is running applications which require large amounts

of memory and is aware of this feature, then it is advantageous to increase this

value to an amount satisfactory to that application or set of applications.