Glance vs. ps -e shows big difference in memory consumption

[Mark Frederickson]

As discussed a couple of days ago, Glance was reporting about 100% memory utilization. This was great to know, but we were in need of knowing which processes were consuming this memory. The following was suggested:

UNIX95= ps -e -o vsz -o pid -o args |sort -k1nr|more

Today, Glace is reporting 100% utilization. However, using the 'ps' command above, I get something like 4.2 GB consumed (out of 8GB total). So, I must ask, why this big discrepancy?

Thanks for any input on this!

[Pete Randall]

Just theorizing here, Mark, but it seems to me that ps would not account for the kernel consumed memory or buffer cache, just the processes' usage.


Pete

[Mark Frederickson]

Thanks for the insights. Can you estimate the memory consumption of these other components?

[Bill Hassell]

Memory consumption is a very complex metric in Unix since there are so many different ways RAM is used. First there is the kernel itself which is relatively static but may change somewhat in newer releases. Then there is the dynamic buffer cache (no command to measure this except Glance and Measureware). Then there are shared libraries that many, many programs use. So far, all the RAM used by these items could be considered to be kernel RAM but shared by virtually all processes.

Now for the buffer cache, I (and other sysadmins) recommend NOT to use the dbc (dynamic buffer cache) percentages because the min and max values will automatically change when RAM changes, and the actual cache size will vary depending on RAM usage by processes. Instead, set bufpages to the number of 4k pages needed, something between 500 and 1000 megs. Then dbc_min_pct and dbc_max_pct kernel params are ignored.

Now there are program-specific areas of RAM that are not accounted in ps because they are shared: shared memory segments and memory mapped files. It's quite difficult to assign these shared components to specific programs because they are shared. ipcs -bmop will show you shared memory usage but can't show all the processes that might be using that memory. Another shared component are the shared text portions of identical programs (ie, each login shell or each httpd process for a web service). To minimize RAM usage, each program is assigned a space for the unchanging instructions (not data) and if another copy of the program is started, this area is shared because the instructions are read-only.

And finally, it's quite difficult to determine what components (programs, shared memory, etc) are actually in RAM or have some portion rolled out to swap. swapinfo -tam tells you about actual usage but not what components are there.

Glance is by far the most accurate in showing memory usage. ps is also quite accurate for what it shows (the unique data areas for each process) but by definition, it won't show usage that isn't associated with a specific program.