another discussion on memory usage

[Gary Yu]

Hi all,

I remember there're some discussion on memory usage last week, I have a similar situation here which kind of confusing me. On one of my dev server(HPUX11.0), we have 2GB memory, /usr/contrib/Q4/bin/kmeminfo shows the following:
Physical memory usage summary (in pages):

Physmem = 524288 Available physical memory:
Freemem = 14933 Free physical memory
Used = 509355 Used physical memory:
System = 234215 by kernel:
Static = 20312 for text and static data
Dynamic = 135260 for dynamic data
Bufcache = 78643 for file-system buffer cache
User = 277644 by user processes
Uarea = 5848 for thread uareas
Disowned = 1024 disowned pages

it tells me that user process consumed 277k pages( more that 1.1GB) memory.

but using UNIX95= ps -e -o 'vsz ruser' | grep -v VSZ | grep -v root|awk '{ sum += $1 } END { printf "%ld\n", sum }' the result is only 683368KB.

Note that I was getting 'vsz' of the process, but still there're about 500MB user process memory difference un-account for.

what may cause this? BTW, we don't have Glance on this server.

any idea is greatly welcomed.

thanks,
Gary

[Gary Yu]

the format of the output of kmeminfo is messed up, I put it in attachment

[James Murtagh]

Hi Gary,

The command you are running is excluding all processes owned by root, hence here is probably where the shortfall is.

Remember root is just another user, albeit with special privileges, so many of the processes owned by it will contribute to user memory usage.

Regards,

James.

[Gary Yu]

Hi James,

thanks for the reply, I guess I should include root, but with root, the result from UNIX95 ps is still only 795820KB

Gary

[James Murtagh · Accepted solution]

Hi Gary,

Maybe this example will help. I've taken a process at random (well, almost at random - one with shared memory usage) and used different methods of checking its memory:

# ps -el|grep 1674
2001 S 30 1674 1662 0 154 20 42b25800 322 42ae44e8 ? 0:00 httpd

*322 is the RSS - equal to real text+data+stack in the next example.

# UNIX95= ps -e -o 'vsz ruser' -o 'pid'|grep 1674
2656 www 1674

# ./procmem 1674

Memory usage for process 1674:

Real Pages Text : 167
Real Pages Data : 147
Real Pages Stack : 8
Real Pages Shared Memory : 47
Real Pages Memory Mapped : 1041
Real Pages U-Area + Kernel Stack : 8
Real Pages I/O mapping : 0

Resident Set Size : 352

Virtual Pages Text : 497
Virtual Pages Data : 159
Virtual Pages Stack : 8
Virtual Pages Shared Memory : 303
Virtual Pages Memory Mapped : 1412
Virtual Pages U-Area + Kernel Stack : 8
Virtual Pages I/O mapping : 0

So you can see the total from the UNIX95 command is the virtual text+data+stack. ((497+159+8)*4096)/1024. It doesn't include shared memory or mapped areas.

procmem is just a program I wrote myself using the pstat interface, see "man pstat" for a full description.

Regards,

James.

[Dietmar Konermann]

Hi, Gary!

I need to take care to compare apple with apples.

Your kmeminfo output reports a physical memory statistic. So you cannot compare that with the vsz metric of ps(1), since this is a virtual memory metric.

OK, you could use ps' rz metric...

UNIX95= ps -e -o sz= | awk '{x=x+$1} END {print x*4}'

But here you have the problem that it does not consider the reference counts, causing pages of shared objects to be counted more than once.

Check if your kmeminfo version supports the -user option. This should give you a really precise stat.

Best regards...
Dietmar.

[System Dude_1]

Dear James,

Could you share the procmem program with me?