topic Re: Memory usage - confusing in Operating System - HP-UX

Memory usage - confusing

MohitAnchlia — Mon, 04 Feb 2008 20:54:40 GMT

Java app runs on HP 11 box. Now after running my code through the profiler I see that only 200 MB of heap space is being utilized but when I run top command I see 200 MB of RES space and total size is 1 GB. I tried to look at profiler but can't see where it's using extra 800 MB of memory. This application extracts blob and then encrypts them and writes to the file system. There are 4 threads that extract different kind of data. Any suggestions on where memory is being utilized

Re: Memory usage - confusing

Hein van den Heuvel — Tue, 05 Feb 2008 01:39:30 GMT

That's just the difference between 'virtual memory' or 'address space' and real, touched memory.

If a program 'mallocs' 1Gb and does nothing else, you might see SIZE=1GB and RES=1MB
And those values coudl be appearing on a 64GB physical memory system, or a 512MB system (if you can still find one that small).

Check out the Java command line options used, notably:

Xms = the initial/minimum Java heap size within the VM.
Xmx = the maximum Java heap size within the VM.

Google for: +java +memory +tuning for more info. Toss in +Xms or +Xmx as desired :-)

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

Re: Memory usage - confusing

MohitAnchlia — Wed, 06 Feb 2008 16:05:08 GMT

Is there a way to find out where it's sucking so much memory. When I did man on top the definition of SIZE is interesting:

--
Total virtual size of the process in kilobytes. This includes virtual sizes of text, data, stack, mmap regions, shared memory regions and IO mapped regions. This may also include virtual memory regions shared with other processes.
---
Interesting part is that virtual memory region could be shared with other processes.

1. So, is there a way to find out more detailed information about memory usage of a process ?
2. How to determine how much virtual memory is being shared by other processes ?

Re: Memory usage - confusing

Zeev Schultz — Wed, 06 Feb 2008 16:20:34 GMT

If you have Glance you can map memory regions of a process. Mostly,for shared regions it would be *.so (shared libraries).

Re: Memory usage - confusing

MohitAnchlia — Wed, 06 Feb 2008 17:35:22 GMT

I'll look at manual. But if somebody could also tell me what steps I need to follow to map a process to shared memory regions and what to deduce from it, and how to understand the output, would be helpful.

Re: Memory usage - confusing

Avishay_Cohen — Wed, 06 Feb 2008 21:25:39 GMT

java uses garbage collector. a facility with different policies for different occations who does all memory utilization for proper java oprations. when a need for memory arrises, the GC according to its policy frees (or accuires more) memory from the OS.
what glance sees is the OS POV. http://www.javaworld.com/javaworld/jw-08-1996/jw-08-gc.html
http://java.sun.com/docs/hotspot/gc1.4.2/

Re: Memory usage - confusing

MohitAnchlia — Wed, 06 Feb 2008 21:55:04 GMT

I am confused because when I do top I see SIZE as 1 GB and RES as 200 MB. But when I run it through the profiler total heap size allocated is just 200 MB by JVM. I see 800MB worth memory unaccounted for.

Re: Memory usage - confusing

Don Morris_1 — Thu, 07 Feb 2008 01:53:17 GMT

I don't know if Java is expressing the Heap in terms of real page usage (SZ) instead of virtual size (VSZ) or if you've just got 800Mb of shared libraries, stack, text, etc... all of which would show up in SIZE for top.

Try compiling the attached (+DD64 or +DD32 -D_PSTAT64), and my apologies in advance if it doesn't compile because I've used something too new... it _should_ be okay barring some printf() format conversion complaints.

Point it at the pid of your Java process with at least one level of verbosity ( -v -p ) and you should get a breakdown by object type for Virtual and Physical memory consumption. That should narrow things down.

Other alternatives are, as always, Glance -- any number of other programs/scripts folks have written to talk to pstat if you don't like this one, kmeminfo if you can get it from support, etc.

Re: Memory usage - confusing

Don Morris_1 — Thu, 07 Feb 2008 01:54:21 GMT

Argh! And I forget to attach the code while blathering on about how to use it. My apologies (and no points on this followup, certainly!)

Re: Memory usage - confusing

MohitAnchlia — Thu, 07 Feb 2008 16:35:36 GMT

I am unable to download the attachment.

Re: Memory usage - confusing

Hein van den Heuvel — Thu, 07 Feb 2008 17:53:17 GMT

I could download just fine.
Something wrong on your side / site.
need to click that 'go ahead it is safe' in the 'yellow bar' in a browser window? It might have appeared in a different browser window if you have multiple. Try again?

Just in case the extention ".C" caused a problem I'll re-attach Don's source as ".txt"

Also... google for pstat_getprocvm will get several alternatives.

fwiw,
Hein.

Re: Memory usage - confusing

MohitAnchlia — Thu, 07 Feb 2008 18:12:22 GMT

I compiled and ran, but I just get this:

$ aCC -o meminfo 298591.c
$ meminfo -v -p 11765
VIRT/PHYS/LOCKED/SWAP summaries in pages.
System page size is 4096 or 0x1000 bytes.

Re: Memory usage - confusing

Don Morris_1 — Thu, 07 Feb 2008 20:02:06 GMT

Just to eliminate the obvious, compile with +DD64 or +DD32 -D_PSTAT64 as I requested. Calling pstat on a 64-bit kernel without handling 64-bit values is a good way to get garbage results [such as perhaps not finding the pid you though you should].

Re: Memory usage - confusing

MohitAnchlia — Thu, 07 Feb 2008 21:06:10 GMT

How do I interpret these results ? Are these in bytes ?

PID 11765:
UNUSED TYPE consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
UAREA consumes 192 VIRT, 192 PHYS, 0 LOCKED and 192 SWAP.
TEXT consumes 16 VIRT, 15 PHYS, 0 LOCKED and 0 SWAP.
DATA/HEAP consumes 10223 VIRT, 10095 PHYS, 0 LOCKED and 10223 SWAP.
MAIN STACK consumes 576 VIRT, 96 PHYS, 0 LOCKED and 576 SWAP.
SYSV SHMEM consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
NULL DEREF consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
MEM MAPPED I/O consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
MMAP consumes 308674 VIRT, 54399 PHYS, 0 LOCKED and 45134 SWAP.
GRAPHICS SPECIFIC consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
GRAPHICS SPECIFIC consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
RSE STACK consumes 0 VIRT, 0 PHYS, 0 LOCKED and 0 SWAP.
PID 11765 Mb in Use:
Virt: 1248 Phys: 253 Swap: 219

Re: Memory usage - confusing

Don Morris_1 — Thu, 07 Feb 2008 21:59:35 GMT

No, those are all in pages -- hence the line about:
"VIRT/PHYS/LOCKED/SWAP summaries in pages"
followed by telling you how many bytes are in a page for your system [yes, it isn't always 4096 ;) ]: "System page size is 4096 or 0x1000 bytes".

So for your Data/Heap -- that process is using 10095*4096 = 41349120 bytes (40380kb... 39.43359375Mb) of Physical Memory [RAM].

MMAPs are the biggest Virtually at 1.17749Gb -- if this is the process you started the thread about, there's your answer. If you want to know the sizes of each object, add another "-v" option [verbose level greater than 1 spits out the virtual range in question for each virtual object in the process]. That may not have much meaning for you -- though in this case it would let you know if that's one big mmap or just a whole pile of little ones.

I'd note though that there's only around 212Mb in use (and if these are MMAPs of shared libraries, some of that cost is going to be independent of the process anyway... any process linked to the library will show at least some consumption for it).... so I really wouldn't think any of this is worth worrying about unless you're actually running out of swap or something.

Re: Memory usage - confusing

MohitAnchlia — Thu, 07 Feb 2008 23:31:43 GMT

Thanks. So if 212MB is in use then why does it report the total size of 1.7 GB ? I know for sure it doesn't need more than 250 MB because I've run it in profiler

Re: Memory usage - confusing

Don Morris_1 — Fri, 08 Feb 2008 01:30:04 GMT

Two possibilities leap to mind:

1) The profiler reports your expected/worst case physical memory consumption for your direct allocations (RES, not SIZE).

2) Your profiler may report your expected virtual memory needs... but doesn't report every single virtual object which will be affiliated with your process when it runs (like a file being mmap()'d, the total virtual size of every shared library attached, etc.). This is my suspicion since you specifically mention you're profiling your heap (which usually means that somewhere there's a malloc() or brk() call... I don't think Java's odd enough to do this stuff on the Stack, though it might use private MAP_ANONYMOUS mmap() calls. Your DATA/Heap virtual usage is well within the profiling boundary -- it is the other mmap() objects in the process that are using the virtual address space.

Re: Memory usage - confusing

MohitAnchlia — Fri, 08 Feb 2008 16:00:45 GMT

What really is mmap ? Is it something that could be shared between other processes too ? Does it mean that some space is reserved but could be used by other processes, if required ?

Re: Memory usage - confusing

Don Morris_1 — Fri, 08 Feb 2008 16:41:30 GMT

http://docs.hp.com/en/B2355-60130/mmap.2.html

Yes, mmap() created objects [files or Anonymous memory ranges] can be either shared or private. Yes, shared objects affect the virtual size of all processes attached to them.

At this point, perhaps you should look over:
http://docs.hp.com/en/1218/mem_mgt.html

Re: Memory usage - confusing

MohitAnchlia — Fri, 08 Feb 2008 17:01:00 GMT

thanks I'll try to read the manual. When I look at glance I see FIle Name as

What does it mean and how can I trace that inode number to actual file ? One of these files is taking 90MB space.