topic Re: Hyperthreading and memory allocation in Operating System - Linux

Hyperthreading and memory allocation

Paul Ambro — Thu, 06 Jan 2005 11:16:43 GMT

Below is a printout of "top" which shows three instances (PID) of a process called src. On "top", each instance seems to be consuming 2.4G of memory however, there is actually only one "src" running. 3 PIDs are the result of the hyperthreading. Is it safe to assume that this process (src) is consuming 7.2G (2.4G x 3)or this is just how "top" displays the process, the total is in reality 2.4G?

PID USER RSS SHARE %CPU %MEM TIME COMMAND
14665 root 2.4G 2422M 21.1 65.2 1229m src
14670 root 2.4G 2422M 0.0 65.2 0:00 src
14671 root 2.4G 2422M 0.1 65.2 14:46 src

tks,
ppaa

Re: Hyperthreading and memory allocation

xyko_1 — Thu, 06 Jan 2005 14:16:57 GMT

Hi Paul,

Are you sure that 3 PIDs are because of HT ?
I don't think so. In my opinion you have 3 instances of "src" running. We can define it better if you show us the ppid for those pids.

regards,
xyko

Re: Hyperthreading and memory allocation

Paul Ambro — Thu, 06 Jan 2005 16:35:09 GMT

Let me clarify. I am starting a single application and it fires up three processes so I agree with you that it may not be due to hypertreading, though is Solaris it comes only as one PID. But the question remains. Does each PID allocate a separate memory space? Most people I talked to say no, but looking at my memory consumption, it is over 7G (memory+swap) which is consistent with (3x2.4G). This is the only big process running on the machine and adding all other processes doesn't come anything near this value.

root 14665 1 5 2004 ? 21:37:02 src
root 14670 14665 0 2004 ? 00:00:00 src
root 14671 14670 0 2004 ? 00:15:25 src

Re: Hyperthreading and memory allocation

HGN — Thu, 06 Jan 2005 16:48:22 GMT

Hi

As xyko mentioned I don't think it is because of Hyper threading.

Rgds

HGN

Re: Hyperthreading and memory allocation

Stuart Browne — Thu, 06 Jan 2005 17:35:06 GMT

The process spawns two children, standard sort of 'threading'.

If you use 'ps -elf | grep src$', you'll see the relations, both with the PPID, PID, as well as the thread type.

Now, as it's a parent/child situation, they can share memory space, so I'd assume most of the 2.4GB allocated memory is shared.

It's the one application/process, just has children spawned off it to control separate functions.

On solaris, it showing only one process is just the way it handles threading.

Re: Hyperthreading and memory allocation

Paul Ambro — Fri, 07 Jan 2005 09:40:34 GMT

Stuart,

It is a standard threading (parent/child situation). Indeed, whether it is hyper or not doesn't really matter.

What I am really curious and would like your thoughts is whether te 2.4G is shared or not, and how can I verify this.

As I mentioned, while the common sense it that it is shared, my machine is consuming 7.2+G of memory - and this is the only major process running.

I want to find out if; this is how it should be or, it is a memory leak. Is there anyway I can verify the pointers for each separate PID?

FYI this is ASE 2.1, on Solaris it doesn't happen, (it is 2.4 max mem).

pa

Re: Hyperthreading and memory allocation

xyko_1 — Fri, 07 Jan 2005 10:56:25 GMT

Paul,

in my opinion each src process is consuming 2.4 Gb of phisical memory. It's what RSS parameter of top is showing to us.

Call TOP and isssue M subcommand to sort the output by memory consumption and see what it shows to you.

Search RSS on TOP man page to understand my point.

Regards,
Xyko

Re: Hyperthreading and memory allocation

Stuart Browne — Sun, 09 Jan 2005 17:14:37 GMT

I disagree.

Yes, 'rss' shows memory used by a process, but 'share' shows how much of it is SHARED, i.e. 2422M of it (all of it).

As for what's usnig the memory, start by looking at the 'cache' and the 'buffers' values in top (or in /proc/meminfo).

Otherwise, 'ps -elf' is your friend.

Re: Hyperthreading and memory allocation

xyko_1 — Mon, 10 Jan 2005 08:42:27 GMT

Hi Paul and Stuart,

I still agree that rss is the amount of real memory allocated to the pid. I may be wrong but ....

I found some information regarding that stuff that may help us.

http://mirror.hamakor.org.il/archives/linux-il/12-2003/7374.html

regards all,
xyko

Re: Hyperthreading and memory allocation

Paul Ambro — Thu, 13 Jan 2005 14:03:20 GMT

Information was posted on the link above.