NUMA policy and how to use it...

[Chris Fadrowski]

I have a rx7640 system running Windows 2003. I am being asked to reconfig NUMA so that we use both cells on this box. I have never heard of this. I tried searching some docs but couldn't find much. Can someone help or point me in the right direction?

[shyamalendu sarkar]

Hi,
I am aware of NUMA policies in HP OVMS but not windows2003. This configuration is hardware dependent, so eventually it will be same and thought to share it.
The rx7640 will be having 2 cells max.
First of all let me explain about possible NUMA configuration on this.

Through partition manager chose the entire partition with 2 cells and select the memory configuration option.
Then,
Out of total system memory some percentage of memory will be allocated to each cell.
Memory to each cell can be allocated by mentioning % of total memory or mention actual value (for example x gig).


For example:
System is having 64gb memory.
Now allocate 16 gb memory for each cell.
This 16 gb memory will be local to each cell. So the memory left is 64-16*2=32gb.
This memory will be treated as common memory for both the cells.

So after this configuration the system will look like:

Cell 1: local memory 16 gb (segment 1)
Cell 2: local memory 16 gb (segment 2)

Rest 32 gb (segment 3) memory is common to cell1 and cell2.

Please note: it does not require any physical change in memory module.

There is no rule about the amount of memory to be allocated to different cells.

After the configuration, it is the Operating System which deals differently with these segregated memory segments (resource locality domains).

Typically it should enhance the overall performance as memory will be accessed in non uniform fashion.
Please let me know if you require more insight on NUMA and further specific clarification.

Regards,
Shyamalendu Sarkar

[Prokopets]

Well... I'm not sure, but if i understood you correctly, you just have to put both cells in one nPar (if they are not already configured like this). Can you clarify you server configuration, i.e.: partitions, cells, CPUs.

[shyamalendu sarkar]

Yes that is what I meant.
My 7640 server configuration is as below:

2 cells in one nPar.

The system is having total 16 gb memory.
I have allocated:
6 gb for cell1 local memory.
6 gb for cell 2 local memory.
4 gb for common memory.

This memory segregation is completely dependent upon the type of application and their memory usage pattern (local memory vs remote memory).
Typically having good amount of local memory will help applications to perform better.

With hyper-thread disabled,
8 cpus (of cell 1) are in first resource locality domain.
8 cpus (of cell 2) are in second resource locality domain.

In ovms, once you segregate the memory like that, cpus will be automatically segregated uniformly and bound to each resource locality domain.

Hope this helps.