Integrity Servers
1748134 Members
3515 Online
108758 Solutions
New Discussion юеВ

Re: rx7620 memory allocation

 
Paul Brain
New Member

rx7620 memory allocation

System : 4x 1.5GHz 6MB Itanium : 24GB RAM : Win2003(64) SQL2000(64) : EVA 3000 SAN etc

I'm shortly going to upgrade both my nodes of rx7620 to 6x CPU from 4 and am wondering about the memory allocation on a per cpu basis.

Currently I have multiple databases running in size of 100GB each and some v.large tables ( in the GB range ). This is an online transaction system where perofrmance is paramount.

When I upgrade to 6cpu, I'll have to re-distribute the memory accross the CPU's ??
so that each CPU will be able to access 4GB. Whereas currently each cpu ( under a 4x system ) can access 6GB RAM.

By the virtue of increasing the number of CPU's in the system and as a side effect reducing the amount of RAM available to each CPU from 6-4GB , combined with the knowleged that I have multiple v.large databases, will the performance of the system be affected in a negative way ..... or is the memory available to all cpu and physical location unimportant ?

If it is cpu specific memory and there is a lot of page swapping going on when using large tables , is the lack of ram per cpu a potential bottleneck and would it be more benefit to install more memory than CPU ?

Any insight into this subject would be appreciated.

Paul.

6 REPLIES 6
kris rombauts
Honored Contributor

Re: rx7620 memory allocation

Hi Paul,

something unclear to me here, why are you redistributing the memory when you move from a 4 CPU to a 6 CPU RX7620 ?
When doing so, it means you are probably adding a extra cell board which will have 2 CPU's right ?

Each cell board needs to have memory on it, so are you planning on removing memory DIMMs from the first cell board to install on the second board then during the upgrade?

If the above upgrade scenario is what is taking place at your site, then why not buy the new cell with some memory on it so that you can keep working with 100% CLM (local memory to the CPU's on their own cell board).



HTH

Kris
Stefan Stechemesser
Honored Contributor

Re: rx7620 memory allocation

Hi Paul,

I'm not an expert in Windows memory management on multiprocessor machines, but basically, the memory is shared by all active CPUs in a hardpartition.

As Kris already outlined, it is possible to tune the ammount of (cell-)interleaved memory and cell local (non interleaved) memory to avoid some overhead of cell-cell transfers if independent processes are running on a multicell system which access only small memory ranges. But even if you only use cell local memory, still the complete memory is accessible by each CPU.
But, if you add two CPUs, then I could imagine that you will also increase the number of database licences and processes and if the workload increases and the memory does not increase, then indeed each process would get less of the physical memory than before. But this would also be true if you simply start more processes on the same amount of CPUs.

=> increasing the number of CPUs does not impact the memory performance unless the number of memory using processes increase

best regards

Stefan
rick jones
Honored Contributor

Re: rx7620 memory allocation

Paul - unlike many systems out there (eg Opteron, US III and others) memory in an HP Integrity server is _not_ per-processor. In the case of the rx7620, memory is per-cell, and there can be anywhere between 1 and 4 CPUs (2 and 8 if you count dual-cores) in a cell.

Any CPU in the rx7620 can access any memory in the rx7620. As such, you do not _need_ to add RAM when you add CPUs.

It is possible you may _want_ to add RAM as you add CPUs. Depends on how much free RAM you had before and whether increasing the throughput of your system necesitates a larger working set.

You may want to look into use and support for Cell Local Memory. By tieing different databased to different Cells and using CLM you can make your N-CPU system look more like N separate systems, which generally is a good thing for scaling. It may also not be required - one of those infamous "it depends" sorts of things. It may even be one of those things were getting some professional consulting might be helpful.
there is no rest for the wicked yet the virtuous have no pillows
timmy b.
Honored Contributor

Re: rx7620 memory allocation

Side note:
Everything above is true, IF you are in one big partition. If you have cut Cell 0 into it's own partition, then the cpus on Cell 0 only see the memory on Cell 0 and the same applies to cell 1. If you further carve the partitions down into VPARs, you can only access the amount of memory assigned to the VPAR.
There are 10 kinds of people in this world: Those who understand Binary, and those who don't.

Re: rx7620 memory allocation

As others have noted, if you're going from 4 to 6 processors I'm guessing you are adding another cell board to your rx7620 (unlesss you already have 2 cell boards with 2 CPUs on each?)

Anyway, when you go to 2 cell boards it absolutely makes sense to distribute the CPUs and memory evenly across the 2 boards. Introducing the 2nd cell board basically turns the system into a NUMA (non-uniform memory access) box - this means that memory latency is different when accessing 'local' (in the same cell) and 'remote' (in another cell) memory. These NUMA features can lead to some interseting performance issues, so HP introduced the capability within Integrity systems to specify a portion of a cell boards memory as 'cell local memory' - cell local memory is dedicated to just the processors in that cell.

Now for W2K3, HP reccomend that you setup 100% of the memory on a cell as 'cell local memory', I'm not sure why but I'm guessing that W2K3 doesn't perform well in a NUMA configuration. This means that in your configuration although Rick is correct when he says that Itanium memory isn't tied to the CPU, following the HP reccomended guidelines you will effectively have 2 x (3 CPUs with 12GB memory) enforced not by the processor, but by the chipset. If you don't think that's enough I guess you have 2 choices:

1. Ignore the HP reccomendation and set CLM at less than 100% - whether this leads to better performance or not I don't know.

2. Buy some more memory - it's probably cheaper than when you first bought the box, but still not that cheap!

For reference, have a look at this doc:

http://docs.hp.com/en/5990-8230/5990-8230.pdf

The reccomendation for 100% CLM is on page 12.

HTH

Duncan

I am an HPE Employee
Accept or Kudo
Ted Buis
Honored Contributor

Re: rx7620 memory allocation

Each cell board has room for 4 memory Quads. I would load memory so that 2 or 4 memory quads are on each cell board. One or three memory quads in a cell is not optimal. HP sell memory as a quad for the rx7620, so if you buy 4GB, you will get 4x1GB. Memory must be installed as a quad.
Mom 6