Proliant G6 Memory SubSystem

[dbx13]

We're setting up a couple DL370 G6 machines as Exchange 2007 servers.

I understand the memory per CPU is three channels across 9 slots (3 slots per channel). Right now we have three 2GB PC3-10600 DIMMs in channel 1 (slots A, B and C) per CPU, using the default Advanced ECC config.

What is the advantage of this new memory subsystem designed with channels? And specifically, if I want to add more memory, is it best to add the DIMM in sets of three, always filling all three slots of a channel? Is there a performance gain in adding memory per channel, or can you just add one DIMM at a time? If I add one DIMM to my existing config as above, will I reduce the performance of the existing 2GB DIMMs which are currently installed on a per channel basis?

Lots of questions! :-) Thank you for any advice.

[Adrian Clint]

Does this help?
http://h18000.www1.hp.com/products/servers/options/tool/hp_memtool.html

And the DDR3 Whitepaper linked from here
http://h18004.www1.hp.com/products/servers/options/memory-description.html

[WFHC-WI]

Hi Jenny,

Per the 370 G6 quickspecs here:
http://h18004.www1.hp.com/products/quickspecs/13242_na/13242_na.html

"It is not required, but it is recommended to load the channels similarly if possible."

Installing one more DIMM per processor would be acceptable in your configuration.

The performance hit as I understand it comes only from mixing speeds or not balancing between processors (although both are acceptable.)

You might also want to have a look at the memory configuration tool for the G6 servers:
http://h18004.www1.hp.com/products/servers/options/tool/hp_memtool.html

good luck, assign points if this answers your question!

[dbx13]

Hi Adrian, the first link to the configurator does not help much in describing the performance of the DDR3 memory channels, nor is it really useful in configuring the system for memory, I've tried it! :-)

The second link has a small reference, stating "With 6 channels for DDR3, this is like driving faster & having more lanes to drive in."

Still have questions here...

[dbx13]

Hi WFH,

The quickspecs says I can add one DIMM at a time, yes, but is there a performqance gain in filling all three slots of a channel? I cannot find that answer in the quickspecs, or anywhere, yet.

[dbx13]

Adrian, post again and I can assign appropriate points. The white paper link on your first link to me was exactly what I needed! Thanks!

[Adrian Clint · Accepted solution]

I found more info from an update email from a guy at HP....... here it comes.....

Symmetrical distribution of DIMMs accross processors:
Each processor has 3 channels and you should populate all in groups of 3 identical DIMMs (this interleaving is distributing memory across all the three channels of a CPU for better bandwidth). In one processor system your optimal memory will be divisible by three, and more the case, in two processor configurations by 6 e.g.12GB, 24GB, 48GB, 60GB, 72GB, 96GB, 108GB, 120GB, 144GB etc (as opposed to the old days of 16GB,36B, 64GB etc).

You can mix DIMM sizes in sets of different capacity DIMMs at the same operating speed. For example, latency and throughput will not be negatively impacted by installing 6 x 4GB Dual-Rank DDR3-1333 DIMMs (one per channel), plus 6 x 2GB Dual-Rank DDR3-1333 DIMMs (one per channel)

Memory Speeds
PC3-10600 or 1333MHz memory speed - 3.5% to 9% higher than 1066MHz. CPUs that support DDR3-1333 memory all start with X, e.g x5570, X5560 etc. HP has engineered its ProLiant G6 servers to reliability operate at 1333MHz speeds with 2 DIMMs per channel which is an industry unique where only one is the norm. Supported at 1333 is activated via a setting in ROM based setup utility (RBSU).

If you have to got to populate three 3 DIMMs per channel, as is possible in the DL380, BL490 etc, you automatically slow down to the PC3-8500 speed of 1066MHz. So be careful here, this is not such an issue with 4GB memory as the price sweet spot is the 4GB dual rack 500658-B21 that is PC3-10600 Memory 1333MHz memory speed. Not so in the 8GB space where there is a vast difference in the PC3-10600 or 1333MHz memory speed price and the PC3-8500 1066MHz.

There is no benefit to buying the expensive 500662-B21 8GB PC3-10600 Memory Kits in 3 DIMM configurations. Likewise if the processor starts with an â Eâ like the E5540, E5520 they all only support the PC3-8500 speed of 1066MHz from the outset. The best memory to buy if you are in the 8GB DIMM space is the 516423-B21 8GB PC3-8500.

However this is worth paying attention to however from the outset. As when you have E processors and 3 DIMMs per channel with PC3-8500 or 1066MHz DIMMs per channel you will automatically drop back to this speed PC3-6400 or 800MHz. This is 20% to 25% less than PC3-8500 or 1066MHz

Quad Rank is something to observe carefully in 18 DIMM configurations. If you use Quad Rank you can only use 12 DIMM bays and it is due to the distance of the signal path. Quad-ranked DIMMs offer slightly more performance at their rated speed and consume about 15% less power than dual-rank DIMMs - hence the price premium.

[dbx13]

Thanks Adrian!