Memory operatiing at lower frequency than expected.

[Capcent]

We have a Proliant Dl380p G9 server in which we recently installed 16 modules of 16 Gb  2133 Mt/s memory. However, the configured speed is 1866 and as far as I´m concerned this configuration should run at 2133 Mt/s with these CPUs (2 x 2686v4).

Any suggestions what might cause this?

All modules are of same spec and brand, samsung m393a2g40db0-cpb and are installed in the correct slots (blue and white)

iLO shows every module as identical and OK, with maximum frequency stated as 2133 on every module. However the configured speed is 1866. AMP Mode is Advanced ECC.

Is there something I´missing here? Any help is appreciated.

Edit: Firmware is 2.94

[Anu_K]

Hello Capcent ,

Could you check in BIOS if the 'Maximum Memory bus frequecy' is set to 2133 MHz?

You could find this option under power management --> Advanced power options --> Maximum Memory bus frequecy.

If this option is set to Auto or 1866 MHz, please modify it to 2133 MGz.

Hope it helps!

[pchops]

Can you confirm the CPU model you stated?:

---

(2 x 2686v4).

Said model is not listed in the ProLiant DL380 Gen9 quickspecs:

https://www.hpe.com/psnow/doc/c04346247

or on the "HPE ProLiant DL380 Gen9 Server - Configuring Memory" page:

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c05240460

and I can't find it on Intel's "Ark" site. The googley finds some references to the E5-2686 as an OEM CPU for Amazon. If the latter is the case, perhaps that has something to do with it? Just a thought...

[Capcent]

Hi,

Thanks for the tip, the option was indeed set to Auto, but after setting it to 2133 it unfortunately still configures as 1866.

I know this CPU isn't publicly supported on the dl380p but from the information I've ben able to gather it should indeed support 2133 memory.

Back to the drawing board it is

[Capcent]

Yes I can confirm it´s that CPU and I´m aware that it´s not supported publicly. However, the CPU itself should be capable of running 2133 Memory from my research. The more relevant question is if it´s artificially capped in the Proliant BIOS.

It might also be that the combination of these RAM modules and this CPU does not match well for some reason, but that´s very unlikely.

Anyway, thanks for the input, any suggestions are welcome.

[pchops]

Thanks for confirming the CPU model. I don't know if that CPU is "artificially capped" in the BIOS, however in your original post, you had stated:

---

All modules are of same spec and brand, samsung m393a2g40db0-cpb and are installed in the correct slots (blue and white)

However, the correct load order should be white slots and black slots, for two DIMMs per channel (16 DIMMs in a two-CPU system). See the previously mentioned "HPE ProLiant DL380 Gen9 Server - Configuring Memory" page.

[Capcent]

Must have had a brainfart when writing the initial post, I can confirm that the modules are indeed in the black and white slots. Sorry for the confusion.

The server was originally configured with V3 CPUs. Does the mainboard model/revision impact maximum frequency? Can't confirm mainboard revision right now as I'm not physically where the servers are located. Seems like it shouldn't though?

Anyway, maybe I just have to accept running at 1866.

[pchops]

FYI -- there is an upgrade procedure required when going from v3 to v4 processors:

https://support.hpe.com/hpesc/public/docDisplay?docId=c04849981

However, if you are already running with v4 processors installed, then the SPS (Management Engine) firmware might already be at, or above, the minimum required version.

[Capcent]

I did actually not know that, but the V4 CPUs runs just fine apart from the memory speed configuration "issue". I´m starting to think that since it´s originally an OEM CPU it probably is artificially restricted for some reason, or that it restricts mem speed with this particular memory module as it´s not original HPE memory. Not sure really, but it feels like I´ve tried everything now.

We do have a spare mainboard so I could give that a try next time I´m hands-on with the server, but other than that I´m kinda out of options.

[pchops]

---

that it restricts mem speed with this particular memory module as it´s not original HPE memory. Not sure really, but it feels like I´ve tried everything now.

---

I think you are correct that the non-HPE memory is probably the issue. You need to use HPE SmartMemory in order to run at 2133 with two DIMMs per channel:

Sorry for the non-HPE source:

https://www.insight.com/content/dam/insight-web/en_US/article-images/whitepapers/partner-whitepapers/HP%20ProLiant%20Gen9%20Servers.pdf

[Capcent]

But as far as I understand, that document and those population rules only applies for v3 CPUs. For v4 CPUs the cutoff is at 2133 for two DIMMS, at least that is my understanding.

Nevertheless, I'm now fairly sure this "issue" is specific to this CPU model and not something that I've overlooked.

[pchops]

Well, since we've not quite beat this topic to death :), I'm not sure I agree with your conclusion. That is, I decided to do a little experimenting with my DL360 Gen9 (yes, different system model than yours, but it uses the same system board and BIOS family as the DL380 Gen9). I tested some 16GB 2133P-R and 2400T-R HPE SmartMemory against "generic" memory of the same size/speed. In my test setup, the closest CPU to your E5-2686v4 that I had available was an E5-2680v4 CPU (supporting DDR4 1600/1866/2133/2400), where the latter CPU model *is* supported on both the DL360 Gen9 and DL380 Gen9 (according to the quickspecs). The results would seem to confirm that it is the non-"HPE SmartMemory" causing the behavior you are seeing (see below). Note that I downgraded the BIOS to v2.94 to match your system (the latest version appears to be v3.02), reset all BIOS settings to defaults, and then did four test configurations as follows:

first run: 16GB 2Rx4 PC4-2133P-R HPE SmartMemory

second run: 16GB 2Rx4 PC4-2133P-R generic

third run: 16GB 2Rx PC4-2400T-R HPE SmartMemory

fourth run: 16GB 2Rx PC4-2400T-R generic

These results lead me to believe that you could most likely rule out that there is some "artificially restricted" thing going on with your particular CPU, since I see the same behavior you are seeing with a supported CPU (unless, of course, your CPU can't run at those memory speeds, but that seems unlikely). However, one could argue that the "restriction" is one of HPE versus non-HPE memory, although my google-foo is not good enough to find a reference that shows what the "standard" DIMMs per channel speed is for the Intel E5-2600v4 CPUs, other than the quite ambiguous "up to 2400 MT/s". Curiously, the older versions of the HPE SmartMemory quickspecs states:

https://www.hpe.com/psnow/doc/c04355083.html?jumpid=in_pdp-psnow-qs&ver=6

NOTE: HPE Proliant Gen9 DL and ML servers will work @2400 MHz at 2DPC whereas all the other competition will go down to 2133 MHz at 2DPC which means [(2400-2133)/2133]=12% better performance.

However, this table in said version of the quickspecs only shows the 2400 MT/s speed with two DIMMs per channel for load-reduced DIMMs:

Also, the above information was removed in later versions of the SmartMemory quickspecs, but the behavior I'm seeing contradicts said table anyway, since I can get 2400 MT/s with two DIMMs per channel using registered DIMMs, providing they are "SmartMemory". I guess the only way to confirm this would be for you to get some HPE SmartMemory and see what happens.

[Capcent]

What an awesome community this is, thank you so much for all the effort!

I assumed only 2400 MT/s needed HPE smart memory but I guess I was wrong, as clearly seen from your tests. This helps a lot, as I now know that I have to swap it for HPE memory should we need more bandwidth.

Huge thanks again!