Hyper-V on DL380 Gen 9 E5-2650V3

[nickexdev]

Hi.

I have a D380 G9 server with dual E5-2650 V3 CPUs running Hyper-V on Windows Server 2012R2 Standard. Everything is patched to the latest versions - Windows, BIOS, firmware, drivers etc. (spp2016040).

Without the hypervisor running, everything performs as expected.

When I run with the hypervisor and the default 'dynamic power saving' scheme which uses c-state c6, the cores on CPU0 all run at the correct speeds but the cores on CPU1 are all locked to multiplier x12 so always run at a very slow speed (1.16ghz).

So for example if I give a certain workload to cores on CPU0 it takes 13 seconds but the same workload given to cores on CPU1 takes 32 seconds.

If I disable the c-states by enabling the 'maximum performance' mode in the BIOS, the second CPU starts running at the correct speed although now I think the Turbo Boost doesn't work anymore - the workload takes 16 seconds in this mode (on either CPU) which is more than the 13 seconds it took before.

I tried the other c-states but c3 seems to be like c6 and c1e is a bit different - it slows down the last four cores of CPU0 leaving only 6 cores running at normal speed and 14 cores running slowly.

All this relates to running things on the Hyper-V host, not from within a VM.

I just need to know if this is expected for the E5-2600V3 CPUs and the only way to run Hyper-V with reasonable performance is to always use the power-hungry 'maximum performance' mode.

[waaronb]

On my Gen8 and Gen9 DL380s, I haven't seen this issue.  The only time I've ever seen one CPU running slower than the other is when there is (probably) some thermal throttling on the 2nd CPU that keeps it from using the maximum turbo boosts available, so it might only be running at stock speed while CPU #1 is turbo boosting.  I use CPU-Z to read the values, FYI.

Maybe there is a thermal issue that's preventing your other CPU from entering higher speeds normally?  Are both CPU's installed from the factory, or did you install one of them after purchase and perhaps the thermal paste wasn't applied smoothly?  Try looking at the temp readings for both CPUs when they're under a full load (use a CPU stress tester) to see if one is running hotter than the other.

On my DL380 Gen9 with max performance and max cooling enabled, I get CPU temps of 43 and 47 Celsius under a full stress test (ambient temp is reading 28 C), so that might help give you an idea of what it would normally be.

Setting the server to max performance shouldn't disable the turbo boost, but like I said, if there are thermal issues, the CPU might not boost to a higher speed.  Depending on the type of virtual machine workload, you may see a better overall performance boost by enabling max performance, which basically keeps the CPU from going into lower power states.  Your power usage will be a little higher, but if your CPU usage is high on average anyway, it won't matter, and it should be a little more responsive in general without toggling into different c-states all the time.