Can't run 64-bit guest on host with 64-bit support

[Shaun McCloud]

We added a new server to our ESXi "cluster" (i.e. made it a cluster). It is a HP DL140 G3 w/ 2 Xeon 5140 CPUs, 32GB RAM & 2 73GB 10k RPM SAS Drives.

I can run 32-bit hosts on it fine and according to the VMware CPUID utility it supports 64-bit guests.
The following are the supported items from the CPUID utility.
[list]
[*]Nx/XD
[*]CMPXCHG16B
[*]Hyperthreading
[*]64-bit Longmode
[*]64-bit VMware
[*]Xeon/Core2 EVC Modes
[*]SSE1,SSE2,SSE3,SSSE3[/list]

Does anyone have a clue why I can't run a 64-bit guest on the machine?

I know its not officially supported, but it should still work if the VMware tools say that VT is supported right?

[Shaun McCloud]

Ok, I've figured out that I have one B-stepping 5140 and one G-stepping 5140. According to Intel, either one processor must be swapped out with one that matches the stepping on the other processor or a BIOS workaround needs to ben enabled. Does the DL140 G3 have this workaround in the BIOS? See the PDF below on page 47 for what I'm reading. http://www.intel.com/assets/PDF/specupdate/313356.pdf

[SMR]

Hey Shaun,

Like you said the processors probably have all the required features for a 64bit guest to run (64bit instruction set, Intel VT support etc) but unless the BIOS on the DL140 G3 has an specific option to enable Intel Virtualization... I believe it won't ever work.

You could try a BIOS update...

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=1842838&swItem=MTX-c4304c249aea4a478d098b5254&prodNameId=3285485&swEnvOID=1005&swLang=8&taskId=135&mode=3


... and see if that helps. If you don't have the option to enable Intl VT then there probably is a limitation on the chipset of the DL140 G3 that keeps it from utilizing the VT capabilities of the processors :(

[Shaun McCloud]

SMR,
There is a BIOS option for VT on the DL140 G3 & it is enabled. Found out the issue is caused by one CPU being a B stepping and the other being a G stepping and HP not implementing the workaround provided by Intel for that setup.

[Michael A. McKenney]

You always want to match stepping on CPUs. Never mix them. Strange things can happen.

[Shaun McCloud]

Well, the second CPU was ordered from HP so specifically for the server. It should have matched but stuff happens.

[Michael A. McKenney]

Stepping changes all the time. I usually order CPUs in pairs when I add them to make sure they match.

[Shaun McCloud]

Yeah,
IT is going to get in touch with HP right after they put out the fire caused by a UPS going bad overnight.

[SMR]

Have you tried removing one CPU and attempting to run the 64bit guest like that?

That should confirm that stepping is indeed th problem.

[Shaun McCloud]

Yes, I have done that and it works fine.

[SMR]

Cool, now the workaround mentioned by Intel may have been delivered to the DL140 G3 in the form of a BIOS update. Did you make sure you're running the 2009.06.03 BIOS version?

[Shaun McCloud]

Yep, I'm sure. I flashed it yesterday.