numa on hpdl385 esx with 2x2gig and 2x1 gig on 2 procs

[paulow1978]

I have an hp dl385 with 2 dual core opterons. I am running vmware esx 3.01 on it. I have 2x2 gig memory and 2x1 gig memory. According to NUMA rules on the dl385 I have to fit exactly the same memory in pairs into each bank. So I have put 2x2 gig in one bank on one side for proc 1 and 2x1 gig in one bank on the other side for proc 2. I have turned node interleaving off so that vmware picks it up as a numa system (any other memory configuration does not allow the system to boot properly). However in vmware under /proc/vmware/numa/hardware I get that 4 gig is assigned to proc1 and 2 gig to proc2. I assumed this would happen. Anyway, I get warinings in my vmkwarning file about numa being misconfigured. I assume this is because the memory is lopsided (i.e. 4 gig to roc 1 and 2 gig to proc 2). I can't find a way to assign 3 gig to each in a way that the dl385 will accept it. My question is therefore... in this configuration am I better off just using node interleaving rather than a lopsided numa based system?

thanks for ay replies or help

Paul W

[Dan Robinson_4]

I think you got it backwards

NUMA says that each Numa "Node" (Each Processor on a DL385/DL585) must contain the same amount of Memory.

However HP specifically says on the 585 (not sure on 385) that you cannot mix Chip Sizes on the same processor card. So you HAVE to keep your 2GB chips on 1 processor and your 1GB chips on the other.

So its a catch 22.
Your basically screwed unless you use ALL the same size chip across the board.
All 1GB or All 2GB



We did something similar to what your doing but with 3 585s and had a mix of 512MB chips, 1GB chips and 2GB chips. No one told us when we first ordered them that this config wasnt gonna cut it.
We ended up doing this:
16GB of 512MB chips in Server 1 (Maxed sockets) + A Bag full of extra 512MB Chips thats still sitting on my desk.
24GB of 1GB Chips in Server 2 (6 of 8 sockets per CPU)
32GB of 2GB Chips in Server 3 (4 of 8 sockets per CPU)

[dcs_1]

We had a similar situation with DL385 G1 servers and ended up dumping the 512MB sticks that came with the boxes. HP really should explain ESX memory configurations to their vendors.