DL580 G5 memory expansion problem

[Michele Pensotti]

Hi all, I have a very peculiar problem with an HP Proliant DL580 G5.
We bought this machine along with another identical one some months ago.
Both machines had 4 quad core CPUs installed and 16GB of ram installed.
Then we ordered a memory expansion to bring both machines to 32GB of ram.

The ram came in two 8GB kits (a total of four dimms of 4GB each) per server.

I started installing the RAM in the first server and when I powered it on it detected only 28GB of the 32GB that were installed.

So I took the second machine and did the very same steps and when powered on, it detected 32GB of ram, both in the BIOS and in the O.S. (Centos 5.2)
To be more specific, the BIOS reports 28GB in the first screen (topmost part of the screen as soon as the white "Proliant" screen disappears), but if I press F9 it reports 32768MB.
HOWEVER in the O.S. it only detects 28GB (MemTotal: 28823264 kB) , while the other server reports 32GB in all three places.

So I tried every possible combination to make the first server detect all the ram, but to no avail (the best I can get is 28GB), and quite frankly I'm completely puzzled.

Is there anyone that can help/explain this to me?

I'm attaching a small file with the exact schema of the banks before and after the expansion.
P.S: the servers have the same bios which is P61 09/01/2008, and also have the same O.S. which is CentOS 5.2 64bits

[KarloChacon · Accepted solution]

basic troubleshooting would be

- test the other memory modules from a server that is detecting 32GB so we can determine that all DIMMs are good

- second if with known good memory modules you still get 28GB so that means and memory slot is faulty so as easy as swap processor memory boards between 2 servers

[Michele Pensotti]

Thank you, I'll try both options and then report back here :-)

Michele

[Michele Pensotti]

Since the other "sane" server is in production and I still haven't got a chance to power it down and swap the dimms, I tried to ascertain if there is indeed a problem with its memory slots/dimms by filling all of its slots with 2GB banks (I have quite a lot of those available).

1st attemppt: first 4 banks 4x1gb + 6 banks 6x2gb (my original configuration) + last 4 banks 4x2gb (from my "reserve").
the server should recognize 28gb but it recognizes only 24gb.
2nd attempt: i removed the first 4x1gb banks and replaced those with 2gb banks from my "reserve".
the server should recognize 32gb ( 2gbx16 ) and instead puts out this error:
207-Memory connectivity problem deteced on Socket 8G
Attempt to reseat the failing DIMM.

So I reseated the dimm, even swapped with that of Socket 7E but to no avail, it keeps reporting the 207-error.

So I did one last try:
I replaced all of the originally-installed 2gb dimms with some other 2gb dimms from my "reserve".
ET VOILA' the server detected 32gb with no problem.
So it would seem a *very strange* dimm-socket compatibility problem.

I say *very strange* because if I put back all of the originally-installed dimms (including the supposedly faulty ones) as they were before the first expansion attempt the server does indeed work ok and detects all of the ram, INCLUDING the dimm inserted in socket 8G.

What do you think of this?

I'll keep up with more updates as soon as I can shut the other server down.

P.S: I also tried to update the BIOS (with the firmware maintenance cd 8.40), and got stuck with no ILO left! (GOSH!)
I got out of this problem by online-flashing the ILO firmware inside linux.

However even with the newer system BIOS the server still doesn't detect all the ram if I use the *suspect* dimms (all is ok with the "reserve" dimms).

Please excuse this post's lenght, but I have lots of informations to communicate and wouldn't know how else to tell you.

Bye,
Michele

[Michele Pensotti]

Hello again, this morning/afternoon I finally had my chance to bring both servers down.
I immediately swapped the "bad" 32GB from the "bad" server to the "good" server and this one did finally detect all of the 32GBs.
Then I put the "good" 32GB on the "bad" server and this too did detect 32GB! :-))

So it seems I have resolved my problem, thanks to you Temp1!
I'm assigning you the points right now :-)

Thanks again and have a nice weekend

Michele

[Michele Pensotti]

The solution was to swap all of the memory banks between the 2 identical servers (I'm glad, really glad I had 2 identical ones ;-) )

Thanks again,

Michele