topic Re: High CPU Temp on Proliant G7 Servers in ProLiant Servers (ML,DL,SL)

High CPU Temp on Proliant G7 Servers

Stephan G — Sun, 26 May 2013 08:19:09 GMT

Hi,

i think there is a bug either with the iLO Firmware 1.50 or BIOS.

Starting with Friday 4 of our Gen 7 Servers stating high temperatures of the CPU while the other servers are operating fine.

You can fix it when you power off the server and make it completely powerless. That's why i think it's a bug of the iLO Firmware. After a reset of the iLO or upgrading the firmware (which also resets the iLO) the CPU temp is back to 40 degrees.

I shut down a server for a whole night and after restarting it this morning it still says to high temperature. 82 degrees CPU. the other temps are just fine.

Does anyone else experience these problems ?

Greets

Stephan

Re: High CPU Temp on Proliant G7 Servers

Jan Soska — Sun, 26 May 2013 12:45:07 GMT

Hello,

Is this behavior the same in all g7 server (any type)?

we have some g7 servers with latest ilo - 1.55 form march 2013 and we do not have such issues.

So my recomendation is to install newer ilo (1.55), here is link for win2008r2 -

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=4154847&prodTypeId=18964&prodSeriesId=4154735&swLang=8&taskId=135&swEnvOID=4064

and come back with results...

Jan

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Sun, 26 May 2013 14:46:53 GMT

I upgraded already all servers.

They are DL380 G7 or DL360 G7.

I just want to make sure it is the iLO and not another fault.

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Mon, 27 May 2013 12:37:37 GMT

Doesn't help. After 12 hours the servers stopping and starting again.

HP Support case is opened.

Re: High CPU Temp on Proliant G7 Servers

Oscar A. Perez — Tue, 28 May 2013 01:33:24 GMT

Does it stop if you revert iLO3 firmware back to version 1.28?

See the below advisory. It is for the DL980 G7 but, I've seen reports that it is happening on other G7 servers.
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c03720197

How often can you reproduce it? I could provide you with a beta 1.57 that fixes the DL980 issue for a quick testing.

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 28 May 2013 05:20:58 GMT

Great. That's exactly what is happening.

At the moment the servers are stable. i can't reproduce it when i want, it just happens.

It would be great if you can send me a link to this beta firmware. So i can try it on one server.

Greets

Stephan

Re: High CPU Temp on Proliant G7 Servers

Oscar A. Perez — Sun, 21 Jul 2013 13:58:22 GMT

You can download it from here:

ftp://ilo4me:G!v3t2me@ftp.usa.hp.com

Re: High CPU Temp on Proliant G7 Servers

Oscar A. Perez — Mon, 03 Jun 2013 13:23:25 GMT

Hi,

Have you had the chance to test the iLO3 beta 1.57?

Thanks

Oscar

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Mon, 03 Jun 2013 17:05:11 GMT

Hello,

not yet. It did not happen again.

If it does i will apply the firmware.

Greets

Stephan

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 02 Jul 2013 09:40:36 GMT

I applied the new non beta 1.57.

It happened again with two servers. HP is on it. Noone else with these problems ?

It's not that cold in our the room where the servers are. But it happens only on G7 Servers.

All others (G4 to G6) are operating normal.

Greets

Stephan

Re: High CPU Temp on Proliant G7 Servers

Sbrown — Thu, 04 Jul 2013 11:37:20 GMT

Disable the thermal shutdown. If the servers are > 2 years old, it is time to regoop with arctic silver if they have been run in a warm condition!

Also some cpu's have failing sensors inside of them, your choice is catastrophic shutdown (thermal shutdown) or buy new cpu - you know what damage happens to your o/s filesystem, would you rather have that or monitor your temperatures another way!

The amount of damage thermal shutdown does with its abrupt cold power down is very bad. I'd take my odds with a server out of warranty and let it overheat and die gracefully, but you can query sea of sensors and use environmental monitoring to do a better job than ILO , but that is only my opinion!

If you like watching RAID-5/6 rebuilds - keep it on! the sensors on the motherboard and in the cpu do fail. it is fact. I'd RMA if I was under warranty but I am not. Once out of warranty, disable thermal shutdown. It is not like there is 1 fan blowing on the cpu's like a white-box desktop!

Re: High CPU Temp on Proliant G7 Servers

LFnet — Tue, 09 Jul 2013 10:13:20 GMT

For the record, we're seeing this same problem on one DL380 G7, iLO Firmware 1.50.

The read temperature has been oscillating between 40°C and 82°C for about 20 minutes now.

All other systems from the same batch are fine, and this one has been working flawlessly since 2011-08.

Edit: Resetting the iLO seems to have fixed the problem for now.

Re: High CPU Temp on Proliant G7 Servers

Torsten. — Tue, 09 Jul 2013 12:21:56 GMT

>> The read temperature has been oscillating between 40°C and 82°C for about 20 minutes now.

I have seen the same, but it looks like ILO fw 1.57 solves this.

7266   Repaired   Environment   7/3/2013 10:37   7/3/2013 10:37   1   System Overheating (Temperature Sensor 3, Location CPU, Temperature 82C)
7265   Repaired   OS   7/3/2013 10:37   7/3/2013 10:36   1   Automatic Operating System Shutdown Initiated Due to Overheat Condition
7264   Repaired   Environment   7/3/2013 10:37   7/3/2013 10:36   1   System Overheating (Temperature Sensor 3, Location CPU, Temperature 40C)
7263   Repaired   OS   7/3/2013 10:36   7/3/2013 10:36   1   Automatic Operating System Shutdown Initiated Due to Overheat Condition
7262   Repaired   Environment   7/3/2013 10:36   7/3/2013 10:36   1   System Overheating (Temperature Sensor 3, Location CPU, Temperature 82C)
7261   Repaired   OS   7/3/2013 10:35   7/3/2013 10:35   1   Automatic Operating System Shutdown Initiated Due to Overheat Condition
7260   Repaired   Environment   7/3/2013 10:35   7/3/2013 10:35   1   System Overheating (Temperature Sensor 3, Location CPU, Temperature 40C)

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 09 Jul 2013 11:19:31 GMT

It says so but it does not. This only applies to DL980 G7.

My call already rised to 2nd level support. Let's see what happens next ;)

Re: High CPU Temp on Proliant G7 Servers

Torsten. — Tue, 09 Jul 2013 11:25:19 GMT

>> It says so but it does not.

Based on your test or on release notes?

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 09 Jul 2013 11:37:16 GMT

Based on my tests.

I installed it on several G7 servers. But some of them still "overheating" (they don't restart and don't really overheat).

I also installed the latest drivers for each component.

Re: High CPU Temp on Proliant G7 Servers

Torsten. — Tue, 09 Jul 2013 11:42:25 GMT

I have seen this on DL360 G7 and DL580 G7 with version 1.55.

What happens if you go back to 1.28?

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 09 Jul 2013 11:54:58 GMT

Not tried that yet.

I'm waiting for more input from HP.

Also .. the ilo gets resetted when applying a "new" firmware. After resetting the fault is not reappearing for some days.

Re: High CPU Temp on Proliant G7 Servers

Torsten. — Tue, 09 Jul 2013 12:08:23 GMT

If there are several servers with this issue, I would consider to downgrade one of them and see what happens.

Keep us posted.

Re: High CPU Temp on Proliant G7 Servers

Stephan G — Tue, 09 Jul 2013 12:16:47 GMT

I installed it on a DL380 G7.

I will get back to you at the end of this week.