BladeSystem - General
cancel
Showing results for 
Search instead for 
Did you mean: 

2 new c7000s - fans blowing at 89% - why?

Willy Wong
Advisor

2 new c7000s - fans blowing at 89% - why?

We have 13 c7000s, and just received 3 new c7000s.

We've populated 2 c7000s with 10 BL460 G6s each and spread them out. In the third c7000, it is populated with 3 x BL460 G5 (old ones), to be populated a little later with newer G6s.

We've upgraded the OAs to 2.60 for all three enclosures. For some reason, the 2 c7000s with the G6s have ALL the fans blowing at 89%.

No reason why - the ambient temperature is 17C & 19C respectively.

None of our fully loaded c7000s are close. Sounds like a jet engine in our data centre just from these 2 enclosures, and that's saying something.

Has anyone seen this? Could it be a OA firmware issue? I haven't upgraded to 2.60 on any of the old enclosures.

Any help is appreciated...
25 REPLIES

Re: 2 new c7000s - fans blowing at 89% - why?

Hello Willy,

Have you updated the Power Management controller firmware on the G6 Blades as well as the iLo firmware?

I think you can get all this at hp.com/go/support and search BL460 g6
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

The Power management controller firmware and iLO firmware are all up-to-date.

Have reseated both OAs as well.

Logged a call to HP, but haven't seen anything yet.
marcus1234
Honored Contributor

Re: 2 new c7000s - fans blowing at 89% - why?



hi willy ,

first ensure no blanking plates are missing,
from blade slots or power supply slots as these fans are designed on a closed enclosure for optimal airflow..

take all firmwares as it is ultimately the OA that manages the enclosure , not the individual blades but blades need firmwares inplace too use guide below

http://h18004.www1.hp.com/products/blades/components/c-class.html#tab3_content

then give oa reboot command from cli when all done and should be okay

enjoy :)

This is how we thank in the forum

http://forums11.itrc.hp.com/service/forums/helptips.do?#33
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

Hi Mark,
Thanks for the advice - I'll assign points later, but I'll give an update to the issue.

The two enclosures are Gen 2 c7000s. Might be a firmware issue with the OA.

Found the cause of the fans blowing at 89%. A number of servers had the virtual fan blowing at 100% (see attached bitmap). You can see it by going to the Web Admin for the iLO of the blade, and then looking at the System Info/Fan tab.

To fix the issue, I simply reset the iLO within Diagnostics.

However, everytime I reboot these particular servers, the fan kicks in again to 100%. I then again reset the iLO to make it go back to normal.

Peculiar to say the least. Working with HP to figure out what is going on. Reseating the blades don't help. I may reflash the firmware of the iLOs first to see if it helps. Just haven't had the time yet.

Will keep everyone posted!
hungry
Frequent Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

perhaps it is indeed related to the ilo fw:

http://www11.itrc.hp.com/service/cki/docDisplay.do?docLocale=en&docId=emr_na-c01113006-14
You got a dream... You gotta protect it. People can't do somethin' themselves, they wanna tell you you can't do it. If you want somethin', go get it. Period. - Pursuit of Happyness

Re: 2 new c7000s - fans blowing at 89% - why?

There is a compatibility matrix showing where all the firmwares should be on the bladesystem page..
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

iLO f/w is up-to-date at 1.80 (sorry - I didn't specify in previous post).

OA f/w is up-to-date at 2.60.

I reflashed the f/w on one of the blades with the issue, but it didn't help. As soon as I reboot it, the fan kicks up again.

Awaiting a response from HP.
hungry
Frequent Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

have you checked whether the "increased cooling" option has been set in RBSU of the G6s? If so please try to set it to "optimal cooling"
You got a dream... You gotta protect it. People can't do somethin' themselves, they wanna tell you you can't do it. If you want somethin', go get it. Period. - Pursuit of Happyness
marcus1234
Honored Contributor

Re: 2 new c7000s - fans blowing at 89% - why?

willy the oa administers the whole enclosure seen similar issues before ensure all firmewares are updated and drivers too but especially firmware

many peopleonly update some firmwwares , on some blades and issue persists , this is because the 0A dynamically administers the enclosure and ift does not work well with differant , mixed firmwares especially if the blades are the same generation seen this many times ..

first update all sys boards then all ilo

then send reboot command to OA from cli all fans will spin up full speed, this is normal they will fall back in circa 60 seconds ,

now see if issue happens again ,,


This is how we thank in the forum

http://forums11.itrc.hp.com/service/forums/helptips.do?#33
marcus1234
Honored Contributor

Re: 2 new c7000s - fans blowing at 89% - why?

wil from the oa on the left click on rack firmware , post screenshot,,ty
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

I've attached a snapshot of the enclosure with the firmware we've installed, which is pretty much up-to-date (except for the VC - there is a 2.31 which I haven't applied yet).

I've restarted the OA from the CLI.

Same issue after I reboot one of the affected servers.

Fortunately, none of the blades are prod yet.

Thanx!
hungry
Frequent Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

thanks for the snapshot.

the fw looks fine to me (but still depends of course how you installed them, ideally the bios of the blades first, then the ilo, then the vc, and last but not least the oa)

do you use 10 fans per enclosure? (which would be advisable of course, or at least 8)

have you checked whether the "increased cooling" option has been set in RBSU of the G6s? If so please try to set it to "optimal cooling"
You got a dream... You gotta protect it. People can't do somethin' themselves, they wanna tell you you can't do it. If you want somethin', go get it. Period. - Pursuit of Happyness
marcus1234
Honored Contributor

Re: 2 new c7000s - fans blowing at 89% - why?

ok willy this looks fine and narrows our possibilities to find the culprit which is good ....

i suggest reboot all blades then OA
if issue persists

verify all firmware installed from the firmware cd and the drivers from the software psp cd too

tape to psu from other enclosure put them in this enclosure and swap around does issue folllow the psu units

if issues persists next logical step appears to be put one of these blades in the enclosure where there are no issues and one of those blades in this enclosure

let us know what happens then
marcus1234
Honored Contributor

Re: 2 new c7000s - fans blowing at 89% - why?

hmm .......nearly forgot also verify power settings in the enclosures are same makes it easier to cross check

Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

Thanks for the suggestions.

Confirmed "Optimal cooling" is set in the RBSU.

Power settings are the same in the enclosures.

To make it weirder:
1. I reboot the server. Fan goes to 100%.
2. Let server stay up for a bit - fan still at 100%.
3. I reboot the server (I don't reset the iLO) - fan goes back to normal and appears to stay that way.

Didn't have much time for testing today - I'll attempt to do more tomorrow...

Richard Newman
Occasional Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

I'm seeing this exact same thing since applying iLO 1.80. Any known resolution before I have to use my own brain power?
Richard Newman
Occasional Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

Having "Optimal cooling" set in the RBSU is the source of this problem. Choose the other option and you'll find this will be resolved.

I guess this must be an issue with 1.80 which will be adjusted sooner or lated.
Timur S Lukovkin
Regular Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

Do you have an OS installed in these servers?

If you dont have one, this behavior is normal, you need the ILO Managament and ILO interface driver installed so the fans can spin according to the server needs, ILO driver will communicate with ILO and OA so it will provide the temperature information for each server.

Without an ILO driver the fans will operate at 89%.

You can check ILO Web admin status information for the virtual fan, should be at 19%.
Richard Newman
Occasional Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

That is entirely NOT true.

These blades worked fine for weeks, right up to the install of iLO 1.80, and from then on we had the 89% fan issue constantly.

We have 2 blades that are part of this discussion, one has an OS on it (Server 2008) and the other has vmware's ESX 4.0.
Timur S Lukovkin
Regular Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

In that this is a new scenario and it should be investigated.
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

I logged a call with HP a while back. After a lot of back and forth, it appears they've been able to replicate the issue.

They have included the fix in 1.81 apparently, but are awaiting validation. No ETA yet.

Workaround is to either go back to 1.79 or to do the iLO reset on the blade exhibiting the fan issue, as in my previous post.

Hope that helps.
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

By the way Richard, are your enclosures second Gen?

Just curious...

HP only got back to me about 1.81 on Dec. 10th, so it still may be a bit of time yet.
Richard Newman
Occasional Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

Yes I believe the enclosure here is 2nd generation (it's brand-spanking new). I have a 1st gen enclosure which has NOT seen this issue.

I'm glad we've been able to agree a couple of different work-arounds and acknowledge that this is an issue which will soon be fixed. I wonder what 1.81 will break.
Willy Wong
Advisor

Re: 2 new c7000s - fans blowing at 89% - why?

HP has released 1.81 firmware for iLO which appears to fix this issue. Will be deploying it shortly.

Seems to fix only fan issues.

Will see if it is introduces anything unwanted.