Power Mgmt Controller, Firmware Update problem

[Nikaja]

Same problem here just using Windows.

It seems to be a problem introduced with 3.4B Power

[Patrick G.]

Making a disaster recovery solve the problem.

"sh CP010989.scexe --disaster-recovery"

[Sobhan Majumder]

I have 8 Blades in a C7000 enclosure. All are running with old iLO2 firmware and BIOS ver except one.
For the server which is up to date, one customer advisory is coming in OA and telling to update the PMC firmware to 3.4B.
Can I update the firmware in that particular server??
If I can then will this activity affect to other server??
I want to know this because rest of the server in production and downtime will not be available.
Please reply if anyone can help.

[Adrian Clint]

You can update the iLO firmware without affecting any server. You should do this first.
Both the BIOS and Power firmware will require a reboot. So you can update both firmware and then do a single reboot at a convenient time to you.

[Kerry Quillen]

Ok, maybe I'm just blind but where in OA does it tell what the PMC firmware version is? I've looked under 'rack firmware' but only see the OA and ILO versions. Can't find it in the SMH either.

[Kerry Quillen]

One other thing - I telent'd to OA and ran show server info cmd and it doesn't show PMC fw version either.

OA-1401-2-F4-ENC1-1> show server info 10

Server Blade #10 Information:
Type: Server Blade
Manufacturer: HP
Product Name: ProLiant BL460c G1
Part Number: 459483-B21
System Board Spare Part Number: 438249-001
Serial Number: 2UX90205YP
UUID: 34393534-3338-5532-5839-303230355950
Server Name: BLADETEST02T
Asset Tag: [Unknown]
ROM Version: I15 11/02/2008

CPU 1: Quad-Core Intel Xeon 3000 MHz
CPU 2: Quad-Core Intel Xeon 3000 MHz
Memory: 16384 MB

Embedded Ethernet
NIC 1 MAC Address: 00:17:A4:77:18:14
NIC 2 MAC Address: 00:17:A4:77:18:16
iSCSI 1 MAC Address: 00:17:A4:77:18:15
iSCSI 2 MAC Address: 00:17:A4:77:18:17

QLogic QMH2462 4Gb FC HBA for HP c-Class BladeSystem
Port 1: 50:01:43:80:03:bd:5e:fc
Port 2: 50:01:43:80:03:bd:5e:fe

Management Processor Information:
Type: iLO2
Name: bladetest02tr
Firmware Version: 1.78 Jun 10 2009
IP Address: 10.45.98.89
MAC Address: 00:23:7D:22:F7:88

[Michael Garner_1]

You must be running OA firmware 2.51 or later to get the Power Management Controller firmware version.

[Adrian Clint]

New power firmware 3.4 rev C is available for download.

[Chris Fricke]

OK - so I see a Windows and a Redhat download for 3.4C - but how do I get this for my ESX server?

[IT Network Administrato]

To update my ESX servers I did the following:

1. Download the Linux version of the update. This should give you a .scexe file

2. Connect to any server's iLO webpage and use the Virtual Media Applet to create a disk image containing the file downloaded

3. Place the ESX Host into Maintenance Mode and reboot

4. Boot from the latest HP Firmware Maintenance CD/ISO (I use iLO to do this as it saves ever having to leave my desk)

5. Once booted, start a firmware update. The hardware will be scanned and you will end up with a list of firmware updates applicable to the server (some or all may not be required depending if you've previously updated the firmware)

6. Attach the Disk Image created in Step 2 via iLO, as a "Virtual Floppy/USBKey"

7. On the Firmware screen on the console, click the "Supplemental Updates" button at the bottom

8. The Disk Image is scanned and the Power Management update is detected and added to the list of available firmware to install

9. Install the firmware as normal

10. Power off the Blade when finished

11. SSH to the Onboard Administrator and issue "reset bay #" (where # is the bay number of the blade). You can achieve this same step by manually removing the blade and re-inserting into the enclosure, but why leave your desk? ;-)

12. Once your blade is powered back on, confirm ESX boots ok, then check the Firmware Overview via the OA Webpage and the Power Management Firmware should now show as updated

I hope this helps and makes sense ... I find it a lot easier to just do it than explain it.

Cheers,
Chris

[mcarroll]

I've been having a problem with the power management on one of our ProLiant BL680c G5s, in that it's redlighting with the status "Major degraded" and with the message "Wrong power management controller version: The power management controller firmware on this blade must be upgraded. Please see customer advisory c01668472."

I've been through and made sure that the power management firmware (and all the other firmware for that matter) is up to date, but it still has the same problem. The other blades in the same enclosure have had the same firmware updates and are fine.

Has anyone else had this problem/has any idea how to solve it?

[Adrian Clint]

I had that problem on some BL495G5s, HP replaced the motherboards.

[mcarroll]

Adrian: you'd suggest we just straight up get HP to replace the hardware then?

[Piyaboot]

Dear All

I Update the BL460c g1 iLO2 = v 1.78
and OA = v2.52

When I click the RACK FIRMWARE in OA Page
It show me message that

"A newer version of Power Management Controller firmware is avaliable. the Power Management Controller firmware should be updated"

Can you guide me for this ???

[Vinicio Hidalgo]

Hi guys,

You may want to try this:


1. Boot the server from Firmware Maintenance CD 8.60.
2. Agree to the EULA and select the language to use.
3. At that point, type CTRL+ALT+d+b+x (pressing and holding CTRL and ALT while typing d , then b , then x ). This will open a debug window on the Firmware CD.
4. Type:
cd /mnt/cdrom/compaq/swpackages
5. Run sh CP011299.scexe .


Please let us know the results!!

[Roman Stiefel]

Hi all
The last post of Vinicio Hidalgo worked for us. Thanks a lot!
Br
Roman

[BNewby]

what are the command-line flags available? When I do a --help on the scexe I receive oly single letters. Patrick mentioned --disaster-recovery from 8.60 FW Maintenance CD, but I cannot get that option to take. I get ./cpqsetup: unrecognized option '--disaster-recovery'

Thanks!

[BNewby]

Is anyone else seeing 504 Watts or 509 Watts being allocated to blades when this error is shown? I've had 5 460c g1 blades allocate two times the power than they should be. I can't get the wattage to go down. I need a better solution than replacing system boards. My case has come up with that solution, but there has to be a better way.

[Michael Garner_1]

When you get excessive power readings, these are normally caused by power values stuck in NVRAM. There are ways to try to clear this that don't require replacing the system board. In the upcoming Firmware Best Practices Implementer's Guide (due out mid-November), the following provides workarounds for this situation.

â ¢ In some instances, after an iLO or PowerPIC firmware update, a server will be shown with a Red â Xâ in the OA web interface and the server will not be allowed to power on. There are several possible causes of this condition. The following items list three possible ways to recover from this scenario once it has been confirmed that all iLO, System ROM, OA and Power Management Controller firmware has been updated.

1. When a blade is first inserted into an enclosure, the Power Management Controller calculates the amount of power need to power on the server. This value is then requested from the OA. With the 0.7 version of Power Management Controller firmware, this value would be incorrectly calculated and cause the request to the OA to be maxed out, usually around 500 watts instead of the normal range of 100-200 watts for a given blade server. When too many servers would request this maxed out value, the OA would become over allocated and would prevent some blades from powering on. Even after the power management controller would be updated, there have been some instances where the incorrect values in NVRAM for the blade server would persist.

â ¢ The first method to correct this problem is to remove the blade and re-insert it. In the event this does not work, continue to the next step to reset the NVRAM.

â ¢ Clear the NVRAM

o Power off blade
o Remove blade from enclosure
o Open blade cover and find â Configuration Resetâ switch and set it to â Onâ
o Re-insert blade into enclosure
o Allow blade to power up and wait until it prompts you to reset â Configuration Resetâ switch to off
o Power off blade again
o Remove blade from enclosure
o Open blade cover and find â Configuration Resetâ switch and set it back to â Offâ
o Re-insert blade into enclosure
o Allow blade to power up and go through initial configuration. At this point, blade can now be flashed to latest firmware version (and the fans will return to normal state along with power allocation to that blade)

Please note that resetting the configuration will cause any RBSU settings to be reset to defaults. After following this set of instructions, it may be necessary to reset the RBSU settings after updating the PowerPIC firmware to the latest version. If clearing the NVRAM does not clear the problem, it may be necessary to reset the iLO.

â ¢ Reset the iLO

o Power off blade
o Remove blade from enclosure
o Open blade cover and put a jumper on the two pins (J102) close to the system maintenance switch.
o Re-insert blade into enclosure
o Allow blade to power up and If system powers on and iLO2 is dead, power system back off.
o Remove blade from enclosure
o Open blade cover and flip the System Maintenance switch One to â Onâ
o Re-insert blade into enclosure
o Allow blade to power up and start FDT to flash the iLO2 firmware
o Power off blade after firmware updates are completed
o Remove blade from enclosure
o Open blade cover and flip the System Maintenance switch One to â Offâ
o Remove jumper from J102
o Re-insert blade into enclosure
o Allow blade to power up and go through normal boot process.

Please note that resetting the iLO will cause all iLO settings to be set back to defaults. After following this set of instructions, it will be necessary to reset their iLO configuration settings after updating the iLO firmware to the latest version if the default settings for the iLO have been changed by the user.


Regards,
Michael Garner
HP HPSUM/PSP/Firmware CD Architect

[Vinicio Hidalgo]

Guys, please remember to update the PMC on the Blades...

[BNewby]

1. Will updating the PMC firmware before upgrading the OA to 2.60 help us avoid this situation?

2. I fixed one of our bl460c g1's with the configuration reset DIP switch. I have another that that hasn't worked. So, moving on to the next steps to try, I am trying to locate J102 so I can add a jumper. Looking at the system board I have J7 and J8 jumpers (next to the SW2 system maintenance DIP switchs). There is an area labeled J2, however there are no rising metal to insert a jumper. Can you clarify where J102 is referring to on a bl460c g1?

Our upgrades are on hold to OA 2.60 from 2.25 because of this problem. I am expecting 20-30 blades to have this problem if I were to push OA 2.60 to everything right now.

Thanks Micheal and everyone else!

[Vinicio Hidalgo]

Hi Guys,

I think yes, updates should be applied in this order:
(talking about critical updates)

1)BIOS
2)iLO
3)PMC
4)OA

Remember, that updating the OA into a 2.60, with iLOs at 1.77 or older, could make the OA to forget the iLO IPs, so then you have to reconfigure everything...

Hope this will work

[BNewby]

Vinicio, I understand the need to update the PMC in the order described and that it is best practice. Can I be assured that updating the PMC firmware before OA will rid us of blades that are oversubscribing on power as described above? Or are the system boards just prone to the problem?

I want to be very certain on this before I align the resources to update 3,500 blades with 3 different operating systems.

[BNewby]

case# 4606643205 I have found answers to some of my questions. Updating the PMC firmware does not help in the case of our blades sucking twice the power. I have 34 blades showing "The Power Management Controller requires disaster recovery." when attempting the PMC update prior to the OA upgrades. It looks like the PMC firmware is locked up on 34 blades prior to 2.60. The 2.60 only unmasks the issue, allocating twice the power.

I need more ideas from the community. I want to avoid changing 34 blades Motherboards or reprogramming them with the re-programmer! Also, booting to the firmware management CD doesn't always work I have tested that. We now have over 4,000 c-class blades, so 35 is actually a large percentage in my opinion.

The nodes in question are returning to this script:










Thank you in advance

[Blade Junkie]

I have this same issue.. however my oa is upgraded and not the blade PMC. I cant get the blade to power on at all even after switch 6 is reset