Strange error: Poll failed at mgmt_cmd.c.1062, errno 4 c7000 OA

[sciunix]

We got the error listed in the subject line early this morning (firmware 2.20) from our Onboard Administrator on a c7000. Any hints?

Log detail:
Oct 23 00:42:10 OA: Poll failed at mgmt_cmd.c.1062, errno 4 (Interrupted system call), fds = 0x10150854, last_cmd=102
Oct 23 00:42:47 OA: Management process failure. Rebooting the Onboard Administrator
Oct 23 00:43:21 Kernel: Network link is up at 100Mbps - Full Duplex
Oct 23 00:43:21 OA: Time zone changed to CDT+5
Oct 23 00:43:23 OA: LCD Status is: OK.
Oct 23 00:43:24 Enclosure-Link: Service started
... etc...

[Gary Sears_1 · Accepted solution]

Management process failure. Rebooting the Onboard Administrator
This error was addressed in OA firmware v2.25.

[Chris House]

I have 8 c7000 enclosures with OA 2.25 firmware and all but 1 experienced this error (Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0) and then rebooted, all within about half an hour of each other late in the evening on 12/26. Very odd, never happened before.

3 enclosures in one datacenter are cabled together and they all rebooted, and then 6 enclosures in another datacenter are cabled together and all but one of them rebooted.

The one that didn't reboot is at the end of the link chain (but the enclosure on the other end of the link chain did reboot). All enclosures have standby OAs and they all along with the active OAs rebooted at the same time (including the standby OA for the enclosure whose active OA did not reboot)

[AStackpole_USCOURTS]

I had this exact same error last night on a c7000 chassis with OA running firmware version 2.25, hardware version B1. I have 6 other c7000 chassis and have only gotten this error once on this one.

Looking for an answer or at least an idea...

[Charlie88]

I had this happen to 2 c7000 chassis in separate datacenters overnight - within 20 mins of each other. Both are OA 2.25, hardware version A1.

Jan 21 04:06:37 OA: Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0, last_cmd=00
Jan 21 04:07:19 OA: Management process failure. Rebooting the Onboard Administrator

Also saw this on some other chassis over Christmas - OA 2.25, hardware version A0. Never encountered this issue prior to OA 2.25.

Seems to be a bug OA 2.25? Can't see any relevant fixes in the OA 2.32 release notes

[Migrationsverket]

Very interesting guys! I had the exact same crash
Jan 21 01:23:22 OA: Poll failed at mgmt_cmd.c.1078, errno 4 (Interrupted system call), fds = 0x101570d0, last_cmd=109
Jan 21 01:23:58 OA: Management process failure. Rebooting the Onboard Administrator

We have a quite good service contract with HP and I have submitted a case to them regarding this. They have not responded yet but it look like they have issued a replacement OA but I sure would like to talk to them before replacing hardware to hear their reasons for that.

[B J Singh]

sciunix;

I also saw the same thing on one of my eclosures with OA @ 2.20...the message was exactly the same as your's.

[RobG_1]

Wow, I got the same error as everyone. This only happened on one server back on the 1/5/09, but I find it odd that all these are occurring around the same timeframe. I can't figure out a diff between the one that failed and the 3 others that didn't. Any ideas from HP on this?

Jan 5 06:42:45 OA: Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0, last_cmd=00
Jan 5 06:43:16 OA: Management process failure. Rebooting the Onboard Administrator
Jan 5 06:43:36 OA: Onboard Administrator entering Standby mode.
Jan 5 06:43:47 OA: Time zone changed to PST+8
Jan 5 06:43:50 OA: Redundant Onboard Administrator detected.
Jan 5 06:43:50 OA: Onboard Administrator booted successfully
Jan 5 06:43:51 Redundancy: Service started (STANDBY)
Jan 5 06:44:03 Kernel: Network link is up at 100Mbps - Full Duplex
Jan 5 14:15:25 OA: cladmin logged into the Onboard Administrator

[Charlie88]

I've got a call in with HP and have got some info back.
Appears there's a bug in 2.25 (they didn't mention other versions) where it reboots after 100 days. Check when you updated to your current OA firmware and when it rebooted - I have 5 chassis, all rebooted 100 days after I upgraded to 2.25.
Recommendation I've been given is upgrade to 2.32.

[AStackpole_USCOURTS]

I have updated two out of seven chassis to OA firmware 2.32. No errors yet but I only had it happen once to one OA module in almost 2 years. We'll see how this goes...

[RobG_1]

Seems Charlie88 was right, traced it back and we got this error exactly 100 days from the 2.25 update. We are in process of upgrading to 2.32 as well as we are hoping this doesn't happen EVERY 100 days.

[shocko]

Hi,
I had the exact same problem with OA fw v2.25. I checked the logs and the same error(s) appearred 100days ago.

I'm planning an upgrade to 2.41 and will report back..... after 100 days ;)

[Mihai Bertalan]

Hello,
Same thing here, with 2.25 firmware, seems that OBA is rebooting after 100 days(no mather if you access it or not)

[CA-Blademan]

FYI everybody - This issue has not been fixed by HP. It has happened in our environment in v2.21, v2.25, v2.41, v2.32, v2.51, and v2.52 and continues to happen. I believe is a hardware issue. Make note that HP considers these OA's obsolete now. They are now pushing the Gigbyte version now.

[Paul_Carroll]

Can confirm the same thing. Version 2.25 rebooted yesterday and previously in November and July. Not sure if it's worth logging with HP after what has previously been said.

[The Brit]

Why not go crazy and upgrade to the latest.

**** 2.60 ****

Why wait until the problem hits before upgrading?


Dave.

[Paul_Carroll]

Actually I was incorrect and we have version 2.51. Correct it's pointless raising with HP as they will just say upgrade the firmware. Chances are it will not fix it, then they will find another item of firmware which is not up to date in the chassis, blades, interconnect modules etc and on it goes without ever giving a proper explanation or solution to the 'actual' issue.

Will get back when we have upgraded but it won't be soon as I have to go via our change control procedures etc etc.

thanks

Paul

[Superwin]

Weird... Had the same message last night as everyone else. I have only got one c7000 enclosure tho.
Will have to upgrade to 2.6....are HP reading this!!!!!!

[Mario Kleinbauer]

Hi !!

Se th the HP Advisory of this failure:
Document ID: c01677205


http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01677205〈=en&cc=us&taskId=101&prodSeriesId=3188465&prodTypeId=329290

greats
Mario