Strange error: Poll failed at mgmt_cmd.c.1062, errno 4 c7000 OA

[sciunix]

We got the error listed in the subject line early this morning (firmware 2.20) from our Onboard Administrator on a c7000. Any hints?

Log detail:
Oct 23 00:42:10 OA: Poll failed at mgmt_cmd.c.1062, errno 4 (Interrupted system call), fds = 0x10150854, last_cmd=102
Oct 23 00:42:47 OA: Management process failure. Rebooting the Onboard Administrator
Oct 23 00:43:21 Kernel: Network link is up at 100Mbps - Full Duplex
Oct 23 00:43:21 OA: Time zone changed to CDT+5
Oct 23 00:43:23 OA: LCD Status is: OK.
Oct 23 00:43:24 Enclosure-Link: Service started
... etc...

[Gary Sears_1 · Accepted solution]

Management process failure. Rebooting the Onboard Administrator
This error was addressed in OA firmware v2.25.

[Chris House]

I have 8 c7000 enclosures with OA 2.25 firmware and all but 1 experienced this error (Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0) and then rebooted, all within about half an hour of each other late in the evening on 12/26. Very odd, never happened before.

3 enclosures in one datacenter are cabled together and they all rebooted, and then 6 enclosures in another datacenter are cabled together and all but one of them rebooted.

The one that didn't reboot is at the end of the link chain (but the enclosure on the other end of the link chain did reboot). All enclosures have standby OAs and they all along with the active OAs rebooted at the same time (including the standby OA for the enclosure whose active OA did not reboot)

[AStackpole_USCOURTS]

I had this exact same error last night on a c7000 chassis with OA running firmware version 2.25, hardware version B1. I have 6 other c7000 chassis and have only gotten this error once on this one.

Looking for an answer or at least an idea...

[Charlie88]

I had this happen to 2 c7000 chassis in separate datacenters overnight - within 20 mins of each other. Both are OA 2.25, hardware version A1.

Jan 21 04:06:37 OA: Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0, last_cmd=00
Jan 21 04:07:19 OA: Management process failure. Rebooting the Onboard Administrator

Also saw this on some other chassis over Christmas - OA 2.25, hardware version A0. Never encountered this issue prior to OA 2.25.

Seems to be a bug OA 2.25? Can't see any relevant fixes in the OA 2.32 release notes

[Migrationsverket]

Very interesting guys! I had the exact same crash
Jan 21 01:23:22 OA: Poll failed at mgmt_cmd.c.1078, errno 4 (Interrupted system call), fds = 0x101570d0, last_cmd=109
Jan 21 01:23:58 OA: Management process failure. Rebooting the Onboard Administrator

We have a quite good service contract with HP and I have submitted a case to them regarding this. They have not responded yet but it look like they have issued a replacement OA but I sure would like to talk to them before replacing hardware to hear their reasons for that.

[B J Singh]

sciunix;

I also saw the same thing on one of my eclosures with OA @ 2.20...the message was exactly the same as your's.

[RobG_1]

Wow, I got the same error as everyone. This only happened on one server back on the 1/5/09, but I find it odd that all these are occurring around the same timeframe. I can't figure out a diff between the one that failed and the 3 others that didn't. Any ideas from HP on this?

Jan 5 06:42:45 OA: Poll failed at mgmt_cmd.c.1078, errno 0 (Success), fds = 0x101570d0, last_cmd=00
Jan 5 06:43:16 OA: Management process failure. Rebooting the Onboard Administrator
Jan 5 06:43:36 OA: Onboard Administrator entering Standby mode.
Jan 5 06:43:47 OA: Time zone changed to PST+8
Jan 5 06:43:50 OA: Redundant Onboard Administrator detected.
Jan 5 06:43:50 OA: Onboard Administrator booted successfully
Jan 5 06:43:51 Redundancy: Service started (STANDBY)
Jan 5 06:44:03 Kernel: Network link is up at 100Mbps - Full Duplex
Jan 5 14:15:25 OA: cladmin logged into the Onboard Administrator

[Charlie88]

I've got a call in with HP and have got some info back.
Appears there's a bug in 2.25 (they didn't mention other versions) where it reboots after 100 days. Check when you updated to your current OA firmware and when it rebooted - I have 5 chassis, all rebooted 100 days after I upgraded to 2.25.
Recommendation I've been given is upgrade to 2.32.

[AStackpole_USCOURTS]

I have updated two out of seven chassis to OA firmware 2.32. No errors yet but I only had it happen once to one OA module in almost 2 years. We'll see how this goes...