HPE 3PAR7400c 2node System recovery in process

[CRD]

Good day

Has someone perhaps had this issue before. Client had a power failure about a month ago so the 3PAR lost its power and went off abruptly.

When power to the DC was restored and the sytem started up the 3PAR is showing a "System power fail recovery in process" feedback on the CLI. Cant even reboot the 3PAR from the CLI. Also cant add 3PAR to IMC.

The client did leave the 3PAR powered on for long time to see if this process eventually completes but it doesn't and all VV are down.

Also when you run showtask command it just says "System power fail recovery in process", please try later

shownode

showcageshowvvshowpd -failed -degradedshowsysmgr -d

Any help or advise will be appreciated.

Thanks

[Moderator edit: Erased the confidential info.]

[veeyarvi]

Hello CRD,

It looks some issue in getting the data out from the power fail partition. The reason must be some hardware issues as well since I see the PDs are failed/degraded.

Also, do we originally have only two cages? Any cages/PDs missing?

The PDs need to be back to 'normal' first. Could you share the below outputs?

showpd -s

showpd -c

Regards,
Veeyaarvi

[CRD]

Hi Veeyaarvi

Thank you for the response.

Only 2 cages originally yes.

See outputs as requested. I do see 6 vacated disks, which means they failed I just dont know if 6 disks can really fail at once but i guess with a power failure anything is possible

[veeyarvi · Accepted solution]

Hi CRD,

They might have failed at the time of the power issue. We need to clear the errors on these PDs and bring them back first. You may try reseating the failed ones but I think we also need to clear the errors. Try the below steps.

showpd -i   >>> Note the wwn of the drives failed

controlpd clearerr <PD wwn>

And now reseat the PD and see the status.

Regards,
Veeyaarvi

[CRD]

Hi Veeyaarvi

Thank you so much for the advice. We followed your recommendation and the after that we were able to do a gracefull reboot of the 3PAR and it came back online.

All VV's are online and all disks are showing normal.

Really much appreciated.

[Sunitha_Mod]

Hello @CRD,

That's Excellent! 

We are extremely glad to know the issue has been resolved and we appreciate you for keeping us posted.