HPE EVA Storage

EVA3000 reboot fails with TC: 041F0A1F

 
Jochen Neusius
Occasional Contributor

EVA3000 reboot fails with TC: 041F0A1F

Hello,
We have an EVA3000 2C2D with Firmware 3.011 and CV6. System had been extended by 5 additional 146GB Disks a few days ago.
Today both controllers terminated unexpectedly. When rebooting after "Scanning for disks" we got the frontpanel messages:
TC: TBD
and then
TC: 041F0A1F
PTO: 30 RC0
The PTO value counted from 0 to 30.
Then the controller went into a reboot loop. After the third loop, booting stops and we had to restart manually.
Both controllers shows the same behaviour.
We diconnected all powercables, restarted the Switches (HP Brocade 2/8), restarted the disk enclosures but nothing will take the controllers online again.

Any idea? Help is very welcome!
Thanks in advance
Jochen
6 REPLIES 6
Uwe Zessin
Honored Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

The error could indicate a software bug (access to page zero) or a non-correctable memory error, but the fact that both controllers show this behaviour makes it more likely it is the first cause.
Maybe something screwed up the EVA's meta-data and this triggers the problem.

To rule out #2, you could disconnect the disk drive enclosures and power-cycle the controllers. They should both run the diagnostics and then wait for entry of a WWN - you can't make the situation worse, because no configuration information is stored in the controllers anyway.
.
Jochen Neusius
Occasional Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

Hello Uwe,
thank you for your answer!
I guess you are right with reason #1. I tested to start a controller with disconnected FC device ports and the controller went into an error first (No FC Loop, displayed for 10 seconds) and then asked for the WWN.

Should it be necessary to reinstall the Firmware and is there any way to do this without the controllers online?
Or is the only option to initialize the system completely and do a firmware update afterwards?

Uwe Zessin
Honored Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

While I have not tried it without any diskk drives attached, I think you should be able to do a firmware flash without an initialized system. Just enter a WWN and the controllers should login on the FC switches and then be accessible by CV-EVA.
.
Víctor Cespón
Honored Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

Try to boot controllers without those 5 new drives. You can also get the Command View logs from the disk and get someone to check them, to see if any of then has been causing errors on the buses.

That termination code is usually the result of some disk blocking both loops. Try to boot with less disks and see if the controllers finish booting or stop before that with a termination code.

Try to reseat the EMUs on both enclosures too.

Ah, and firmware 3.011 is WAAAAAY out of support, those 5 new disks are compatible with firmware 3.011? Are you sure? We already had two customers ending having to un-unitialize the EVA after adding unsupported drives.
Jochen Neusius
Occasional Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

@Uwe:
I started EVA without disk enclosures, entered the WWN and booting went on. CV access was possible but there was an uninitialized storage system only. No change to configure anything or to do a firmware update.

@vcespon:
will try your tips, thanks.
The VCS version 3.011 is a typo, sorry. It's 3.110, the latest available 3.x version. I guess there's only one 4.x version afterwards for EVA3000 which enables active/active mode then.

Will let you know my results ...
Jochen Neusius
Occasional Contributor

Re: EVA3000 reboot fails with TC: 041F0A1F

Hello,
the only way to take our EVA3000 alive again was to uninitialize the system and reconfigure it.
We installed two other controllers, tried to boot without the ne disks, but the error was still there.
The problem must have been located on the disks.
After initializing we could rebuild the system with a captured configuration copy, had to reinstall some ESX-VMs and did a lot of restores with DP :-(
EVA is running well since then and hopefully - it will do so further on.
Thanks for your help!