error after replacing controller on p2000 g3 fc

[MediaLive]

hello

after replacing one of the controller i found that the replacement part had a very old firmware

(100xx something) so I had to align it to my actual TS252P005

With ftp I tried to upload directly TS252P005 but was told that I had first to update to TS201 so I did

after that I was able to upload again the TS252P005 

But now every 5 minutes or so, I receive this error from the P2000

Enclosure event: fault. (enclosure: 1, WWN: 500c0ff014d20b3c) I/O module A, I/O module status: The firmware version is not compatible, HP SPS-CHASSIS 2028

EVENT ID:#A1293

EVENT CODE:168

EVENT SEVERITY:Error

EVENT TIME:2021-10-28 08:34:54

This is very strange because from the GUI i can see the firmwares are identical

https://www.dropbox.com/s/94uu9146kylfsrg/Screenshot%202021-10-28%20at%2008.46.53.png

https://www.dropbox.com/s/v2plzw372hfefou/Screenshot%202021-10-28%20at%2008.47.12.png

[tapelibraryfixr]

The expander firmware hasnt loaded on, maybe a fault with the controller.

You could try reloading the firmware

[SUBHAJIT KHANBARMAN_1]

@MediaLive 

As per the picture it's clearly visible that Controller A Expander Module firmware 2028 missing. Also we can see Controller A SAS output port shows "?". Controller B also shows Health state unknown.

Ideally you should have followed below upgrade notes,

UPGRADE NOTE:
If upgrading from an older version of firmware it may require a multiple step process to get to TS252.
If below TS230 then upgrade to TS230P008 first, then TS240P004, and then TS250P003 before upgrading to TS252P005. If the firmware is between the mentioned versions, it is recommended to follow through the steps as well.

Make sure all VDISKs are in FTOL state before performing anything further.

It's better to shutdown MSA and removes all data drives half of the bay just to avoid any impact on data. Power up One controller at a time and check if all shows ok or not. If not then try to reload the firmware and see. Repeat same for other controller as well.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

**********************************************************************

[MediaLive]

hello

thanks gents for the hints, will try to reload firmware following suggested steps

btw, is ok to turn on the controllers with disk not connected ?

the configuration of vdisk / volumes will be back when i will restart the p2000 with disk inserted ?

Thank you

EDIT:

I found that looking at "update firmware" page from both controllers

give me different results 

this is from conrtoller A

https://www.dropbox.com/s/ddxb5jql94siv5q/update%20controller%20A.png

this is from controller B

https://www.dropbox.com/s/p9ycieekj5fydwe/update%20controller%20B.png

maybe just a restart of controller A is needed ?

[SUBHAJIT KHANBARMAN_1 · Accepted solution]

@MediaLive 

Yes it's absolutely fine if you disconnect drives because Vdisk and Volume information stored in drive metadata section. So till your drives are safe your data along with data structure is also fine.

Make sure you shouldn't have any production IO running which means controller cash needs to be empty. To verify that you run below command,

# show unwritable-cache

Both controller should show 0 value. If it shows any othe rvalue then unwritable data exists in Controller cache which is not yet written to backend drives.

So you need to shutdown controllers properly so that all data gets written to drives.

You can follow the instructions as I have mentioned in my last post. It looks like firmware update not done properly. So better to do controller firmware update 1 at a time without data drives for safety.

Hope this helps!
Regards
Subhajit

I am an HPE employee

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************

[MediaLive]

amazing help !

thanks a lot