HPE MSA 2060 FC connectivity problem

[Mircea Giurgeu_1]

We have two HPE ProLiant DL380 G10 2U server connected directly through a pair of HPE StoreFabric SN1200E 16 Gb Single Port FC HBAs to a MSA 2060FC (dual controller) in redundand fashion 

The servers are running Windows Server 2022 Std with Failover Clustering configured properly. Upon rebooting Srv1 none of the servers are able to access any of the cluster enabled volumes on the SAN.

The MSA is reporting a failed initiator on Srv1 so the redundancy on that server has been lost. I do not have hands on to troubleshoot this issue, but the cluster should continue to operate normally. I checked the MPIO on both servers and it shows the MSA properly. 

I have upgraded the firmware on both controllers to the latest (IN210R004) but the situation persists

In the GUI, the same info is showing under Attached Hosts for each volume in the system

Here is what is shown for the volumes attached to Srv1:

I understand that the Redundacy has failed but why are the volumes not accessible in Failover Clustering on the hosts?

Here is the output from a few relevant CLI commands:

# show maps all

Volume View [Serial Number (**Confidential info erased**) Name (Vol1) ] Mapping:
Ports LUN Access Identifier Nickname Profile
-----------------------------------------------------------------------------
1-4 5 read-write 00c0ff533b5e00003f69ea6301010000 Srv1.* Standard
1-4 5 read-write 00c0ff533b5e00003a69ea6301010000 Srv2.* Standard
not-mapped all other initiators Standard

Volume View [Serial Number (**Confidential info erased**) Name (Vol4) ] Mapping:
Ports LUN Access Identifier Nickname Profile
-----------------------------------------------------------------------------
1-4 6 read-write 00c0ff533b5e00003f69ea6301010000 Srv1.* Standard
1-4 6 read-write 00c0ff533b5e00003a69ea6301010000 Srv2.* Standard
not-mapped all other initiators Standard

Volume View [Serial Number (**Confidential info erased**) Name (Vol2) ] Mapping:
Ports LUN Access Identifier Nickname Profile
-----------------------------------------------------------------------------
1-4 7 read-write 00c0ff533b5e00003f69ea6301010000 Srv1.* Standard
1-4 7 read-write 00c0ff533b5e00003a69ea6301010000 Srv2.* Standard
not-mapped all other initiators Standard

Volume View [Serial Number (**Confidential info erased**) Name (Vol3) ] Mapping:
Ports LUN Access Identifier Nickname Profile
-----------------------------------------------------------------------------
1-4 4 read-write 00c0ff533b5e00003f69ea6301010000 Srv1.* Standard
1-4 4 read-write 00c0ff533b5e00003a69ea6301010000 Srv2.* Standard
not-mapped all other initiators Standard

# show initiators
Nickname Discovered Mapped Profile Host Type ID
-----------------------------------------------------------------------
initiator0007 Yes Yes Standard FC 10009440c9d3d8c4
initiator0008 Yes Yes Standard FC 10009440c9d3d802
initiator0009 Yes Yes Standard FC 10009440c9d3d818
initiator0010 No Yes Standard FC 10009440c9d3d838
-----------------------------------------------------------------------
Success: Command completed successfully. (2024-01-14 10:10:31)

# show volumes details
Pool Name Total Size Alloc Size Usage Type Serial Number WR Policy Cache Opt Read Ahead
Type Tier Affinity Snap Retention Priority Desc WWN Large Virtual Extents
Role Metadata In Use Health Reason Action
--------------------------------------------------------------------------------------------------------------------
A Vol1 5999.9GB 268.4MB Unknown 00c0ff533b5e00004469ea6301000000 write-back standard Adaptive
base No Affinity medium 600C0FF000533B5E4469EA6301000000 Disabled
1114.1KB OK
A Vol2 10.9GB 218.1MB Unknown 00c0ff533b5e0000243ab56401000000 write-back standard Adaptive
base No Affinity medium 600C0FF000533B5E243AB56401000000 Disabled
65.5KB OK
A Vol3 499.9GB 243.4GB Unknown 00c0ff533b5e0000cb4b1c6401000000 write-back standard Adaptive
base No Affinity medium 600C0FF000533B5ECB4B1C6401000000 Disabled
1015.8KB OK
A Vol4 4999.6MB 272.6MB Unknown 00c0ff533b5e00001152b56401000000 write-back standard Adaptive
base No Affinity medium 600C0FF000533B5E1152B56401000000 Disabled
65.5KB OK
--------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully. (2024-01-14 10:07:55)

ANY help would be highly appreciated.

[support_s]

System recommended content:

1. HPE MSA 1060/2060/2062 CLI Reference Guide

2. HPE MSA 1060/2060/2062 Installation Guide

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[ArunKKR]

Hi,

The server with wwn 10009440c9d3d838 is currently not discovered by the MSA. Please check whether the link is down from server ilo menu as well as from MSA SMU.

From MSA firmware version IN110P002 release  notes:

The following issues have been resolved in the IN110P002 release:

• Disabling or de-activating active paths may result in volume access issues.

The above mentioned issue is specific to firmware version IN110P001. Please confirm whether the original firmware version when issue occurred was IN110P001.   I would suggest to get an HPE support case logged to get the MSA as well as server logs reviewed.

[Mircea Giurgeu_1]

Both servers have 2 FC single port HBAs installed. These are connected to separate controllers on MSA. Yes, there is a problem with one of the connections, but the server should be discovered on the other initiator. The GUI shows that the redundancy has failed, but why is not discovered on the other link? I plan to go on site in the next few days and troublehoot, but I'm puzzled why it doesn't work with only one link active. This has started happening before upgrading from IN110P002 and it continues with the new firmware, as well.

What is the best way to reach HP support to have the case logged?

[Mircea Giurgeu_1]

This is what the GUI shows about the server with a failed connection. Based on this, the server should still be discoverable, should it not?

[ArunKKR]

Hi, IN110P002 firmware does not have the issue that I have mentioned. We can rule out firmware being the cause. Even with one initiator link up the volume should have been detected on server 1.

Please confirm whether the FC cable being used is multimode and not single mode. I have seen strange behavior when single mode cable is being used in direct connection.

It would require analysis from MSA debug logs and server HPS report to troubleshoot the issue further. You could log an HPE support ticket online from below web link:  https://support.hpe.com/connect/s/?language=en_US.

[Mircea Giurgeu_1]

the FC cables used are HPE p/n QK733A: HPE Premier Flex LC/LC Multi-mode OM4 2 Fiber 2m Cable

I'm opening a case with HPE Support (thanks for the link)