Active / Active paths OS 6.1.1.100

[J-Philippe]

Hi, can someone from Nimble explain this please ?

"Starting with array OS 6.1.0.0, Fibre Channel arrays use active/active paths. If you upgrade an FC array to 6.1.0.0, it will automatically start using active/active paths."

From : HPE Alletra 6000, Alletra 5000, HPE Nimble Storage Array OS 6.1.1.100 Release Notes

Nimble Controlers are Active StandBy nomally or does it mean that the array will work in an Active / Active fashion like a 3PAR ???

Thanks a lot in advance for who will provide an answer. That's just a major feature announced between the pear and the cheese.

Jean-Philippe

[PatrickLong · Accepted solution]

Nimble admin here.  It means that running any Nimble OS version PRIOR to 6.1.1.100 on a Nimble FC-based array, you will have an Active/Passive array where only one controller has active data paths - the other controller is a standy strictly for failover purposes.  Install Nimble OS 6.1.1.100 and your Nimble array is now magically an Active/Active array, with every path on both controllers able to serve data simultaneously.  I agree that this is a MAJOR feature addition/improvement that got little to no fanfare but it is a big deal!

[J-Philippe]

Crazy, hope update will work from 5.3.x because Infosight seems to report issue when upgrading directly from 5.3x to 6.1.x

[PatrickLong]

It is warning you only because you cannot go directly from 5.3 to 6.1.1.100 - That upgrade path is blocked per the notices shown - do not attempt it.  You will need to perform a multi-step upgrade to get from your version 5.3.1.100 to the newest 6.1.1.100  version.

I believe the steps needed would be: 

5.3.1.100 (released Jan 2021) >>> 6.0.0.500 (released Sept 2022)

then

6.0.0.500 (released Sept 2022) >>> 6.1.1.100 (released Dec 2022)

In my experience Nimble OS upgrades have been fairly painless, so long as you have proper multi-pathing configured and functional on all hosts connected to the array.

[feigenL]

Hello @PatrickLong, quick question related to this new Active/Active mode in Nimble controllers...related to VMware ESXi hosts, which MPIO policy should I configure? Currently, the nodes are on ESXi 8 and configured as: NIMBLE_PSP_DIRECTED.

After the Nimble goes up to 6.1.1. Should that policy be changed to Round Robin or leave it as is?

Thanks in advance.

[J-Philippe]

Hi Patrick, 

I'm thinking about the balance between controllers.

Will the Nimble balance ramdomly the LUN between both controllers now they will be both active or a new menu will appear in the Nimble to allow IT staff to assign LUN 1 to CTRL A and LUN 2 to CTRL B and so on ?

Thank you.

Jean-Philippe.

[PatrickLong]

@feigenLI have not seen guidance from HPE or Nimble to indicate that any changes are required to the NIMBLE_PSP_DIRECTED PSP - which is created by the Nimble Connection Manager and displayed as "HPE-Storage-Connection-Service" and  "HPE-Storage-psp " in ESXi.  From my datastore on Nimble, the current PSP shows:

Path Selection Policy: NIMBLE_PSP_DIRECTED
Path Selection Policy Device Config: {policy=iops iops=1 bytes=0 useANO=0 isVvolPE=0 GROUP_ID=0 numIOsForIssueInq=10000 numIOsSinceLastInq=5601 NUM_OF_MEM_ARRAY=1 SYNC_REPLICATED=0 lastPathIndex=2 NumIOsPending=2 numBytesPending=131072 }

If I am understanding it correctly, the default setting for a VMW_SATP_ALUA-claimed device would show policy=rr with iops=1000  unless you've changed the properties of the rule to another value like iops=1.  The fact that the Nimble disk devices show policy=iops iops=1 indicates that this property is set at on a per-device basis (by the NCM components listed above) but it is effectively equal to policy=rr iops=1.  This subtle change from rr to iops indicates that this device config has been altered from the default rule for VMW_SATP_ALUA

[PatrickLong]

@J-Philippeactually this is a very interesting question to which I do not know the answer.  I don't know what type of algorithm Nimble uses to determine volume "ownership" by one controller or the other.  A previous vendor I have experience with (XIO, no longer in operation) allocated volume ownership in a very non-intuitive way - IIRC the odd-numbered controller CT1 owned all the even-numbered volumes and the even-numbered controller owned all the odd-numbered volumes.  But I confess I have NO idea how Nimble assigns volume ownership, since I don't know if this new firmware 6.1.1.100 reconfigures the array as an Asymmetrical Active-Active array or a Symmetric Active-Active array.  Perhaps someone from Nimble can chime in and clarify?

[SprinkleJames]

@J-Philippe, @PatrickLong:

The controllers are still active/standby, so all volumes are still owned by the active controller. What has changed is that the paths are now active/active, meaning the paths connected to the standby controller are able to service I/O via internal passthrough from the standby controller.

[PatrickLong]

So that makes Nimble FC-based arrays on NimbleOS version 6.1.1.100 Asymetrical Active Active arrays, since both controllers can receive I/O commands (the active-active part), but only the owner-controller of a LUN can pass those I/O commands to that LUN.  Any I/O's sent by a host to any port on the non-owner controller is first transferred to the owner controller internally, which is what you have clarified so thank you for that.  I think the open question is:  what is the mapping of array luns to the "owning" controller for Nimble? Is that knowable or part of the Nimble secret sauce?

[J-Philippe]

@PatrickLong  @SprinkleJames 

Hi thanks for your answers. Not so sure about ALUA because if one controller remain StandBy it means that we are still in an Active / Passive array when for ALUA both controller should be Active if I'm right.

I understand that in this situation the interconect path is used to forward I/O from the StandBy controller to the Active one.

Otherwise we speak about if a controller is referred to as active or passive relative to the LUNs they own but it mean that both controller are active A + B when A is owning LUN 1 & 3 and B is owning LUN 2 & 4

 CTRL A Active (owner) Lun 1 Lun 3 / Passive of Lun 2 Lun 4

CTRL B Active (owner) Lun 2 Lun 4 /  Passive of Lun 1 Lun 3

Thanks to NImble to help us to clarify the Array status in 6.1..100

[J-Philippe]

From the : GUI_Administration_Guide_6_1_x.pdf

Page 30 :

Updated Fibre Channel Arrays Use Active/Active Paths
Starting with array OS 6.1.0.0, HPE Storage arrays running the Fibre Channel protocol use active/active paths. If you upgrade an array to version 6.1.0.0, that array will automatically start using active/active paths for FC clusters.
Note: There is no change to arrays using the iSCSI protocol. iSCSI arrays continue to use active/standby. If your array
uses mixed protocols, the active/active paths only apply to the FC protocol.
You can confirm that your array is using active/active by checking the FC ALUA state in the GUI. From the GUI, select either Administration > Network > FC Protocol or Monitor > Interfaces to see information about the FC paths.

Page 193 :

Fail Over a Controller
A failover switches management of the array from the active controller to the standby controller.
Note: In this case, the terms active controller and standby controller refer to the physical controllers on the array,
while FC Active/Active refers to the behavior of the controllers.

Should we understand that a full FC array is still working in an Active / Passive way regarding the controllers but I/O may use any path available as the Passive controller behavior allow I/O to travel through the interconnect path to the Active controller ?

In which category should we place the aray ?

A\ The Active/Active or Symmetric Active/Active array will have all the ports on its storage controllers to allow simultaneous processing of the I/O offering the same levels of performance and access to all LUNs. In a symmetric active/active array, there is no concept of a standby controller.

B\ Active/passive arrays will have active and standby storage controllers. A controller is referred to as active or passive relative to the LUNs they own. In an active/passive array an I/O to a set of LUNs can only be processed by a storage controller that owns them. The controller that owns a set of LUNs is referred to as an active controller for those LUNs

C\ The Asymmetric Logical Unit Access (ALUA) array is a type of active/active array, which has the ability to concurrently use all its controllers to process I/O and also stand-in as failover partners of each other. Much like in an active/passive array, a LUN or a set of LUNs can only be owned by a single controller. The owning controller will have symmetric (direct) access to the LUN, hence offering the best performance. The non-owning controller can process the I/O indirectly (asymmetric) for the LUN by transferring the I/O via the interconnect to the owning controller. Hence, there is a performance hit owing to time required to transfer the I/O to the owning controller.

[SprinkleJames]

@J-Philippe,

You are correct, and it's closest to your category C. The paths are ALUA: the paths to the active controller are active/optimized, and the paths to the standby controller are active/non-optimized.

[PatrickLong]

All of this is correct, it is an Asymmetric Active/Active (AAA) array.  But the question remains - how can I as the array admin determine which controller owns each LUN?

[SprinkleJames]

@PatrickLong,

The physical controllers are still active/standby; they don't both simultanesouly own a set of LUNs. All LUNs are always owned by the active controller. You as the array admin can change which contoller owns the LUNs by initating a controller failover, which will cause the standby controller to be come active, and therefore take over ownership of all the LUNs. When this happens, the paths that were active/non-optimized will switch to active/optimized, and vice versa.