multipath

[George O'Reilly]

Hello All,

I have 4 luns mapped up two sides of the SAN and using multipath am able to have full redundancy. It works perfectly well except for the fact it seems to go up one side or the other. Ideally, I would like to set the two primaries going up each side.
I thought this would be easy by just changing the priority of the 'Path Group'. They all seem to be set to (prio=1).
However I found it very hard to find a command to do this.
Here is my multipath -ll output.

mpath2 (SDataCoreSANsymphony_RHfiscal_lun3) dm-4 DataCore,SANsymphony
[size=1.0T][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 0:0:0:3 sdc 8:32 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 1:0:1:3 sdg 8:96 [active][ready]
mpath1 (SDataCoreSANsymphony_RHfiscal_lun1) dm-2 DataCore,SANsymphony
[size=1.0T][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 0:0:0:1 sda 8:0 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 1:0:1:1 sde 8:64 [active][ready]
mpath4 (SDataCoreSANsymphony_RHfiscal_lun4) dm-5 DataCore,SANsymphony
[size=1.0T][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 0:0:0:4 sdd 8:48 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 1:0:1:4 sdh 8:112 [active][ready]
mpath3 (SDataCoreSANsymphony_RHfiscal_lun2) dm-3 DataCore,SANsymphony
[size=1.0T][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 0:0:0:2 sdb 8:16 [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 1:0:1:2 sdf 8:80 [active][ready]


Any help is much appreciated.

Regards
George

[Matti_Kurkela]

Some SAN storage systems allow the use of all FC interfaces simultaneously without restrictions. This is known as "active/active" or "symmetric" multipathing.

Other storage systems primarily use one interface, and the other interface is in standby mode. There is usually some delay or overhead associated with switching the interfaces at the SAN side (known as a "trespass event"). This is known as "active/passive" or "asymmetric" multipathing.

With asymmetric storage systems, it is important that all connected hosts detect and use the currently-active path: if the hosts attempt to use the paths in random or round-robin fashion, it will cause constant trespass events, which can seriously hurt your disk I/O performance.

There is a standard called ALUA (Asymmetric Logical Unit Access) which allows the storage system to identify itself as asymmetric and to announce the currently active path. Before the standard was developed, there were several manufacturer-specific techniques for the same purpose. The multipath subsystem in Linux supports ALUA and all the manufacturer-specific techniques that had free documentation available to dm-multipath developers.

Your system seems to have detected that it's been connected to an asymmetric storage system: this is exactly why it has created multiple path groups for each LUN. In that case, trying to force the multipath system to use both paths simultaneously is probably a bad idea.

If you're sure your storage system is capable of symmetric-mode operation, see if your storage manufacturer provides a configuration guide for Linux and verify that all documented conditions for symmetric-mode operation are met.

MK

[George O'Reilly]

Matti,

Thanks for your reply.

I can understand that I might have an active/passive arrangement. But as I have 4 luns cannot I not make one side of the SAN active for two luns and the other side active for the other two.
It would still be active/passive setup for each lun but I get to dictate which one is active where through the use of priorities. This is a test system so I am willing to trial first.
So don't worry about the risk element of this. Are you able to tell me how the set the priority within a Path Group?

Thanks
George

[Matti_Kurkela · Accepted solution]

> cannot I not make one side of the SAN active for two luns and the other side active for the other two.

The active/passive state is typically not per disk/LUN, but by storage controller/interface: in other words, *all* the paths of the passive controller are passive together. When one LUN switches to the other side, all the other LUNs must switch too.

One of the ground rules of dm-multipath is "if a LUN has multiple path groups, only one of them can be active at a time".

The priority of the path group is simply the sum of the priorities of all non-failed paths within it. The priority of an individual path is determined by either one of the dm-multipath's internal functions (selectable by the "prio" keyword) or with an external program (determined with "prio_callout" keyword), depending on your dm-multipath version. In other words, to modify the path and path group priorities, you would have to write your own prio_callout program/script.

The above-mentioned settings are usually set in the "devices" section of /etc/multipath.conf, or built into dm-multipath's internal defaults according to the recommendations of storage system manufacturers. In RHEL, the defaults of each version are described in an example configuration file, located in /usr/share/doc/device-mapper-multipath- directory.

If both storage controllers can accept I/O requests simultaneously, you have an active/active storage system. In that case, the solution would be to change the "path_grouping_policy" to "multibus": that would add all the LUN's paths to a single path group. This would effectively take the path group selection mechanism out of the loop, so all paths would be usable simultaneously. Then, the path selector (round-robin in your case) would choose which paths to use for each individual I/O operation. The path selector works only within a path group.

According to some quick Googling, DataCore says its SANsymphony product is supported with Linux dm-multipath. Have you checked if DataCore has some specific dm-multipath configuration instructions for your SANsymphony version?

MK

[George O'Reilly]

Matti,

Again many thanks for your time.

Give me a few days to digest this and trial and then I'll award your well deserved points.

Regards
George