vgextend of multiple paths to the same physical disk

[compiler]

Hi all.

I have a Linux Redhat 4.7 EL machine that connected to an EVA using 2 different HBAs in the same machine. Due to this, when I create and present a new disk, I can see it through 4 different /dev/sd* devices.

Under HP-UX, you can do:

vgextend myvg /dev/rdsk/mydisk1 /dev/rdsk/mydisk2 (and son on)

And then LVM detects that all disks are the same and adds the first device as primary and all the other devices as PVLINKS (alternate paths to switch to when the primary paths is lost).

Does Linux support this kind of behaviour? I mean, can I do:

vgextend myvg /dev/sdj /dev/sdp /dev/sdo /dev/sdx

And will work as 4 paths for the same Physical disk, or LVM will try to add "4" different physical disks to the volume group.

PS: This machine is "inherited" and multipath was not configured on it so I don't know if the "HP-UX" method would work for me. But I'm not sure if Linux's LVM supports multipath in the LVM layer itself...

[TwoProc]

The good news is that HP never needed multipath software to detect that two paths were the same disk. My guess is that it will work the same in Linux, and that lvm will know the difference between two different disks and two paths (or more) to the same disk (ne' lun).

[Steven E. Protter]

Shalom,

If all four disks are the same LUN, do the vgextend one disk at a time.

SEP

[Ivan Ferreira]

You must configure multipath for this case, or use a multipath capable FC adapter driver.

http://www.cocoanet.us/dantepasquale/blog/tech-tips/multipath-setup-in-linux

[compiler]

Hi.

I have 3 100% contradictory answers:

1.- John and Steven says that I can:

# vgextend myvg /dev/sdj
# vgextend myvg /dev/sdp
# vgextend myvg /dev/sdo
# vgextend myvg /dev/sdx

And if all 4 device files point to the same disk in the EVA via different PATHs, then LVM will notice that and will add just 1 disk with 3 extra paths instead of trying to add 4 different disks.

This WORKS in HP-UX (actually tested lots of times and PVLINKS work perfectly when a fiber path is lost, LVM layer switched to a different disk path).

and


2.- Ivan says that it won't work and I need to configure multipath.


What's the right answer? :?

[Prasanth V Aravind]

check this pdf.

http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c00710472/c00710472.pdf

GUdluck
Prasanth

[Prasanth V Aravind · Accepted solution]

forums for same issues..

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1161770

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1375018

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1161770

http://telinit0.blogspot.com/2009/03/multipathing-with-rhel5-and-hp-eva-8k.html

Gudluck
Prasanth

[TwoProc]

Simplest Answer:

a) When in doubt - go with Steven.
b) If you disagree with me, go with Steven.
c) If you disagree yourself - go with Steven.
d) If Steven disagrees with himself, ask Steven to clarify, and then go with Steven.

[Gerardo Arceri]

You need dm-multipath.
Check:
http://www.softpanorama.org/Commercial_linuxes/linux_multipath.shtml

It's pretty easy to setup and maintain, after you got it setup you will need to use the mpath devices as your PVs
like
pvcreate /dev/mpathX
vgextend myvg /dev/mpathX.

Believe me, it's very easy to setup/use

[compiler]

Hi all.

HP support (who was who installed this server) said us that multipath is configured via "qlogic driver":

# cat /etc/modprobe.conf
options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=30 ql2xloginretrycount=30 ql2xfailover=1 ql2xlbType=0 ql2xautorestore=0x80 ConfigRequired=0

They say that with ql2xfailover=1, the qlogic driver uses multipathing capabilities.

But when we list physical devices in the LVM VG they created we see:

# pvdisplay |grep "PV Name"
PV Name /dev/sdn1

# vgdisplay -v myvg
Using volume group(s) on command line
Finding volume group "myvg"
Found duplicate PV K12897688979sd87hjgtuy7t: using /dev/sdb1 not /dev/sdq1
(...lots of similar warnings like that...)
Found duplicate PV asklja9s82ji3oisd9038ujd: using /dev/sdo1 not /dev/sdl1

(skipped lvm info)

--- Physical volumes ---
PV Name /dev/sdn1
PV UUID AI89s9-2x44-2124-TOFi-OmOQ-v47R-nQXfoS
PV Status allocatable
Total PE / Free PE 38000 / 12000

PV Name /dev/sdo1
PV UUID mQ1tss-CadM-RC6G-b6F9-gaa9-IPUQ-123daS
PV Status allocatable
Total PE / Free PE 38000 / 0


I was expecting to see PV NAMES like /dev/md0 or other multipath-style names, not directly sdn or sdo...

Multipath seems not configured but LVM things appearing in device-mapper:

# cat /etc/multipath.conf
cat: /etc/multipath.conf: No such file or directory

# dmsetup ls --target=multipath
(nothing)

# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 63 May 12 20:55 control
brw-rw---- 1 root disk 253, 2 May 26 02:19 myvg-data
brw-rw---- 1 root disk 253, 0 May 12 20:55 VolGroup00-LogVol00
brw-rw---- 1 root disk 253, 1 May 12 20:55 VolGroup00-LogVol01

# ls -l /dev/dm-*
brw-r----- 1 root root 253, 0 May 12 20:55 /dev/dm-0
brw-r----- 1 root root 253, 1 May 12 20:55 /dev/dm-1
brw-r----- 1 root root 253, 2 May 26 02:19 /dev/dm-2


And that's my doubt: if qlogic driver is being used with ql2xfailover=1, it is right that HP did the pvcreate over physical devices like /dev/sdn1 and /dev/sdo1? If I unplug the HBA fiber cables that are pointed by sdo and sdn, will the driver switch to a different path and will continue working the VG?

Is multipathing properly configured in this machine via "qlogic driver"?

If I need to add a new disk to the VG, I can safely add directly the /dev/sdX device file (only 1 of all that can be seem) to it? Would LVM use that /dev/sdX device directly and CHANGE to another /dev/sdX device if that path is lost?

[Gerardo Arceri]

If you choose to use DM-Multipat, which i strongly suggest you do since the failover feature on the driver has given me nothing but trouble, add the following line to /etc/lvm/lv.conf
filter = [ "a|^/dev/mpath*|","a|^/dev/cciss/.*|","r/.*/" ]


This will include any dm-multipath pseudo devices and any pvs you might have on your cciss array controllers.
This should make these ugly messages about dupe pv's

[AVV]

Hi,

Try to download a multipath software from HP Site and install the same in your server. It is something called HP DMP (google it). You will need to restrt the services with

#service multipathd restart

Then it shows you the logical path of four paths to a single LUN and you can use that to do the pvcreate and vgcreate.

Example:

#multipath -ll


mpath11 (3600c0ff000da0ba80411e94b01000000) dm-7 HP,MSA2324fc
[size=4.7G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 1:0:1:1 sdae 65:224 [active][ready]
\_ 0:0:0:1 sda 8:0 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:1 sdk 8:160 [active][ready]
\_ 1:0:0:1 sdu 65:64 [active][ready]

In the above example, you may have to use mpath11

#pvcreate /dev/mapper/mpath11
#vgcreate vg01 /dev/mapper/mpath11

Hope it is clear to you now!!