System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

vgextend of multiple paths to the same physical disk

 
SOLVED
Go to solution
compiler
Frequent Advisor

vgextend of multiple paths to the same physical disk


Hi all.

I have a Linux Redhat 4.7 EL machine that connected to an EVA using 2 different HBAs in the same machine. Due to this, when I create and present a new disk, I can see it through 4 different /dev/sd* devices.

Under HP-UX, you can do:

vgextend myvg /dev/rdsk/mydisk1 /dev/rdsk/mydisk2 (and son on)

And then LVM detects that all disks are the same and adds the first device as primary and all the other devices as PVLINKS (alternate paths to switch to when the primary paths is lost).

Does Linux support this kind of behaviour? I mean, can I do:

vgextend myvg /dev/sdj /dev/sdp /dev/sdo /dev/sdx

And will work as 4 paths for the same Physical disk, or LVM will try to add "4" different physical disks to the volume group.

PS: This machine is "inherited" and multipath was not configured on it so I don't know if the "HP-UX" method would work for me. But I'm not sure if Linux's LVM supports multipath in the LVM layer itself...
11 REPLIES
TwoProc
Honored Contributor

Re: vgextend of multiple paths to the same physical disk

The good news is that HP never needed multipath software to detect that two paths were the same disk. My guess is that it will work the same in Linux, and that lvm will know the difference between two different disks and two paths (or more) to the same disk (ne' lun).

We are the people our parents warned us about --Jimmy Buffett
Steven E. Protter
Exalted Contributor

Re: vgextend of multiple paths to the same physical disk

Shalom,

If all four disks are the same LUN, do the vgextend one disk at a time.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Ivan Ferreira
Honored Contributor

Re: vgextend of multiple paths to the same physical disk

You must configure multipath for this case, or use a multipath capable FC adapter driver.

http://www.cocoanet.us/dantepasquale/blog/tech-tips/multipath-setup-in-linux
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
compiler
Frequent Advisor

Re: vgextend of multiple paths to the same physical disk


Hi.

I have 3 100% contradictory answers:

1.- John and Steven says that I can:

# vgextend myvg /dev/sdj
# vgextend myvg /dev/sdp
# vgextend myvg /dev/sdo
# vgextend myvg /dev/sdx

And if all 4 device files point to the same disk in the EVA via different PATHs, then LVM will notice that and will add just 1 disk with 3 extra paths instead of trying to add 4 different disks.

This WORKS in HP-UX (actually tested lots of times and PVLINKS work perfectly when a fiber path is lost, LVM layer switched to a different disk path).

and


2.- Ivan says that it won't work and I need to configure multipath.


What's the right answer? :?
Prasanth V Aravind
Trusted Contributor

Re: vgextend of multiple paths to the same physical disk

TwoProc
Honored Contributor

Re: vgextend of multiple paths to the same physical disk

Simplest Answer:

a) When in doubt - go with Steven.
b) If you disagree with me, go with Steven.
c) If you disagree yourself - go with Steven.
d) If Steven disagrees with himself, ask Steven to clarify, and then go with Steven.

We are the people our parents warned us about --Jimmy Buffett
Gerardo Arceri
Trusted Contributor

Re: vgextend of multiple paths to the same physical disk

You need dm-multipath.
Check:
http://www.softpanorama.org/Commercial_linuxes/linux_multipath.shtml

It's pretty easy to setup and maintain, after you got it setup you will need to use the mpath devices as your PVs
like
pvcreate /dev/mpathX
vgextend myvg /dev/mpathX.

Believe me, it's very easy to setup/use
compiler
Frequent Advisor

Re: vgextend of multiple paths to the same physical disk


Hi all.

HP support (who was who installed this server) said us that multipath is configured via "qlogic driver":

# cat /etc/modprobe.conf
options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=30 ql2xloginretrycount=30 ql2xfailover=1 ql2xlbType=0 ql2xautorestore=0x80 ConfigRequired=0

They say that with ql2xfailover=1, the qlogic driver uses multipathing capabilities.

But when we list physical devices in the LVM VG they created we see:

# pvdisplay |grep "PV Name"
PV Name /dev/sdn1

# vgdisplay -v myvg
Using volume group(s) on command line
Finding volume group "myvg"
Found duplicate PV K12897688979sd87hjgtuy7t: using /dev/sdb1 not /dev/sdq1
(...lots of similar warnings like that...)
Found duplicate PV asklja9s82ji3oisd9038ujd: using /dev/sdo1 not /dev/sdl1

(skipped lvm info)

--- Physical volumes ---
PV Name /dev/sdn1
PV UUID AI89s9-2x44-2124-TOFi-OmOQ-v47R-nQXfoS
PV Status allocatable
Total PE / Free PE 38000 / 12000

PV Name /dev/sdo1
PV UUID mQ1tss-CadM-RC6G-b6F9-gaa9-IPUQ-123daS
PV Status allocatable
Total PE / Free PE 38000 / 0


I was expecting to see PV NAMES like /dev/md0 or other multipath-style names, not directly sdn or sdo...

Multipath seems not configured but LVM things appearing in device-mapper:

# cat /etc/multipath.conf
cat: /etc/multipath.conf: No such file or directory

# dmsetup ls --target=multipath
(nothing)

# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 63 May 12 20:55 control
brw-rw---- 1 root disk 253, 2 May 26 02:19 myvg-data
brw-rw---- 1 root disk 253, 0 May 12 20:55 VolGroup00-LogVol00
brw-rw---- 1 root disk 253, 1 May 12 20:55 VolGroup00-LogVol01

# ls -l /dev/dm-*
brw-r----- 1 root root 253, 0 May 12 20:55 /dev/dm-0
brw-r----- 1 root root 253, 1 May 12 20:55 /dev/dm-1
brw-r----- 1 root root 253, 2 May 26 02:19 /dev/dm-2


And that's my doubt: if qlogic driver is being used with ql2xfailover=1, it is right that HP did the pvcreate over physical devices like /dev/sdn1 and /dev/sdo1? If I unplug the HBA fiber cables that are pointed by sdo and sdn, will the driver switch to a different path and will continue working the VG?

Is multipathing properly configured in this machine via "qlogic driver"?

If I need to add a new disk to the VG, I can safely add directly the /dev/sdX device file (only 1 of all that can be seem) to it? Would LVM use that /dev/sdX device directly and CHANGE to another /dev/sdX device if that path is lost?

Gerardo Arceri
Trusted Contributor

Re: vgextend of multiple paths to the same physical disk

If you choose to use DM-Multipat, which i strongly suggest you do since the failover feature on the driver has given me nothing but trouble, add the following line to /etc/lvm/lv.conf
filter = [ "a|^/dev/mpath*|","a|^/dev/cciss/.*|","r/.*/" ]


This will include any dm-multipath pseudo devices and any pvs you might have on your cciss array controllers.
This should make these ugly messages about dupe pv's
AVV
Super Advisor

Re: vgextend of multiple paths to the same physical disk

Hi,

Try to download a multipath software from HP Site and install the same in your server. It is something called HP DMP (google it). You will need to restrt the services with

#service multipathd restart

Then it shows you the logical path of four paths to a single LUN and you can use that to do the pvcreate and vgcreate.

Example:

#multipath -ll


mpath11 (3600c0ff000da0ba80411e94b01000000) dm-7 HP,MSA2324fc
[size=4.7G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 1:0:1:1 sdae 65:224 [active][ready]
\_ 0:0:0:1 sda 8:0 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 0:0:1:1 sdk 8:160 [active][ready]
\_ 1:0:0:1 sdu 65:64 [active][ready]

In the above example, you may have to use mpath11

#pvcreate /dev/mapper/mpath11
#vgcreate vg01 /dev/mapper/mpath11

Hope it is clear to you now!!