device-mapper: table: 253:4: multipath: error getting device

[Tony Cicone]

We are converting from EMC PowerPath to RHEL MPIO, and having a problem with the root mpath device getting created. PowerPath has been removed from the server, and MPIO installed. The rootvg and uservg are under LVM control.

(lablinux1)# vgs
Found duplicate PV dWeaYIOdsoowdRkNFOUkNTGtx2fIZf6J: using /dev/sdc2 not /dev/sda2
VG #PV #LV #SN Attr VSize VFree
rootvg 1 1 0 wz--n- 18.00G 0
uservg 1 2 0 wz--n- 33.69G 28.69G
(lablinux1)# vgs
Found duplicate PV dWeaYIOdsoowdRkNFOUkNTGtx2fIZf6J: using /dev/sdc2 not /dev/sda2
VG #PV #LV #SN Attr VSize VFree
rootvg 1 1 0 wz--n- 18.00G 0
uservg 1 2 0 wz--n- 33.69G 28.69G
(lablinux1)# vgdisplay -c
Found duplicate PV dWeaYIOdsoowdRkNFOUkNTGtx2fIZf6J: using /dev/sdc2 not /dev/sda2
rootvg:r/w:772:-1:0:1:1:-1:0:1:1:18874368:32768:576:576:0:TfdtT7-BhS3-iorQ-AF1h-ush3-2oRO-gqvxd7
uservg:r/w:772:-1:0:2:2:-1:0:1:1:35323904:32768:1078:160:918:oZEgvO-Hz9r-Q6eB-SRdX-TybD-3UTK-MLLH59
(lablinux1)# pvs
Found duplicate PV dWeaYIOdsoowdRkNFOUkNTGtx2fIZf6J: using /dev/sdc2 not /dev/sda2
PV VG Fmt Attr PSize PFree
/dev/dm-1 uservg lvm2 a- 33.69G 28.69G
/dev/sdc2 rootvg lvm2 a- 18.00G 0
(lablinux1)# pvdisplay -c
Found duplicate PV dWeaYIOdsoowdRkNFOUkNTGtx2fIZf6J: using /dev/sdc2 not /dev/sda2
/dev/sdc2:rootvg:37752750:-1:8:8:-1:32768:576:0:576:dWeaYI-Odso-owdR-kNFO-UkNT-Gtx2-fIZf6J
/dev/dm-1:uservg:70709760:-1:8:8:-1:32768:1078:918:160:P7Ja29-U3cY-hoO9-rkyX-Zlx4-gMqq-F3l8Z0

It will create the mpath1 device fine, but not mpath0.

(lablinux1)# multipath -l
mpath1 (36006048000018772146653594d314337) dm-1 EMC,SYMMETRIX
[size=34G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:2 sdb 8:16 [active][undef]
\_ 1:0:0:2 sdd 8:48 [active][undef]

(lablinux1)# multipath -v9
(lablinux1)# tail /var/log/messages
Feb 4 11:10:16 lablinux1 kernel: device-mapper: ioctl: error adding target to table
Feb 4 11:10:16 lablinux1 multipathd: dm-4: remove map (uevent)
Feb 4 11:10:27 lablinux1 snmpd[2869]: Connection from UDP: [127.0.0.1]:32894
Feb 4 11:10:28 lablinux1 snmpd[2869]: Connection from UDP: [127.0.0.1]:32894
Feb 4 11:10:33 lablinux1 kernel: device-mapper: table: 253:4: multipath: error getting device
Feb 4 11:10:33 lablinux1 kernel: device-mapper: ioctl: error adding target to table
Feb 4 11:10:33 lablinux1 multipathd: dm-4: remove map (uevent)
Feb 4 11:10:33 lablinux1 kernel: device-mapper: table: 253:4: multipath: error getting device
Feb 4 11:10:33 lablinux1 kernel: device-mapper: ioctl: error adding target to table
Feb 4 11:10:33 lablinux1 multipathd: dm-4: remove map (uevent)

I'll attach the multipath debug output.

Thanks for anyones help.

[Jimmy Vance]

The main thing I've gathered from researching this is you have to use non_friendly_names in grub and fstab. The devices will be named something like
/dev/mapper/WWID[p].

This Paper should help

Booting Linux x86 and x86_64 systems from a Storage Area Network with Device Mapper Multipath - Enabling the High Availability feature for Operating System Partitions v1.1


Here is the URL

http://h20000.www2.hp.com/bizsupport/TechSupport/CoreRedirect.jsp?redirectReason=DocIndexPDF&prodSeriesId=406734&targetPage=http%3A%2F%2Fbizsupport2.austin.hp.com%2Fbc%2Fdocs%2Fsupport%2FSupportManual%2Fc01513866%2Fc01513866.pdf

Also this Red Hat Bugzilla entry has a lot of information on making this work

https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=445734

[Tony Cicone]

Hi Jimmy,

Thanks for the info, but the first link did not work. I can configure multipathing on the boot drive fine from kickstart. But, my problem is that we have servers installed with PowerPath boot from SAN, and I want to convert them to MPIO. I have removed PowerPath completely, but I can't get multipath to create the mpath0 device, while I'm booted off the drive. I have also tried booting off the rescue disk, and creating the multipath devices, with the same result.

[Jimmy Vance]

updated 1st link

http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01513866/c01513866.pdf

[Jimmy Vance]

what version of RHEL are you working with?

I see in your output it's listing entries like /dev/sda etc, they need to be /dev/mapper/

Try booting into rescue mode with

linux mpath rescue

or once in rescue mode start the multipathd deamon.

I just built a default install of RHEL5.3 x64 system using a BL465 and an EVA4400 and it's booting from SAN without issue. The SAN LUN is 100GB and mount displays

/dev/mapper/VolGroup00-LogVol00 on / type ext3
/dev/mapper/mpath0p1 on /boot type ext3

multipath -l shows

mpath0 (3600508b4000985f40000a00000150000) dm-0 HP,HSV300
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:1 sda 8:0 [active][undef]
\_ 1:0:1:1 sdd 8:48 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:1:1 sdb 8:16 [active][undef]
\_ 1:0:0:1 sdc 8:32 [active][undef]

[Tony Cicone]

I have no problem from kickstart.

It's RHEL 5.1, and here's the procedure I'm trying.

boot off the rescue CD and skip the search part

# mkdir /mnt/sysimage
# mkdir /mnt/sysimage/boot
# multipath -F
# multipath -v0
# multipath -l

I can see the mpath devices now

# kpartx -a /dev/mapper/mpath0
# lvm vgscan --config 'devices{ filter = [ "a/mapper/", "r/.*/" ] }'
# lvm vgchange -ay --config 'devices{ filter = [ "a/mapper/", "r/.*/" ] }'
# mount -t ext3 /dev/mapper/VolGroup00-LogVol00 /mnt/sysimage/ # mount -t ext3 /dev/mapper/mpath0p1 /mnt/sysimage/boot/ # mount -t proc none /mnt/sysimage/proc # chroot /mnt/sysimage # dmsetup mknodes mpath0 # dmsetup mknodes mpath0p1 # dmsetup mknodes mpath0p2

# mount /dev/mapper/mpath0p1 /boot
# grub-install /dev/mapper/mpath0

When I reboot, it comes up on the old sda drives and not the multipath devices?

Thanks again.

[Jimmy Vance]

After you complete the above what does /etc/fstab and /boot/menu.lst show?

have you made a new initrd image? according to the bugzilla I posted that is something you must do.

[Tony Cicone]

I was able to fix the issue with the following changes to my procedure:

vi the /etc/sysconfig/mkinitrd/multipath file and set "MULTIPATH=yes".

mount /dev /mnt/sysimage/dev
mount /sys /mnt/sysimage/sys

run the mkinitrd to create a new kernel.

Thanks for your help.

[Tony Cicone]

solution above