lvm mirror between EVA & EMC

[Season Chu]

Hi,All:

I have a existent vg on HP EVA. Now I want to add a new lun from EMC to this vg and make them mirror by HP pvlink. But when I did this mirror, it's failed. Pls check the below. Thanks a lot.

[root@:/]#insf -C disk
insf: Installing special files for sdisk instance 69 address 0/5/1/0.2.19.0.0.0.2
[root@:/]#ioscan -fnCdisk
Class I H/W Path Driver S/W State H/W Type Description
=============================================================================
disk 0 0/0/2/0.0.0.0 sdisk CLAIMED DEVICE _NEC DVD+-RW ND-6650A
/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0
disk 1 0/1/1/0.0.0 sdisk CLAIMED DEVICE HP 73.4GST373454LC
/dev/dsk/c2t0d0 /dev/dsk/c2t0d0s2 /dev/rdsk/c2t0d0 /dev/rdsk/c2t0d0s2
/dev/dsk/c2t0d0s1 /dev/dsk/c2t0d0s3 /dev/rdsk/c2t0d0s1 /dev/rdsk/c2t0d0s3
disk 50 0/5/1/0.1.2.0.0.0.2 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c18t0d2 /dev/rdsk/c18t0d2
disk 54 0/5/1/0.1.2.0.0.0.3 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c18t0d3 /dev/rdsk/c18t0d3
disk 58 0/5/1/0.1.2.0.0.0.4 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c18t0d4 /dev/rdsk/c18t0d4
disk 48 0/5/1/0.1.4.0.0.0.2 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c19t0d2 /dev/rdsk/c19t0d2
disk 52 0/5/1/0.1.4.0.0.0.3 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c19t0d3 /dev/rdsk/c19t0d3
disk 56 0/5/1/0.1.4.0.0.0.4 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c19t0d4 /dev/rdsk/c19t0d4
disk 49 0/5/1/0.1.11.0.0.0.2 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c20t0d2 /dev/rdsk/c20t0d2
disk 53 0/5/1/0.1.11.0.0.0.3 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c20t0d3 /dev/rdsk/c20t0d3
disk 57 0/5/1/0.1.11.0.0.0.4 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c20t0d4 /dev/rdsk/c20t0d4
disk 51 0/5/1/0.1.15.0.0.0.2 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c21t0d2 /dev/insf936 /dev/rdsk/c21t0d2
disk 55 0/5/1/0.1.15.0.0.0.3 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c21t0d3 /dev/rdsk/c21t0d3
disk 59 0/5/1/0.1.15.0.0.0.4 sdisk CLAIMED DEVICE HP HSV100
/dev/dsk/c21t0d4 /dev/rdsk/c21t0d4
disk 68 0/5/1/0.2.19.0.0.0.0 sdisk CLAIMED DEVICE DGC CX4-240WDR5
/dev/dsk/c23t0d0 /dev/rdsk/c23t0d0
disk 71 0/5/1/0.2.19.0.0.0.1 sdisk CLAIMED DEVICE DGC CX4-240WDR5
/dev/dsk/c23t0d1 /dev/rdsk/c23t0d1
disk 69 0/5/1/0.2.19.0.0.0.2 sdisk CLAIMED DEVICE DGC CX4-240WDR5
/dev/dsk/c23t0d2 /dev/rdsk/c23t0d2

[root@:/]#pvcreate /dev/rdsk/c23t0d0
Physical volume "/dev/rdsk/c23t0d0" has been successfully created.

[root@:/]#vgextend /dev/vg02 /dev/dsk/c23t0d0
vgextend: Warning: Max_PE_per_PV for the volume group (25599) too small for this PV (38399).
Using only 25599 PEs from this physical volume.
Current path "/dev/dsk/c19t0d2" is an alternate link, skip.
Current path "/dev/dsk/c20t0d2" is an alternate link, skip.
Current path "/dev/dsk/c21t0d2" is an alternate link, skip.
Volume group "/dev/vg02" has been successfully extended.
Volume Group configuration for /dev/vg02 has been saved in /etc/lvmconf/vg02.conf

vg04:

[root@:/]#pvcreate /dev/rdsk/c23t0d2
Physical volume "/dev/rdsk/c23t0d2" has been successfully created.

[root@:/]#vgextend /dev/vg04 /dev/dsk/c23t0d2
Current path "/dev/dsk/c19t0d4" is an alternate link, skip.
Current path "/dev/dsk/c20t0d4" is an alternate link, skip.
Current path "/dev/dsk/c21t0d4" is an alternate link, skip.
Volume group "/dev/vg04" has been successfully extended.
Volume Group configuration for /dev/vg04 has been saved in /etc/lvmconf/vg04.conf

[root@:/]#vgdisplay -v /dev/vg04
--- Volume groups ---
VG Name /dev/vg04
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 16
Cur PV 2
Act PV 2
Max PE per PV 5119
VGDA 4
PE Size (Mbytes) 4
Total PE 10238
Alloc PE 5118
Free PE 5120
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg04/lv_gccwls
LV Status available/syncd
LV Size (Mbytes) 20472
Current LE 5118
Allocated PE 5118
Used PV 1


--- Physical volumes ---
PV Name /dev/dsk/c18t0d4
PV Name /dev/dsk/c19t0d4 Alternate Link
PV Name /dev/dsk/c20t0d4 Alternate Link
PV Name /dev/dsk/c21t0d4 Alternate Link
PV Status available
Total PE 5119
Free PE 1
Autoswitch On

PV Name /dev/dsk/c23t0d2
PV Status available
Total PE 5119
Free PE 5119
Autoswitch On



[root@:/]#lvextend -m 1 /dev/vg04/lv_gccwls /dev/dsk/c23t0d2
The newly allocated mirrors are now being synchronized. This operation will
take some time. Please wait ....
lvextend: Couldn't re-synchronize stale partitions of the logical volume:
Device offline/Powerfailed

lvextend: Couldn't resynchronize logical volume "/dev/vg04/lv_gccwls".

[Season Chu]

Now the state of "/dev/vg04/lv_gcclws" is the below :

root@zengzhi:/]#lvdisplay -v /dev/vg04/lv_gccwls | more
--- Logical volumes ---
LV Name /dev/vg04/lv_gccwls
VG Name /dev/vg04
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 20472
Current LE 5118
Allocated PE 10236
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c18t0d4 5118 5118
/dev/dsk/c23t0d2 5118 5118

--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 /dev/dsk/c18t0d4 00000 current /dev/dsk/c23t0d2 00000 stale
00001 /dev/dsk/c18t0d4 00001 current /dev/dsk/c23t0d2 00001 stale
00002 /dev/dsk/c18t0d4 00002 current /dev/dsk/c23t0d2 00002 stale
00003 /dev/dsk/c18t0d4 00003 current /dev/dsk/c23t0d2 00003 stale
00004 /dev/dsk/c18t0d4 00004 current /dev/dsk/c23t0d2 00004 stale
00005 /dev/dsk/c18t0d4 00005 current /dev/dsk/c23t0d2 00005 stale
00006 /dev/dsk/c18t0d4 00006 current /dev/dsk/c23t0d2 00006 stale
00007 /dev/dsk/c18t0d4 00007 current /dev/dsk/c23t0d2 00007 stale
00008 /dev/dsk/c18t0d4 00008 current /dev/dsk/c23t0d2 00008 stale
00009 /dev/dsk/c18t0d4 00009 current /dev/dsk/c23t0d2 00009 stale
00010 /dev/dsk/c18t0d4 00010 current /dev/dsk/c23t0d2 00010 stale
00011 /dev/dsk/c18t0d4 00011 current /dev/dsk/c23t0d2 00011 stale
00012 /dev/dsk/c18t0d4 00012 current /dev/dsk/c23t0d2 00012 stale
00013 /dev/dsk/c18t0d4 00013 current /dev/dsk/c23t0d2 00013 stale
00014 /dev/dsk/c18t0d4 00014 current /dev/dsk/c23t0d2 00014 stale
00015 /dev/dsk/c18t0d4 00015 current /dev/dsk/c23t0d2 00015 stale
00016 /dev/dsk/c18t0d4 00016 current /dev/dsk/c23t0d2 00016 stale
00017 /dev/dsk/c18t0d4 00017 current /dev/dsk/c23t0d2 00017 stale
00018 /dev/dsk/c18t0d4 00018 current /dev/dsk/c23t0d2 00018 stale
00019 /dev/dsk/c18t0d4 00019 current /dev/dsk/c23t0d2 00019 stale
00020 /dev/dsk/c18t0d4 00020 current /dev/dsk/c23t0d2 00020 stale

...........SKIP............

[Elmar P. Kolkman]

It seems the system has problems using the new disk.
Try removing it from the LVOL and VG and do a dd to it to check its functionality...

[Season Chu]

test dd is okay and I try to create a new vg in the same lun of EMC and it's still okay.

[Elmar P. Kolkman]

That seems strange. Have you tried copying data from the c18t0d4 to the c23t0d2 using dd ?

(It looks like copying data (LVM mirror) doesn't work...)

[Bill Costigan]

Are you using powerpath for MPIO to the EMC or are you using 11iv3? If EMC, what does the command 'powermt display dev=c23t0d2' show

[Season Chu]

Thanks for your reply.

Im not using the powerpath but PVLINK. My hpux is 11.23.

[Season Chu]

I just find out some information in syslog, when Im doing "lvm mirror",pls check that:

Jun 22 13:09:19 vmunix: LVM: VG 64 0x040000: PVLink 31 0x130400 Failed! The PV is still accessible.
Jun 22 13:09:19 vmunix: LVM: VG 64 0x040000: PVLink 31 0x140400 Failed! The PV is still accessible.
Jun 22 13:09:25 vmunix: LVM: VG 64 0x040000: PVLink 31 0x130400 Recovered.
Jun 22 13:09:57 vmunix: LVM: VG 64 0x040000: PVLink 31 0x120400 Failed! The PV is not accessible.
Jun 22 13:09:57 vmunix: LVM: VG 64 0x040000: PVLink 31 0x130400 Failed! The PV is not accessible.
Jun 22 13:09:57 vmunix: LVM: VG 64 0x040000: PVLink 31 0x150400 Failed! The PV is not accessible.
Jun 22 13:10:02 vmunix: LVM: VG 64 0x040000: PVLink 31 0x120400 Recovered.
Jun 22 13:10:07 vmunix: LVM: VG 64 0x040000: PVLink 31 0x150400 Recovered. resynchronize logical volume "/dev/vg04/lv_gccwls".

[Bill Costigan]

That could be a problem. The Clariion was typically a active/passive controller array. This meant that you could only access a LUN through one controller. Access through the other was blocked unless the LUN was switched (tresspassed) to that controller. PowerPath was required to make that work correctly. EMC add the ALUA capability with one of the newer FLARE releases. However, the correct setting must be selected and the OS must support this. I know it is supported under 11iv3, but I'm not sure about 11iv2.

[Season Chu]

thanks Bill.

I just did the same action to mirror a lun between this EVA and EMC on another hpux (OS is 11.11). Its done ! So I guess the problem is on that hpux not the EMC.

BTW,now the path of lun from EMC is only one.

PV Name /dev/dsk/c23t0d2 >>>>>>>it's EMC
PV Status available
Total PE 5119
Free PE 5119
Autoswitch On