Re: EVA LUNS PVLink Failed! The PV is still accessible

Hi Gurus, need advise about this issue. This happened after I added 2 new LUNs to my server. At the moment, it giving me these sets of errors every 2minutes and it's flooding the syslog already.

I've checked the EVA logs and didn't see any errors. Also checked the FC cards and they are both fine.

On the switch side though, I don't know what and how to check it.

But for this particular error, what other things can I do to fix it?

sample log entry
==============================================
Dec 8 15:35:33 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x0c0200 Failed! The PV is still accessible.
Dec 8 15:35:33 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x0d0200 Failed! The PV is still accessible.
Dec 8 15:35:33 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x100200 Failed! The PV is still accessible.
Dec 8 15:35:33 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x110200 Failed! The PV is still accessible.
Dec 8 15:36:04 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x0c0200 Recovered.
Dec 8 15:36:04 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x0d0200 Recovered.
Dec 8 15:36:04 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x100200 Recovered.
Dec 8 15:36:04 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x110200 Recovered.
=============================================

Thanks!

15 REPLIES 15

>>>>Dec 8 15:36:04 syfcispr vmunix: LVM: VG 64 0x010000: PVLink 31 0x110200 Recovered.<<<

take look at

EMS - events,

/var/opt/resmon/log/event.log -> check this file,

vgdisplay -v |more --> look for any stale extents

for me it looks like "PV" has been recovered,but you need check with "dd" command to perfrom through check of the disk

demsg

or if you post the vgdisplay -v output will helpfull to narrow down

Problems are common to all, but attitude makes the difference

Hi Johnson,

I didn't find any EMS events which is connected to this issue.

Nor where there anything on the event.log file

No stale extents from vgdisplay (pls. see o/p below)

Here's the vgdisplay -v o/p:

# vgdisplay -v VGSAP
--- Volume groups ---
VG Name /dev/VGSAP
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 23
Open LV 23
Max PV 16
Cur PV 3
Act PV 3
Max PE per PV 64000
VGDA 6
PE Size (Mbytes) 16
Total PE 92785
Alloc PE 62864
Free PE 29921
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/VGSAP/LVSWAP
LV Status available/syncd
LV Size (Mbytes) 20000
Current LE 1250
Allocated PE 1250
Used PV 1

LV Name /dev/VGSAP/sapinst
LV Status available/syncd
LV Size (Mbytes) 35040
Current LE 2190
Allocated PE 2190
Used PV 1

LV Name /dev/VGSAP/tmp_backup
LV Status available/syncd
LV Size (Mbytes) 350000
Current LE 21875
Allocated PE 21875
Used PV 1

LV Name /dev/VGSAP/PDE_TRANS
LV Status available/syncd
LV Size (Mbytes) 30000
Current LE 1875
Allocated PE 1875
Used PV 1

LV Name /dev/VGSAP/oracle
LV Status available/syncd
LV Size (Mbytes) 192
Current LE 12
Allocated PE 12
Used PV 1

LV Name /dev/VGSAP/ora_PDE
LV Status available/syncd
LV Size (Mbytes) 112
Current LE 7
Allocated PE 7
Used PV 1

LV Name /dev/VGSAP/ora_client
LV Status available/syncd
LV Size (Mbytes) 112
Current LE 7
Allocated PE 7
Used PV 1

LV Name /dev/VGSAP/ora_stage
LV Status available/syncd
LV Size (Mbytes) 6000
Current LE 375
Allocated PE 375
Used PV 1

LV Name /dev/VGSAP/ora_PRE
LV Status available/syncd
LV Size (Mbytes) 736
Current LE 46
Allocated PE 46
Used PV 1

LV Name /dev/VGSAP/PRE_92064
LV Status available/syncd
LV Size (Mbytes) 5504
Current LE 344
Allocated PE 344
Used PV 1

LV Name /dev/VGSAP/PRE_mirrlogA
LV Status available/syncd
LV Size (Mbytes) 208
Current LE 13
Allocated PE 13
Used PV 1

LV Name /dev/VGSAP/PRE_mirrlogB
LV Status available/syncd
LV Size (Mbytes) 208
Current LE 13
Allocated PE 13
Used PV 1

LV Name /dev/VGSAP/PRE_oraarch
LV Status available/syncd
LV Size (Mbytes) 40000
Current LE 2500
Allocated PE 2500
Used PV 1

LV Name /dev/VGSAP/PRE_origlogA
LV Status available/syncd
LV Size (Mbytes) 208
Current LE 13
Allocated PE 13
Used PV 1

LV Name /dev/VGSAP/PRE_origlogB
LV Status available/syncd
LV Size (Mbytes) 208
Current LE 13
Allocated PE 13
Used PV 1

LV Name /dev/VGSAP/PRE_sapdata1
LV Status available/syncd
LV Size (Mbytes) 120000
Current LE 7500
Allocated PE 7500
Used PV 1

LV Name /dev/VGSAP/PRE_sapdata2
LV Status available/syncd
LV Size (Mbytes) 120000
Current LE 7500
Allocated PE 7500
Used PV 1

LV Name /dev/VGSAP/PRE_sapdata3
LV Status available/syncd
LV Size (Mbytes) 120000
Current LE 7500
Allocated PE 7500
Used PV 1

LV Name /dev/VGSAP/PRE_sapdata4
LV Status available/syncd
LV Size (Mbytes) 120000
Current LE 7500
Allocated PE 7500
Used PV 1

LV Name /dev/VGSAP/PRE_sapreorg
LV Status available/syncd
LV Size (Mbytes) 5008
Current LE 313
Allocated PE 313
Used PV 1

LV Name /dev/VGSAP/PRE_mnt
LV Status available/syncd
LV Size (Mbytes) 4096
Current LE 256
Allocated PE 256
Used PV 1

LV Name /dev/VGSAP/PRE_sap
LV Status available/syncd
LV Size (Mbytes) 8192
Current LE 512
Allocated PE 512
Used PV 1

LV Name /dev/VGSAP/LVSWAP2
LV Status available/syncd
LV Size (Mbytes) 20000
Current LE 1250
Allocated PE 1250
Used PV 1

--- Physical volumes ---
PV Name /dev/dsk/c12t0d1
PV Name /dev/dsk/c13t0d1 Alternate Link
PV Name /dev/dsk/c14t0d1 Alternate Link
PV Name /dev/dsk/c15t0d1 Alternate Link
PV Name /dev/dsk/c16t0d1 Alternate Link
PV Name /dev/dsk/c17t0d1 Alternate Link
PV Name /dev/dsk/c18t0d1 Alternate Link
PV Name /dev/dsk/c19t0d1 Alternate Link
PV Status available
Total PE 63991
Free PE 1127
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c12t0d2
PV Name /dev/dsk/c13t0d2 Alternate Link
PV Name /dev/dsk/c14t0d2 Alternate Link
PV Name /dev/dsk/c15t0d2 Alternate Link
PV Name /dev/dsk/c16t0d2 Alternate Link
PV Name /dev/dsk/c17t0d2 Alternate Link
PV Name /dev/dsk/c18t0d2 Alternate Link
PV Name /dev/dsk/c19t0d2 Alternate Link
PV Status available
Total PE 12797
Free PE 12797
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c12t0d3
PV Name /dev/dsk/c13t0d3 Alternate Link
PV Name /dev/dsk/c14t0d3 Alternate Link
PV Name /dev/dsk/c15t0d3 Alternate Link
PV Name /dev/dsk/c17t0d3 Alternate Link
PV Name /dev/dsk/c16t0d3 Alternate Link
PV Name /dev/dsk/c18t0d3 Alternate Link
PV Status available
Total PE 15997
Free PE 15997
Autoswitch On
Proactive Polling On

Thanks again.

than you no need to worry .

can you post the below output

ll /dev/vg*/group

in order to confirm, exactly "0x010000" is refering to VGSAP

Problems are common to all, but attitude makes the difference

Here's the o/p:
# ll /dev/*/group
cr--r--r-- 1 root sys 64 0x010000 Jan 25 2006 /dev/VGSAP/group
crw-r----- 1 root sys 64 0x000000 Jan 6 2006 /dev/vg00/group

The trouble is, there is high I/O on these LUNs when I look at them thru the monitoring software we're using. The service time goes to as high as 100000ms! and it worries me when it does that.

if you are realy worried, about I/O ,

we can check using the "dd" command to verify the disks, still if you see any issue, better to log call to HP - HW call

run below command to check

dd if=/dev/rdsk/c12t0d1 of=/dev/null bs=256k &
dd if=/dev/rdsk/c12t0d2 of=/dev/null bs=256k &
dd if=/dev/rdsk/c12t0d3 of=/dev/null bs=256k &

take note "record in & out " should be same, if not you have "BAD" disk

sample :-

560057+1 records in
560057+1 records out

Problems are common to all, but attitude makes the difference

Hi,

As i can see one link is missing in the vgdisplay o/p, total should be 8 but here its showing7 and please check the same or recreate the device file by insf -H harware path command.

PV Name /dev/dsk/c12t0d3
PV Name /dev/dsk/c13t0d3 Alternate Link
PV Name /dev/dsk/c14t0d3 Alternate Link
PV Name /dev/dsk/c15t0d3 Alternate Link
PV Name /dev/dsk/c17t0d3 Alternate Link
PV Name /dev/dsk/c16t0d3 Alternate Link
PV Name /dev/dsk/c18t0d3 Alternate Link
PV Status available
Total PE 15997
Free PE 15997
Autoswitch On
Proactive Polling On

Reagards,
Prasanth Thomas.

insf -e

ioscan -fn

ioscan -fnC disk

vgdisplay -v VGSAP,

Problems are common to all, but attitude makes the difference

>>As i can see one link is missing in the vgdisplay o/p, total should be 8 but here <<<

# insf -e

-e Reinstall the special files for pseudo-drivers and
existing devices. This is useful for restoring
special files if one or more have been removed.

Once device file has been claimed or shown

# vgextend

Problems are common to all, but attitude makes the difference

Once your issue has been resolved...

D'not forget assign points :)

Problems are common to all, but attitude makes the difference