PVLink 31 0x080100 Failed! The PV is not accessible

[nataris]

I get this message below from syslog.
This environment has 2 couple clusters which connect with EVA. The problem is the active cluster switch to standby cluster in time nearly happen this log below. (another cluster which connect this EVA occur this same log at the same time)

vmunix: LVM: VG 64 0x020000: PVLink 31 0x080100 Failed! The PV is not accessible.
vmunix: LVM: VG 64 0x020000: PVLink 31 0x0a0100 Failed! The PV is not accessible.
inetd[24903]: registrar/tcp: Connection from indus12 (192.168.249.94) at Thu Feb 28 06:19:16 2008
vmunix: LVM: VG 64 0x020000: Reestablished quorum.
LVM: VG 64 0x020000: PVLink 31 0x080100 Recovered.
LVM: VG 64 0x020000: PVLink 31 0x0a0100 Recovered.


My question is
1. How I find that how many disks are impacted from this PVLink and what is the HW Path for each disk.

2. How I map disk HW Path with EVA.
3. How I know what is group, vdisk, lun get this impact from PVLink Failed

4. Is it possible this issue impact cluster switch to standby. What do u think about that?

[Steven E. Protter]

Shalom,

1)http://docs.hp.com/en/5992-2854/5992-2854.pdf
2) ioscan, unless I misunderstand teh question.
3) If the cluster is properly configured taking it to "standby" means cmhaltcl which will stop everything.

SEP

[TTr]

Nataris,

The hardware path for the failed disk is shown in the syslog. It looks like you hae twoo disks that failed. One disk has major number 31 and minor number 0x080100. From the 0x080100, it is c8t0d1. You can do "ll /dev/dsk" to verify it.

The other disk mas minor number 0x0a0100 so it is probably c10t0d1 but again do "ll /dev/dsk" and look for 0x0a0100.

The volume group that is affected of course is the one that contins the two disks but also it is the one with 0x020000 minor number (syslog VG 64 0x020000). You can do "ll /dev/*/group" and see which VG has group with that number.

Unfortunately I don't have an EVA but you probably can use "armdsp" and other commandview commands to get a mapping of each LUN to the disk paths.

[Carsten.Cimander]

Hi Nataris,

you can download a tool "evainfo" from software.hp.com which displays the mapping of the devicefile to the LUNs:

#evainfo
usage: evainfo [-a|-d ] [-l] [-f fmt][-u KB|MB|GB][-w [-W][-g]
-a : Display information for all EVA Luns presented to this host.
-d : Display information for the specified EVA Lun.
-l : Display information in long/full format.
-v : Display EVAInfo tool version details.
-f : Specify output format. The supported values for are:
tab (default)
csv
-u : Specify output format for Capacity {KB|MB|GB}, default is MB.
-W : Collate output based on WWN of Luns.
-w : Display information for the specified Lun WWN.
-g : Use SCSI generic (sg) interface on Linux.
-h : Help


.... e.g.
# evainfo -a
...
/dev/rdsk/c87t1d5 5000-1FE1-5009-0D20 6005-08B4-0007-022A-0000-6000-1172-0000 205824MB Ctl-B/FP-1/Optimized
...

lssf /dev/rdsk/c87t1d5
sdisk card instance 87 SCSI target 1 SCSI LUN 5 section 0 at address 0/4/2/0/4/0.101.0.9.1.1.5 //dev/rdsk/c87t1d5

So LUN-Number on EVA is (take the last three dot-seprated-digits: (1*128)+(1*8)+5=141

To find the correspondend HBA:
rcimande@cobume01:/etc/rc.config.d #ioscan -fnH 0/4/2/0/4/0

Class I H/W Path Driver S/W State H/W Type Description
========================================================================
fc 6 0/4/2/0/4/0 fcd CLAIMED INTERFACE HP A9784-60002 PCI/PCI-X Fibre Channel FC/GigE Combo Adapter (FC Port 1)
/dev/fcd6
.....

=> so /dev/fcd6 is the HBA which sees the device

You can then check, if the HBA might be failed, the SAN-Cable, the EVA-Controller-Port, etc.


Use:
# fcmsutil /dev/fcd6 get remote all

to see the targets (i.e. the EVA Controller Ports) on the "other end" of the SAN.

You can test if you can reach these destinations by sending a SAN test packaage to them:

#fcmsutil /dev/fcd6 devstat all | grep Nport
Device Statistics for Nport_id 0x650002
Device Statistics for Nport_id 0x650005
Device Statistics for Nport_id 0x650006
Device Statistics for Nport_id 0x650007
Device Statistics for Nport_id 0x650008
Device Statistics for Nport_id 0x650009
Device Statistics for Nport_id 0x65000f
Device Statistics for Nport_id 0x650010
Device Statistics for Nport_id 0xb90002
Device Statistics for Nport_id 0xb90003
Device Statistics for Nport_id 0xb90004
Device Statistics for Nport_id 0xb90005
Device Statistics for Nport_id 0xb90007
Device Statistics for Nport_id 0xb90008
Device Statistics for Nport_id 0xb90009
Device Statistics for Nport_id 0xb9000c

Then send a test package which should look like this (if there is no defect):
#fcmsutil /dev/fcd6 test 0x650002
Sent a Test frame of size 228 bytes to nport_id 0x650002

Test for all the other FibreChannelNportIDs also.



Reagrds,
Carsten