HPE EVA Storage

Autopath: Claimed devices failed

 
SOLVED
Go to solution
Henrik Rasmussen
Occasional Advisor

Autopath: Claimed devices failed

On our EVA8000 I have two disk groups, one called "Default Disk Group" with 300 GB Fibre Channel Disks, and one called "FATA Disk Group" which is 1TB Near-Online FATA disks.

 

Our topology consists of two SAN switches cross-connected to both the two figure interfaces on the EVA and two HBAs on each server (redundant setup). Therefore each LUN are

presented to each of the servers 8 times (8 paths). Both servers are configured in the same member zone in the SAN switches.

 

I've just added a new LUN to the "Default Disk Group", presenting it to the HP-UX 11.23 servers (two servers in a Service Guard Cluster). I have been running ioscan and insf. 

 

I have done this procedure many times before, adding the new LUNs to the LVM config, extending filesystems.

As usual, the devices (8 paths) are created and is Claimed (to debug the problem I unpresented the LUN which was confirmed by NO_HW on the device files). Re-presenting them

 

again also showed that they are Claimed. This confirms that the servers are actually able to see the LUN.

However, when I tried issuing the pvcreate on the device (no matter which of the 8 paths I chose), pvcreate displayd an error message: pvcreate: Couldn't open physical volume"/dev/rdsk/cXXt0d2": No such device or address.

 

When I run 'autopath display' it shows "Failed" on each 8 paths associated with some of the LUNs including the new one I lust presented, but Active for each of the 8 paths on other LUNs. And it is the same on the other server in the cluster.

I've tried deleting the new LUN (by rmsf -H device files prior to deletion), re-adding it, presenting it, and run ioscan and insf, but to no avail. Issuing autopath discover doesn't help either.

 

No systems seem to be affected by it, however I can't extend a logical volume as I can't pvcreate the new device. The SAN switches seems to be okay. If it was a fibre problem, only some path's to each LUN would be affected, not each path on only some LUNs.

 

The new LUN is managed by controller A on the EVA, the same controller as LUNs that do not have the problem LUNs that autopath shows as active).

 

Concentrating on one of my VGs, it contains 4 LUNS which autopath shows as active. On the EVVA they are all configured Default Disk Group, Vraid5, Read cache on, Mirrored,

 

Write cache: Write-back, and in LUNs of 30GB, 165GB, 25GB, 60GB, 100GB, 25GB, and the new LUN is 100GB configured as the other's.

 

I can't seem to figure out what is wrong. Anyone?

3 REPLIES 3
Torsten.
Acclaimed Contributor

Re: Autopath: Claimed devices failed

Maybe the access id not allowed after the unpresenting/presenting again. Anything related in syslog?

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Henrik Rasmussen
Occasional Advisor

Re: Autopath: Claimed devices failed

The problem was there before unpresenting. No associated entries prior to unpresenting. But when I unpresented (after removing the special files), the syslog of course recorded I/O errors. Also problem is not only regarding the unpresented LUN, but several others already being presented for months. I also tried re-adding the new LUN with a new free LUN number, but this didn't solve it either. Here is syslog info and some autopath outout

 

[root] artemis:/var/adm/syslog> grep vmunix syslog.log.0

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 09:47:07 artemis vmunix: DIAGNOSTIC SYSTEM WARNING:

Jun 10 09:47:07 artemis vmunix:    The diagnostic logging facility has started receiving excessive

Jun 10 09:47:07 artemis vmunix:    errors from the I/O subsystem.  I/O error entries will be lost

Jun 10 09:47:07 artemis vmunix:    until the cause of the excessive I/O logging is corrected.

Jun 10 09:47:07 artemis vmunix:    If the diaglogd daemon is not active, use the Daemon Startup command

Jun 10 09:47:07 artemis vmunix:    in stm to start it.

Jun 10 09:47:07 artemis vmunix:    If the diaglogd daemon is active, use the logtool utility in stm

Jun 10 09:47:07 artemis vmunix:    to determine which I/O subsystem is logging excessive errors.

Jun 10 09:47:19 artemis vmunix: DIAGNOSTIC SYSTEM WARNING:

Jun 10 09:47:19 artemis vmunix:    The diagnostic logging facility is no longer receiving excessive

Jun 10 09:47:19 artemis vmunix:    errors from the I/O subsystem.  2 I/O error entries were lost.

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 14:01:06 artemis vmunix: 0/2/1/0.2.12.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.10.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.14.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.8.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.8.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.11.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.9.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.10.0.0.3.6 sdisk

Jun 10 14:28:50 artemis vmunix: LUN deleted succesfully..Porceeding to update KRS -600508B400074442000170000CBE0000

Jun 10 14:30:23 artemis vmunix: AUTOPATH: Device 600508B400074442000170000CBE0000 added back

Jun 10 14:31:12 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:32:04 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:33:39 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:39:14 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 14:40:02 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB0E3600> error returned <5>

Jun 10 14:40:11 artemis vmunix:

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc0e3600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB213600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc213600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB123600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc123600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB233600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc233600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB253600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc253600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB273600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc273600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB293600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc293600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB2B3600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc2b3600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : All the paths to the end device 0xbc2b3600 failed!

Jun 10 14:40:11 artemis vmunix:

[root] artemis:/var/adm/syslog> grep vmunix syslog.log.0

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 09:34:27 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 09:34:30 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 09:47:07 artemis vmunix: DIAGNOSTIC SYSTEM WARNING:

Jun 10 09:47:07 artemis vmunix:    The diagnostic logging facility has started receiving excessive

Jun 10 09:47:07 artemis vmunix:    errors from the I/O subsystem.  I/O error entries will be lost

Jun 10 09:47:07 artemis vmunix:    until the cause of the excessive I/O logging is corrected.

Jun 10 09:47:07 artemis vmunix:    If the diaglogd daemon is not active, use the Daemon Startup command

Jun 10 09:47:07 artemis vmunix:    in stm to start it.

Jun 10 09:47:07 artemis vmunix:    If the diaglogd daemon is active, use the logtool utility in stm

Jun 10 09:47:07 artemis vmunix:    to determine which I/O subsystem is logging excessive errors.

Jun 10 09:47:19 artemis vmunix: DIAGNOSTIC SYSTEM WARNING:

Jun 10 09:47:19 artemis vmunix:    The diagnostic logging facility is no longer receiving excessive

Jun 10 09:47:19 artemis vmunix:    errors from the I/O subsystem.  2 I/O error entries were lost.

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 09:53:21 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 14:01:06 artemis vmunix: 0/2/1/0.2.12.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.10.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.14.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/2/1/0.2.8.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.8.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.11.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.9.0.0.3.6 sdisk

Jun 10 14:01:08 artemis vmunix: 0/5/1/0.1.10.0.0.3.6 sdisk

Jun 10 14:28:50 artemis vmunix: LUN deleted succesfully..Porceeding to update KRS -600508B400074442000170000CBE0000

Jun 10 14:30:23 artemis vmunix: AUTOPATH: Device 600508B400074442000170000CBE0000 added back

Jun 10 14:31:12 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:32:04 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:33:39 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:39:14 artemis vmunix: WARNING: Unable to configure alternate path for dump devices.

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.11.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.8.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.9.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/5/1/0.1.10.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.10.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.12.0.0.0.2 sdisk

Jun 10 14:39:58 artemis vmunix: 0/2/1/0.2.8.0.0.0.2 sdisk

Jun 10 14:40:02 artemis vmunix: 0/2/1/0.2.14.0.0.0.2 sdisk

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB0E3600> error returned <5>

Jun 10 14:40:11 artemis vmunix:

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc0e3600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB213600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc213600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB123600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc123600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB233600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc233600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB253600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc253600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB273600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc273600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB293600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc293600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : PPM Failing path <CB2B3600> error returned <5>

Jun 10 14:40:11 artemis vmunix: AUTOPATH : Path 0xbc2b3600 failed! Rerouting to alternate path

Jun 10 14:40:11 artemis vmunix: AUTOPATH : All the paths to the end device 0xbc2b3600 failed!

 

 

[root] artemis:/var/adm/syslog> autopath display

 

==================================================================

HPswsp Version        : A.3.0F.04F.01F

==================================================================

Auto Discover         : ON

==================================================================

Array Type            : EVA-AA

Array WWN             : 5000-1FE1-500B-FD30

Read Optimize         : ON

Path Verification Period : 00:10

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-036F-0000

Virtual Device File   : /hpap/dsk/hpap17

 Load Balancing Policy : No Load Balancing

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d2               Failed

 /dev/dsk/c33t0d2               Failed

 /dev/dsk/c18t0d2               Failed

 /dev/dsk/c35t0d2               Failed

 /dev/dsk/c37t0d2               Failed

 /dev/dsk/c39t0d2               Failed

 /dev/dsk/c41t0d2               Failed

 /dev/dsk/c43t0d2               Failed

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-0365-0000

Virtual Device File   : /hpap/dsk/hpap18

 Load Balancing Policy : No Load Balancing

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d3               Failed

 /dev/dsk/c33t0d3               Failed

 /dev/dsk/c18t0d3               Failed

 /dev/dsk/c35t0d3               Failed

 /dev/dsk/c37t0d3               Failed

 /dev/dsk/c39t0d3               Failed

 /dev/dsk/c41t0d3               Failed

 /dev/dsk/c43t0d3               Failed

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-0360-0000

Virtual Device File   : /hpap/dsk/hpap19

 Load Balancing Policy : No Load Balancing

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d4               Failed

 /dev/dsk/c33t0d4               Failed

 /dev/dsk/c18t0d4               Failed

 /dev/dsk/c35t0d4               Failed

 /dev/dsk/c37t0d4               Failed

 /dev/dsk/c39t0d4               Failed

 /dev/dsk/c41t0d4               Failed

 /dev/dsk/c43t0d4               Failed

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-036A-0000

Virtual Device File   : /hpap/dsk/hpap20

 Load Balancing Policy : No Load Balancing

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d5               Failed

 /dev/dsk/c33t0d5               Failed

 /dev/dsk/c18t0d5               Failed

 /dev/dsk/c35t0d5               Failed

 /dev/dsk/c37t0d5               Failed

 /dev/dsk/c39t0d5               Failed

 /dev/dsk/c41t0d5               Failed

 /dev/dsk/c43t0d5               Failed

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-0EDC-0000

Virtual Device File   : /hpap/dsk/hpap32

 Load Balancing Policy : No Load Balancing

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d6               Failed

 /dev/dsk/c33t0d6               Failed

 /dev/dsk/c18t0d6               Failed

 /dev/dsk/c35t0d6               Failed

 /dev/dsk/c37t0d6               Failed

 /dev/dsk/c39t0d6               Failed

 /dev/dsk/c41t0d6               Failed

 /dev/dsk/c43t0d6               Failed

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-0C27-0000

Virtual Device File   : /hpap/dsk/hpap0

 Load Balancing Policy : Shortest Queue Length

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t0d7               Active

 /dev/dsk/c33t0d7               Active

 /dev/dsk/c18t0d7               Active

 /dev/dsk/c35t0d7               Active

 /dev/dsk/c37t0d7               Active

 /dev/dsk/c39t0d7               Active

 /dev/dsk/c41t0d7               Active

 /dev/dsk/c43t0d7               Active

 

==================================================================

 

==================================================================

Lun WWN               : 6005-08B4-0007-4442-0001-7000-0C2A-0000

Virtual Device File   : /hpap/dsk/hpap1

 Load Balancing Policy : Shortest Queue Length

 Lun Timeout           : Infinite Retry (-1)

==================================================================

Device Path                    Status

==================================================================

/dev/dsk/c14t1d0               Active

 /dev/dsk/c33t1d0               Active

 /dev/dsk/c18t1d0               Active

 /dev/dsk/c35t1d0               Active

 /dev/dsk/c37t1d0               Active

 /dev/dsk/c39t1d0               Active

 /dev/dsk/c41t1d0               Active

 /dev/dsk/c43t1d0               Active

 

==================================================================

Henrik Rasmussen
Occasional Advisor
Solution

Re: Autopath: Claimed devices failed

I soved the problem. Actually not quite sure what the solution was, but the steps I went through was as follows:

 

1) Change LUN Preferred path/mode from "No preference" to "Path A-failover/failback"

2) ioscan # again

3) insf # again, but this time with -e option

4) autopath discover #again

5) autopath display

 

This time autopath displayed two different LUNs on the same WWN, but with the "new" LUN not existing in the ioscan output (probably an old LUN having the same WWN which autopath remembered).

 

6) autopath delete -l <WWN> #which deleted the WWN in question

7) autopath display # confirming the WWN was removed and the pathses along with it

8) autopath add -l <WWN>

7) autopath discover

8) autopath display # confirming tat the WWN was added and not without the non-existing device path

9) pvcreate /dev/rdsk/c14t0d2

...

 

Problem solved