Cannot 'see' EMC LUNs

[Tony Scully_2]

Hello all,

Have been spending the last week or so implementing a test platform consisting of an EMC DMX801 array, 2 EMC (brocade) DS-32 FC switches and a mix of 18 HP-UX and Sun hosts (all HPs are PA-RISC RP3440 or RP8420) Suns are V440s. There are a couple of winders hosts as well, but I'm not touching those!

I have an issue with Sun host, which I have logged with Sun.

All the HP hosts are fine, except one.

Its an RP3440 intended for use as a backup server. It is running 11.11 December 2003 with patches (HWenable, Online Diags and GOLDQPK of June2004). HBAs are a pair of dual port 2Gb A6826A HBAs, driver is fcd, version 11.11.03.

As this is the backup server, there are a fairly large number of LUNs presented to it. Down both ports of one card we can see all the LUNs we expect (380). Down the other card (both ports again) all we can see are the VCM database devices and the EMC 'gatekeeper' devices. There should be about 340 LUNs visible.

Before someone points this out, we do intend to split the LUNs across the cards in a more HA efficient way, ie so the loss of card does not lose all those LUNs.

So my issue are these 'invisible' LUNs. I have a case open with EMC, and will log formally with HP, but there is so much experience on this forum that I thought I would make this the first port of call. I am calling it a day now as well, so please do not think I have ignored your posts, I will pick this up first thing in the AM (here in the UK its about midnight).

Any suggestions, patches, config changes etc most welcome! Let me know if you need more output etc.

Regards,

Tony.

[Tony Scully_2]

Any ideas are welcome on this, however obvious!

EMC are baffled as we all are here,

I have changed the masking and zoning so that I could ensure the card that was not seeing the 'data' disks was OK, and it is, so I now have the disks shared across the cards, ie set of LUNs one is visible on card 1 port 1 and card 2 port 1. Set of LUNs two is still not visible.

Maybe EMC will have an answer for me tomorrow, I'm tearing out my hair!

Cheers,

Tony.

[Michael Tully]

Have EMC actually presented the missing LUNS to the secondary front end adaptor?

Are you using PowerPath?

See an example of one LUN I have which is presented down two paths. (3aa & 14aa)


Symmetrix ID: 000285502575

Device Physical Name : /dev/rdsk/c20t11d5

Device Symmetrix Name : 0117
Device Serial ID : 75117000
Symmetrix ID : 000285500000

Device Group Name : serverx_rdf
Device Logical Name : DEV025

Attached BCV Device : N/A

Attached VDEV TGT Device : N/A

Vendor ID : EMC
Product ID : SYMMETRIX
Product Revision : 5568

Device Emulation Type : FBA
Device Defined Label Type: N/A
Device Defined Label : N/A
Device Sub System Id : 0x0201

Device Block Size : 512

Device Capacity
{
Cylinders : 18414
Tracks : 276210
512-byte Blocks : 17677440
MegaBytes : 8632
KiloBytes : 8838720
}

Device Configuration : RDF1+Mir

Device is WORM Enabled : No
Device is WORM Protected : No

SCSI-3 Persistent Reserve: Disabled

Dynamic Spare Invoked : No

Dynamic RDF Capability : None

Device Service State : Normal

Device Status : Ready (RW)
Device SA Status : Ready (RW)

Front Director Paths (2):
{
----------------------------------------------------------------------
POWERPATH DIRECTOR PORT LUN
--------- ---------- ---- -------- ---------
PdevName Type Type Num Sts VBUS TID SYMM Host
----------------------------------------------------------------------
/dev/rdsk/c20t11d5 PARENT FA 03A:0 RW 002 0B 005 005
Not Visible N/A FA 14A:0 RW 002 0A 004 N/A
}

Mirror Set Type : [Data,R2 Remote,Data,N/A]

Mirror Set DA Status : [RW,RW,RW,N/A]

Mirror Set Inv. Tracks : [0,0,0,0]

Back End Disk Director Information
{
Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [02B, D, 1]
Disk Director Volume Number : 132
Hyper Number : 20
Disk Capacity : 173017m
Disk Group Number : 0

Hyper Type : R2 Remote
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [N/A,N/A,N/A]
Disk Director Volume Number : N/A
Hyper Number : N/A

Hyper Type : Data
Hyper Status : Ready (RW)
Disk [Director, Interface, TID] : [15A, D, 1]
Disk Director Volume Number : 132
Hyper Number : 20
Disk Capacity : 173017m
Disk Group Number : 0
}

RDF Information
{
Device Symmetrix Name : 0117
RDF Type : R1
RDF (RA) Group Number : 1 (A)

Remote Device Symmetrix Name : 00F7
Remote Symmetrix ID : 000285502576

R2 Device Is Larger Than The R1 Device : False

RDF Mode : Synchronous
RDF Adaptive Copy : Disabled
RDF Adaptive Copy Write Pending State : N/A
RDF Adaptive Copy Skew (Tracks) : 65535

RDF Device Domino : Disabled

RDF Link Configuration : Fibre
RDF Link Domino : Disabled
Prevent Automatic RDF Link Recovery : Enabled
Prevent RAs Online Upon Power ON : Disabled

Device RDF Status : Ready (RW)

Device SA Status : Ready (RW)
Device RA Status : Ready (RW)
Device Link Status : Ready (RW)

Device Suspend State : N/A
Device Consistency State : Disabled
RDF R2 Not Ready If Invalid : Disabled

Device RDF State : Ready (RW)
Remote Device RDF State : Write Disabled (WD)

RDF Pair State ( R1 <===> R2 ) : Synchronized

Number of R1 Invalid Tracks : 0
Number of R2 Invalid Tracks : 0
}

[Todd McDaniel_1]

I can only hope you have this figured out by now. But just in case, here goes. I would check a few things.

I have had a few times where zoning was out of whack.

0) First id make sure the switches involved, that all ports are active... We have had to reinitialize switches with "dead" ports on occasion. Look for error messages like "Login Retries".

1) HBA active and good? I would run fcmsutil reset

2) Are there any switches in between this host and Frame? I would ensure you are using the right switch ports. Login to the switch and check the active ports and disabled ones.

3) Move your fiber on the switch connected to the "bad" hba to another port on that switch.

4) What type of zoning are you doing? Switch based or HBA based... not sure if that is the proper way to say that, sorry.

5) Check to see if "read only " has been set on any of hte disks.

6) Check to ensure the zoning has been "committed". So your host can would even be able to see it.

7) Are the HBAs the same type/model etc... if not, are you on the latest patches?

8) is the card seated properly? hate it if that is actually it. Is it actually ONLINE?

9) Is the topology of the card correct? Ie... fiber optic vs direct connect?

10) Kick the Symm Frame and pray...

[Tony Scully_2]

Michael, Todd,

Thanks to both of you for your replies - We finally managed to fix this with EMC backroom help, but it took about a week in the end - fortunately this is still pre-production!

That checklist should be useful - I do still have a small issue in that I cannot see all the LUNs dosn one port - I think its a physical issue as PowerPath appears to be failing the path.

So, the fix to the issue. Well, I probably didn't provide the crucial piece of information, in that I did say that there were HPUX, Solaris and Windows hosts in the environment, but I faield to mention that they are SHARING the same FA on the sym.

It's a long story, but its mainly becasue of requirements for SRDF reducing the number of available FAs.

So, the bin file had the setting for the FA to use all possible addressing types, and the EMC implementation specialist and us thought that that was enough, but there is one further step required:

symmask -sid -wwn -dir -p set heterogeneous HPUX

Best Check that syntax, I am not in the office so I am trying to remember it(!)

That then forces the correct volume addressing (or whatever its called) for the path between the hosts and the Sym.

Hope it helps you in the future.

Cheers,

Tony.

[Tony Scully_3]

Oh dear - spoke too soon.

Now I can see all the devices down one path, but on the other I can only see the devices with a disk id of 0 - so the target ID appears to go up not the LUN, like this:

ioscan -fnuCdisk |grep 0/4/1/1
disk 11 0/4/1/1.1.1.0.0.0.0 sdisk CLAIMED DEVICE EMC SX
disk 29 0/4/1/1.1.1.0.0.0.1 sdisk CLAIMED DEVICE EMC SX
disk 35 0/4/1/1.1.1.0.0.0.2 sdisk CLAIMED DEVICE EMC SX
disk 318 0/4/1/1.1.1.0.1.14.0 sdisk CLAIMED DEVICE EMC SX
disk 353 0/4/1/1.1.1.0.1.15.0 sdisk CLAIMED DEVICE EMC SX
disk 4 0/4/1/1.1.1.0.2.0.0 sdisk CLAIMED DEVICE EMC SX
disk 44 0/4/1/1.1.1.0.2.1.0 sdisk CLAIMED DEVICE EMC SX
disk 97 0/4/1/1.1.1.0.2.2.0 sdisk CLAIMED DEVICE EMC SX
disk 122 0/4/1/1.1.1.0.2.3.0 sdisk CLAIMED DEVICE EMC SX
disk 159 0/4/1/1.1.1.0.2.4.0 sdisk CLAIMED DEVICE EMC SX
disk 190 0/4/1/1.1.1.0.2.5.0 sdisk CLAIMED DEVICE EMC SX
disk 224 0/4/1/1.1.1.0.2.6.0 sdisk CLAIMED DEVICE EMC SX
disk 259 0/4/1/1.1.1.0.2.7.0 sdisk CLAIMED DEVICE EMC SX
disk 295 0/4/1/1.1.1.0.2.8.0 sdisk CLAIMED DEVICE EMC SX
disk 332 0/4/1/1.1.1.0.2.9.0 sdisk CLAIMED DEVICE EMC SX
disk 366 0/4/1/1.1.1.0.2.10.0 sdisk CLAIMED DEVICE EMC SX
disk 416 0/4/1/1.1.1.0.2.11.0 sdisk CLAIMED DEVICE EMC SX
disk 453 0/4/1/1.1.1.0.2.12.0 sdisk CLAIMED DEVICE EMC SX
disk 506 0/4/1/1.1.1.0.2.13.0 sdisk CLAIMED DEVICE EMC SX
disk 546 0/4/1/1.1.1.0.2.14.0 sdisk CLAIMED DEVICE EMC SX
disk 599 0/4/1/1.1.1.0.2.15.0 sdisk CLAIMED DEVICE EMC SX



This is killing me - any ideas????

[Tony Scully_2]

That was weird - I was able to assign points to my last reply, even as the question author. I almost gave myself ten and a rabbit in a hat, but better not!

Anyway, this is FINALLY solved - the outstanding problem (only the zero'th LUN being seen) has been fixed. This was a legacy of trying to fix the earlier problem, as we used the

symmask ......... set offset lun on ......

command.

Just had to reverse it:

symmask .......... set lun offset off ......

The full command syntax can be seen using symmask -h.

I'm soooo happy. The last couple of weeks have been a big learning curve coming from an XP512 environment to EMC.

Cheers,
Tony.