Cannot see EMC drives (strange "visibility" issue)

[cninham]

The following problem has me, and our EMC storage guy, stumped! I know this is most likely an EMC configuration issue, but I am taking a chance that someone in this forum can help us, or point us in the right direction... (I have always been getting helpful advice from these ITRC forums!)

We have a 9000/800/L-class server (L2000), running HP-UX 11.11 (11i). The server has two A5158 fibre channel cards installed, but we are currently using only one. We have not installed PowerPath. I have installed the latest Dec 03 software and hardware patch bundles. Lets call this server 'ServerA' -- the server with the visibility problem.

First, the server/OS sees the A5158 fibre channel cards. I did an 'ioscan' and 'insf -e' -- all is like it should be...

When I do a 'ioscan -funC disk' or use 'sam' to look at the available drives, to install/configre the EMC drives, I see *ONLY* the small VCM database drive from the EMC... I do not see the other three disk partitions assigned to this card+server.

Some time back, some EMC drives were created and assigned to another/different server. Lets call this server 'ServerB'. ServerB sees its EMC drives just fine. If the EMC guy reassigns these EMC drives that were originally assigned to ServerB, to ServerA, I can see the VCM database/drive and the other EMC drives/partitions.

If we go back to ServerB (the other server), and reassign the drives that were created for ServerA, to ServerB, I can *NOT* see the drives...!!! Like on ServerA, I see only the small VCM drive/database....

To summarize, from ServerA and ServerB, if the EMC system admin assigned the drives that originally was created for ServerB, I can see any/all of these drives, on either server. However, on ServerA and ServerB, if the EMC system admin assigns the new drives that were created for ServerA, I can *NOT* see these drives. I seen only the small VCM database/drive....!!!???

The EMC guy is guessing that the PORT numbers assigned to the new drives (intended for ServerA) might be the culprit...?

Any ideas...?

B.t.w., yes -- as far as I can tell, both servers are the same. They use the same hardware, the same fibre channel cards, the same OS, and the same patches are applied... :-)

Thank you,

Cameron

[Jeff Schussele]

Hi Cameron,

I suspect that they are indeed zoned incorrectly.
Rerun fcmsutil /dev/tdX to get the WWN & copy/paste/send it to the EMC-dude.
I suspect someone typo'd it.

My 2 cents,
Jeff

[Steven E. Protter · Accepted solution]

HP fiber cards with some arrays have an 8 lun limit. That should not be a problem with EMC

Have you booted since drive assignment?

If not

insf -e

Have the SAN guy verify the LUN assignment on the array.

Next question is do the fiber cards work right?

fcmsutil /dev/td0

Makes sure the card is working and the World Wide Name matches the settings on the SAN.

SEP

[cninham]

The fibre cards *must* be working fine -- because I can see the "other" drives that were originally assigned to the "other" server. Also, I can see the VCM drive/database from the EMC, just not the actual drives. This problem is the same on either server...

Yes, I have rebooted the servers since the drives were installed.

Here are the results from ioscan and fcmsutil:



# fcmsutil /dev/td1

Vendor ID is = 0x00103c
Device ID is = 0x001028
TL Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x000006
Previous Topology = UNINITIALIZED
Local N_Port_id is = 0x000000
Local Loop_id is = 126
N_Port Node World Wide Name = 0x50060b000009d559
N_Port Port World Wide Name = 0x50060b000009d558
Driver state = AWAITING_LINK_UP
Hardware Path is = 0/7/0/0
Number of Assisted IOs = 0
Number of Active Login Sessions = 0
Dino Present on Card = NO
Maximum Frame Size = 960
Driver Version = @(#) PATCH_11.11: libtd.a : Jun 28 2002, 11:08:35, PHSS_26799



# ioscan -funC disk
Class I H/W Path Driver S/W State H/W Type Description
========================================================================
disk 0 0/0/1/0.1.0 sdisk CLAIMED DEVICE HP DVD-ROM 305
/dev/dsk/c0t1d0 /dev/rdsk/c0t1d0
disk 1 0/0/1/1.0.0 sdisk CLAIMED DEVICE SEAGATE ST39204LC
/dev/dsk/c1t0d0 /dev/rdsk/c1t0d0
disk 2 0/0/1/1.2.0 sdisk CLAIMED DEVICE FUJITSU MAJ3182MC
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
disk 3 0/0/2/0.0.0 sdisk CLAIMED DEVICE SEAGATE ST39204LC
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
disk 4 0/0/2/0.2.0 sdisk CLAIMED DEVICE FUJITSU MAJ3182MC
/dev/dsk/c2t2d0 /dev/rdsk/c2t2d0
disk 5 0/3/0/0.97.27.255.1.3.0 sdisk CLAIMED DEVICE EMC SYMMETRIX
/dev/dsk/c4t3d0 /dev/rdsk/c4t3d0
#

[cninham]

Sorry -- wrong fibre card driver... Here is the correct one:


# fcmsutil /dev/td0

Vendor ID is = 0x00103c
Device ID is = 0x001028
TL Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x000006
Topology = PTTOPT_FABRIC
Local N_Port_id is = 0x611c13
N_Port Node World Wide Name = 0x50060b00000a0be5
N_Port Port World Wide Name = 0x50060b00000a0be4
Driver state = ONLINE
Hardware Path is = 0/3/0/0
Number of Assisted IOs = 45
Number of Active Login Sessions = 0
Dino Present on Card = NO
Maximum Frame Size = 960
Driver Version = @(#) PATCH_11.11: libtd.a : Jun 28 2002, 11:08:35, PHSS_26799

[James A. Donovan]

Could you run the symcfg command, and let us know if the following director port parameters are enabled?

symcfg -SA -p -v list|grep Enabled
Tagged_Commands : Enabled
Common_Serial_Number : Enabled
Disk_Array : Enabled
Volume_Set_Addressing : Enabled
Global_3rdParty_Logout : Enabled
Init_Point_to_Point : Enabled
Unique_WWN : Enabled

[Tim Sanko]

Cameron,

I saw a similar problem when I had 2 rp5470's in the same rack do that.

Apparently in EMC's ECC, one of my compatriots had zoned things incorrectly. We put Solutions enabler/timefinder and PP as well as other EMC software on the server.
Then we mis toggled a front switch on the symm. (Actually I found the switch wasn't turned on.)

See if you can install the ECC agent and try to load Solutions enabler if you have it to do a syminq. Then if you have loaded PP you can do a powermt config, and then a powermt display dev=all.

You may find a dead link to the switch fron the SYMM, or else just mis-configuration. I would suspect a dead link or the switch on the front of the SYMM not being on... It may just be that simple. Check the front panel...

Tim

[cninham]

Jim & Tim (hey, that rhymes :-)

I do not have the EMC software installed, so I will have to get that from someone/somewhere...

I assume 'symcfg' is an EMC [software] related command? I did not find 'symcfg' on the server, nor am I familiar with it.

I will update you when the EMC software is available/installed.

Cameron

[James A. Donovan]

symcfg is one of the many very useful command line utilities available with the Solutions Enabler package. Usually the binaries (well the symlinks to the binaries, anyway) can be found under /usr/symcli/bin

[cninham]

Steven / SEP,

You are onto something! When the LUN numbers were changed to 3 & 4, the server could see the EMC drives just fine! Is there a way around the 8 LUM limitation...??? Obviously we have MANY HP servers, all using the HP A5158 fibre channel cards. We are a services & hosting provider company. We have many other clients using the EMC storage solution. We seperate all clients and assign each client to a seperate port (or more if required). All of *OUR* [inhouse] servers need to be on the same EMC port. This would imply that for our HP servers, using the HP A5158 cards, we are LIMITED to the first 8 LUN's...!!!??? Whichever servers were assigned with the first 8 LUN's would be the only ones whom can 'see' their EMC drives...?

Is there a known workaround past the 8 LUN limitation?

Thank you,

Cameron

[James A. Donovan]

Each target, "t", can only see up to 8 LUNs, "d" on an HP-UX system. You can assing up to 16 targets per controller for a total of 128 devices per controller.

[cninham]

Is the 8 LUN limitation due to HP-UX or the HP A5158 hba/fibre channel cards...??? If we were to get a different brand of hba cards, will we still see the 8 LUN limitation?

Thanks,

Cameron

[Stuart Abramson_2]

The number scheme is:

...vbus 0 - 15
...target 0 - 15
...lun 0 - 7

So the maximumum number of devices is:

...15 x 15 x 7 = 1575
...(minus 1, because v0t0l0 is the VCMDB.)

The limit of HyperVolumes per FA port is 256.

So, you don't need to buy any other hardware or software.

You should learn how to us "symconfigure" to set vbus = 1, 2, 3, 4, etc.. On our systems you can't use ECC/SDR to set vbus.

Your EMC guy should have know about LUN limitations in HP-UX; and vbus/target/lun numbering. Complain to your EMC salesperson.

[Stuart Abramson_2]

The limit is 16 x 16 x 8 ! Sorry! But, it's really 256/FA, so no harm done.

[Todd McDaniel_1]

Cameron,

Have you run symcfg discover? that sometimes needs to be run on a host that is newly setup. then rerun ioscan and insf -e.

Also, I have the 5158 cards on many of my boxes and I have several hundred LUNs on many of them. I dont believe the lun limitation applies to EMC frames. My boxes are N-class/Keystone/Superdome systems.


You didnt say if these are direct-connect or go thru a switch? If it is going thru a switch, do you have path validated on the switch that the ports are active? I would telnet to the switch and check the port assignments.