SAN DISK Identification process

[Robert Jacobs]

Hi All


I have been working with various SAN disks and technologies during the last three months as my forum history my show. I have what I hope is an easy question.

Is the SYSMAN command

SYSMAN> io auto

suposed to return newly discovered SAN disks in the manner that the INIT command does at the CONSOLE Prompt >>>


This has not worked for me in the past. My associate wants to know if there is a way we can discover new SAN attached disks WITHOUT having to bring down the system and do an INIT.

He indicates a command in Tru-64 Unix called

#hwmgr scan scsi

discovers on the fly the new SAN attached storage disks.

#hwmgr show scsi

will verify the storage is there.


So is there an equivalent command when OpenVMS system is up and running.



Thank you

Robert J.

[Andy Bustamante]

I'm not sure what your problem is. wwidmgr is used to define a disk for boot or writing a dump file (dump off system disk). Once VMS has booted you use SYSMAN to detect any new disks added to the SAN without downtime.

If you dismount and delete a disk you can use
SYSMAN> IO SCSI_PATH_VERIFY
SYSMAN> IO AUTOCONFIGURE

The first command verifies SCSI and fibre channel devices and disconnects any which have changed. The second detects new devices.


Andy

[Robert Jacobs]

Thank you for your response

When I issue the sysman commands

SYSMAN> IO SCSI_PATH_VERIFY

or > IO FIND_WWID

or > IO AUTO

The system just does not pick up the SAN device on the fly!.


When I do an INIT at the console >>> prompt then things are fine! So I guess the issues is why does the system fail to get these devices on the fly!! So does anybody have an idea why certain scaning would not work until an


A side note:

I view the UDID # via the ww -sh ww.

Numbers within the SAN storage product I use get translated to for example $1$dga12 if I set a # 12 within the SAN storage utility.

[Uwe Zessin · Accepted solution]

Can you provide some background info:
- what storage array is involved?
- version of OpenVMS?
- output of ">>> wwidmgr -show wwid -full" attached as a .TXT file?

[Andy Bustamante]

Use
SYSMAN> IO SCSI_PATH_VERIFY
to remove any deleted units and validate connections.

SYSMAN> IO AUTOCONFIGURE will detect new units.

If this fails at the operating system level and works at the console, I'd suspect a hardware issue.

Your original post states "various SAN disks and technologies" can you be more specific? What HBAs are you using and have you loaded recent firmware? What version of VMS and what patches are installed?

Andy

[Mike Reznak]

Hi,

use SYSMAN> IO AUTO/ LOG to see whats going on.
And please take time to answer Uwe questins. It could be helpfull.

Mike

[Mike Reznak]

Sorry for typing mistake

SYSMAN> IO AUTO /LOG

[Robert Jacobs]

Hi


We are running V7.3-2 with Firmware

AlphaServer Console V6.9-1, built on Nov 18 2004 at 09:46:43

The patches and ww output are in an attached MS-word doc file.

The storage we are using is IBM DS6000/DS8000 with SVC (TotalStorage SAN Volume Controller

Thank you for looking into this issue.

Robert J.

[Peter Zeiszler]

Since you post that you have worked with SAN environments for the past few months all of this should already be second knowledge to you.

In our environment we use HP XP1024 SAN environments and EMCs. So for the VMS it really doesn't matter the frame - as long as they remember to setup an unused LUN 0 (might be part of the XP1024 requirement for servicing VMS).

I will try to explain as best as I can some of these numbers.

"[16] UDID: 1 WWID:01000010:6005-0768-0183-8032-6000-0000-0000-004e"

The ITEM is 16
UDID is 1
The WWID contains your WWN (world wide number) of the device accessing the disks. So its one of your Fiber cards.
The very last part 004e should actually map to the CU:LDEV. This is very important for the SAN configuration.

On the VMS system the very most important number is the UDID. This will make the device show up as DGA1:

It appears that you have already gone through the steps of:
set mode diag
wwidmgr -show wwid
wwidmgr -set wwid -item (#) -unit (###)
(example: wwidmgr -set wwid -item 16 -unit 1)

Now at the boot prompt you should see devices setup in the wwid and n# locations. These are limited to only 4 devices. So the most important ones to setup in there are the system disk and the dump device. Those need to be understood from the boot prompt.

As we add new devices I do not have to set the UDID on each device. By default the devices will inherit the defined LUN. On the XP1024 however the LUN means nothing. Its all in the CU:LDEV.

The CU:LDEV is converted to a number that is presented to the VMS. The CU is multipled by 256 and then added to the LDEV. We also have multiple SAN frames accessible from each VMS system. These numbers need to be unique when presented to the VMS.

So using 004E we would see as a device DGA78. We could NOT see any other devices with the same number from other frames or if we forced a device to a UDID.

When we add disks we always verify that information prior to adding it to the VMS group.

Once the device is added all we have to do is force a scan of the devices:
mc sysman io auto
The new DGA devices are then visible to us.

[Robert Jacobs]

Yes Peter this is great information.


In my case The SYSMAN IO auto or any SYSMAN commands DO NOT return the expected results.

I am NOT seeing the devices on the fly.
without bringing down the system, When I issue the command $ show dev d

That is the problem I have.

I was hoping to get to this root cause.

Why is this happening.

I can issue an INIT at the Console >>> prompt and then I do see the new devices I just presented to the SAN.

Any thoughts

Thank you Robert J.

[Peter Zeiszler]

The devices are unique UDID? Meaning that one of the devices you manually set the UDID does not correspond to the new devices being added if their CU:LDEV was converted?

That is the only thing I can think of that I know is an issue.

[David Huber_1]

Did you ever finish working through OpenVMS operating with SVC? We are about to embark on that journey...

thanks
david

[Jon Pinkley]

I always do this:

$ mc sysman io scsi
$ mc sysman io auto/log

If I don't do the "io scsi_verify" before, sometimes it does not pick up the new devices. However, if I do, then as long as I have remembered to present a non-zero O/S unit id, then the above sequence works.

Did you do both (even though you didn't unpresent any devices)?

This was with EVA6000 with latest (as of Feb, 2007) firmware/CV software, and ES47 with VMS 8.3 Factory Installed AVMS83R1.

Jon

[Anton van Ruitenbeek]

Robert,

Maybe a question you already know, but are all the areas on the switches open for the OpenVMS system to see the SAN ?
It looks like there is a breach somewere.
Are connections switched, are groups on the switch changed, is the systemname still the systemname (for the switch) ?
Are on the SAN the securitys correct ?

AvR

[William Brown_2]

I'm late to this discussion but am having a similar issue when adding new vdisks to a 4 node VMS 7.3-2 cluster. I have found that the LUN on the SVC numbered 0 - 15 are visible to VMS with SYSMAN IO AUTO. LUNs 16 and above are not. One of our node had vdisks that the others did not so it stopped seeing new vdisks before the others. I finally tracked it down, so it seems that VMS is seeing the SVC LUNs and one SCSI channel and is limited by that. We have the question into our IBM vendor.

Our 'old' storage is HP EMA12000 and each shelf of 14 drives is seen as a different SCSI channel. I wonder how EVA handles this as that was the second choice to the SVC based storage (DS4800 controllers).

Thanks,

Bill Brown
St Jude Children's Res Hospital
Memphis

[Uwe Zessin]

> I wonder how EVA handles this

The EVA has a Fibre Channel Arbitrated Loop (FC_AL) back-end.

Every disk drive is dual-ported and connects to a loop pair. The EVA5000 + 8000 have two loop pairs - all others have one loop pair.

Depending on the size and age of the model, there is a 'loop switch' in the back-end so that the controllers and disk enclosures are connected in a 'star topology'.

The latest models even use small 'cut-through switches' (CTS) in their disk enclosures.


But this is nothing what the server sees. At the front-end, the EVA looks much like a HSG in SCSI-3 mode (controller device at LUN address 0), except that today's models run in (asymetrical) active/active mode.

[William Brown_2]

I did a fiber scan and it looks as if our VMS system is seeing several SCSI 'targets' which I think is the SCSI channel. LUNs 0-16 are the same on each of these targets. Thus we only see the first 16 vdisks presented to a node. We are in a 4 node cluster and one has several disks the others do not...each only sees the first 16 mapped to it. I'm attaching FC scan results.

Thanks,

Bill

[marsh_1]

try running this

MCR SYS$ETC:FC$CP FGA 2 1 8

worked for me with ds20 vms v7.3-1 , v7 firmware and ibm svc and ds8000.

[marsh_1]

sorry then run another sysman io a

[William Brown_2]

Mark...re:

try running this

MCR SYS$ETC:FC$CP FGA 2 1 8

worked for me with ds20 vms v7.3-1 , v7 firmware and ibm svc and ds8000.

Is that 2 space 1 space 8 ? Did not show up ver well in the answer on the forum. Looks more like that now since I pasted it in.

BTW, I booted one of our DR nodes in this development cluster Saturday night. At boot (probably after the INIT) and coming back up on VMS the system saw the disks past 0 - 15 LUN...but after that I still could not add more disks with IO Auto.

I then went to another single node cluster system on OpenVMS 8.3 and added LUN 23 without problem. Scanning the FC shows LUN 23 on this 8.3 system where the 7.3-2 system still tops out showing 0-15 even though additional disks are recognized above that after a boot.

We will be upgrading our systems to 8.3 over the next couple of months, but want to have the SVC storage in first :-).

Thanks all

[marsh_1]

just run the exe with no params and it will show you that example.

[Torsten Rothenwaldt]

Hello William, I'm from IBM, and I hope that you already received the following explanation from our support team.

As we learned from tests, there are some prerequisites for identifying a new Fibre Channel disk in OpenVMS:

1. There must be a LUN 0 available from this FC target. If there is no LUN 0 mapped to the server, no other LUNs are scanned, and you don't see any disks. (This is not unusual in "FC world".)

2. OpenVMS at least up to V7.3-2 expects that LUN 0 presents itself as a storage controller (with a Peripheral Qualifier/Device Type field set to '0x0C'). This is the well-known Command Console LUN (CCL), shown as device $1$GGAx in OpenVMS. If this is fulfilled, you can add any LUN dynamically (without reboot) to OpenVMS using MCR SYSMAN IO AUTOCONFIGURE. If this is not fulfilled, you can add LUNs only up to LUN Id 15 dynamically. Adding higher LUNs requires reboot.

3. With some version V8.x (we tested v8.3, bu it might be available earlier), requirement 2. has been removed. Now LUN 0 may be a CCL or a normal disk (with a Peripheral Qualifier/Device Type field set to '0x00'). Probably this has been made to add/improve support of the HP XP storage arrays. Now you can add disks up to LUN 255 dynamically.

HP HSG80, HP EVA, and IBM DS8000 present LUN 0 as CCL. IBM SVC presents LUN 0 with Peripheral Qualifier/Device Type '0x00' --- now you understand why you don't see a $1$GGA device with SVC and what happened with your disks.

Conclusions:

* Never unmap the LUN 0 of your SVC. If you do this (because you don't need this volume anymore), you will not see any other impact immediately (beside that this disk is no longer accessible, of course). But when the system reboots, no disks will be seen. So after each change you may double-check that there is still a LUN 0 mapped to each OpenVMS server.

* If you have v7.3-2, adding SVC disks beyond LUN 15 requires a reboot, as you wrote. There is a workaround: You can "jump" across the LUN 15 border dynamically with adding multiple disks as long as you are below LUN 15. Example: You have already 15 disks (LUN 0-14). Now add two disks (LUNs 15+16) in a single step! Because of some oddity in OpenVMS drivers, both disks can be added dynamically. And now you can add even more disks beyond LUN 15 dynamically.

* Upgrade to V8.3, and these issues should disappear (except that OpenVMS still needs a LUN 0).

I hope this helps.

[William Brown_2]

Torsten,

Thank you very much for your explanation. We were able to come to this conclusion about 10 days ago after extensive testing here at our site. We are in the process of upgrading to OpenVMS 8.3 which will allow us to use the SVC based storage completely as we want.

If we had gotten your answer a month ago it would have helped tremendously. IBM support response was they could map and mount more that 16 SVC disk on both VMS 7.3-2 and 8.2-1, LUN 0 must exist and maybe it is a firmware level issue. This was from Level 3 in the UK. Your response should be added as a footnote to the SVC Host Attachment guide as it would clear things up considerably. (Or at least say upgrade to VMS 8.3 and be happy :-)).

I'll share your answer with our technical team.

Good day,

Bill