Booting I64 from SAN lun (EMC)

[James T Horn]

I have a rx6600 with HBA's, connected to a EMC Clarion CX4-960.
We setup a SAN drive, did the initial installation of OpenVMS 8.3 on to the SAN drive, it setup the "boot options", but when I try to boot the system from one of the entries it comes up with:
Loading.: OpenVMS on $1$DGA3: PGB0.5006-0169-3B20-131D
Load of OpenVMS on $1$DGA3: PGB0.5006-0169-3B20-131D failed: Not Found

I know with AlphaServer we use the command wwidmgr... command to setup the boot device.
===
Entry Description Options
----- ---------------------------------------------------------- -------------
1 OpenVMS on $1$DGA3: PGB0.5006-0169-3B20-131D
$1$DGA3 PCI(0|c5|0|0) Fibre(500601693B20131D,Lun2000000000000)

[Hoff · Accepted solution]

http://labs.hoffmanlabs.com/node/786

[The Brit]

Hi James,
I am running OpenVMS 8.3-1H1, on Blades, booting from EVA and XP storage. I know nothing about EMC storage, however I have seen this failure before using EVA storage. (I have a suspicion about why this occurs, but I am not sure so I won't muddy the water), however;

If, as I assume, your luns are multi-pathed, then there should be more than one entry for each lun in the list of boot options displayed in the boot menu. In my case, with my configuration, I get 8 paths for each lun.

When I look at my boot menu, I see all 8 boot paths. If this failure occurs, I simply move to the next option and try that.
It will usually boot from either the first or second option.

I suspect that it is due to the path to the LUN not being available through the disk controller with the specified WWN. I also suspect that with my EVA storage, this is related to the fact that the controllers are NOT Active/Active.

HTH

Dave.

[James T Horn]

When following the instructions in the section titled "For OpenVMS I64", doing the commands:
vms_bcfg boot add 1 dga3 -flags 0,0 "$1$DGA3 OpenVMS 8.3"

returns:
vms_bcfg: Could not find VMS Device dga3 )/usb(0,0)

I also tried the $1$dga3 for the device name with still no success. Using "vms_show device" does not list the Fibre Channel drives.
Using SYS$MANAGER:BOOT_OPTIONS and during installing of OpenVMS on the SAN drive worked fine, setup the boot entries, but where wwidmgr sets up the AlphaServer with the Fibre Channel device, I'm still not seeing the devices listed, but when the system is up it has the SAN Lun's

[The Brit]

James,
If you select the EFI Shell option from the boot menu, it should display the device map, and if your SAN volumes are seen they should appear something like

fs2 : Acpi(HWP0002,500)/Pci(2|0)/Fibre(WWN21000020375AE714,Lun0)/HD(Part1,Si
gF7D00000)
fs3 : Acpi(HWP0002,500)/Pci(2|0)/Fibre(WWN21000020375A5E5B,Lun0)/HD(Part1,Si
g7D930000)

(the "fs" number will probably be different), but you should see the "/Fibre" in there somewhere, followed by a WWN and an LUN #.

you could also try issuing a "reconnect -r", to rescan the busses (although this is part of the normal power-up.)

Dave.

[James T Horn]

The EFI SHell does not show any fibre device listed.
"reconnect -r" does not even show the drives.

[Hoff]

If you've followed the sequence listed at the URL I posted and if you're (still) not seeing the SAN drives, then it's time to have a conversation with the folks that are maintaining and configuring the SAN. If that's (also) you, then you'll best be discussing this (lack of) visibility with EMC support.

[Toine_1]

Hi James,

You should set EFI Variable EFIFCScanLevel to 1 via the EFI console to scan all the devices in the SAN and to display all available active paths on the Fibre Channel.

Reinit you system so it scan the SAN.

See: http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1266530397489+28353475&threadId=1140093

or

http://www.dectrader.com/docs/set3/ucr_na-CYBBOOTING_FC_164.pdf-1.pdf

/Toine

[cnb]

Hi James,

Just curious as to which model/part number HBA and FC switch you're using?

Have you tried EMC's E-lab Interoperability Navigator to make sure you're meeting their minimum requirements?

Rgds,

[cnb]

FYI:

Minimum Flare code level required for external boot is 04.28.000.5.003

Rgds,

[The Brit]

James,

Only one thing to be aware of. If you set EFIFCScanLevel=1, then it should return ALL FC devices which are presented to your system. If there are a large number of LUNs presented, the scan can take a very long time. I had inadvertently set this value on one of my blades, and it was taking 5-10 minutes for the scan to complete.

When the scan is complete, you will see that most of the devices appear as "blkn" devices. These are not really of much use to you at the EFI level. Take a look at the device map and check for "fsn" devices (normally listed first). These normally represent "bootable" volumes/devices.

If it is appropriate for your configuration, the best value for EFIFCScanLevel is "0", since this should return only "bootable" volumes. After resolving your LUN visibility issue, you might want to consider resetting the parameter to this value.

(Note: fs: = "bootable" is not an official description, just a generalization based on observation)

Dave.

[Hoff]

An fs device is a disk partition containing a FAT file system.

A blk device is a disk partition.

Some fs partitions are bootable. Some are not.

Some blk partitions contain FAT file systems, and some are "protective" and one, two, three (of two, three or four total partitions on an OpenVMS disk) are present prevent an errant console command from clobbering an OpenVMS disk.

The maintenance partition is a non-bootable fs device.

fs and blk devices are somewhat randomized, and the order of appearance is not entirely determinate. This is where the GUIDs and a few other factors are used to "fingerprint" and follow and identify the desired partition.

EFI has substantial UI issues here, yes, and that OpenVMS doesn't support disk partitioning makes this particularly messy.

Low-level details here:

http://labs.hoffmanlabs.com/node/28

And FWIW, the "flare" stuff mentioned earlier in this thread is more technically known as the "EMC Clariion FLARE Code Operating Environment"; it's the controller firmware, and an analog to the CRONIC operating system resident on various of the older HS-class controllers.

[Peter Zeiszler]

Do you have other systems in a cluster using that same disk?
If so - you may have an A and B path (assuming 2 paths) to the disk inside the EMC. Do you have other paths to the disk? Can you force the other systems to use the same path that you are trying to boot from?

We have this same issue on bootup when mounting devices. Have to set the devices on ALL nodes to use the same path for accessing or the disk wouldn't show up.

[James T Horn]

This system is not part of a cluster. It is a AD299A

I've tried doing:
vms_bcfg boot fibre 1
Reconnect -r
map -r

and the fibre cards still are not showing up.

I've applicated the Update 7.0 patch set
I've got the correct firmware:
System Firmware: 4.11 [4842]
BMC Version: 5.25
MP Version: F.02.23


Dispvpd all
VPD Info for Card [0]: AD299A

VPD Version: 3
Product ID: FE00
Product Name: AD299A
Part Number: AD299-60001
Eng Date Code: A-4717
Serial Number: JP48390895
Misc Info: PCIe 2.5Gb/s, x4
Mfg Date Code: 4839
Checksum: F1
EFI Version: ZE3.21A3
Asset Tag: NA
ROM Version: ZS2.70X5
WWN: 50060B0000FD7392
VSD1: 103C

VPD Info for Card [3]: AD299A

VPD Version: 3
Product ID: FE00
Product Name: AD299A
Part Number: AD299-60001
Eng Date Code: A-4717
Serial Number: JP48390526
Misc Info: PCIe 2.5Gb/s, x4
Mfg Date Code: 4839
Checksum: F9
EFI Version: ZE3.21A3
Asset Tag: NA
ROM Version: ZS2.70X5
WWN: 50060B0000FD7366
VSD1: 103C

Device mapping table
fs0 : Acpi(HWP0002,PNP0A03,0)/Pci(2|1)/Usb(0, 0)/CDROM(Entry0)
fs1 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part1,Sig4CC7D671-BD4D-11DE-81D4-AA000400FFFF)
fs2 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part3,Sig4CC7D670-BD4D-11DE-81D5-AA000400FFFF)
blk0 : Acpi(HWP0002,PNP0A03,0)/Pci(2|0)/Usb(1, 0)
blk1 : Acpi(HWP0002,PNP0A03,0)/Pci(2|1)/Usb(0, 0)
blk2 : Acpi(HWP0002,PNP0A03,0)/Pci(2|1)/Usb(0, 0)/CDROM(Entry0)
blk3 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)
blk4 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(00)/Scsi(Pun0,Lun0)/HD(Part1,Sig4CC7D671-BD4D-11DE-81D4-AA000400FFFF)
blk5 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part2,Sig4CC7D670-BD4D-11DE-81D3-AA000400FFFF)
blk6 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part3,Sig4CC7D670-BD4D-11DE-81D5-AA000400FFFF)
blk7 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part4,Sig4CC7D670-BD4D-11DE-81D4-AA000400FFFF)
blk8 : Acpi(HPQ0002,PNP0A08,600)/Pci(0|0)/Pci(0|0)/Pci(4|0)/Pci(0|0)/Scsi(Pun0,Lun0)/HD(Part5,Sig4CC7D671-BD4D-11DE-81D3-AA000400FFFF)

[cnb]

I'm confused (ok, nothing new about that ;-)), if the HBAs aren't showing up then how did it see them to begin with? Did something get moved around in hardware after the installation?

EMC docs indicate that the rx6600/AD299A confguration appears to be supported with CX4-960 but there are *many* specific requirements for external boot. If the cards aren't showing up then we haven't even got to any external boot configuration issues (yet ;-)).

Have you tried using efiutil to see if the HBAs are showing up there?

fs0:\> efiutil info
Fibre Channel Card Efi Utility 1.20 (1/30/2003)
2 Fibre Channel Adapters found:
Adapter Path WWN Driver (Firmware)
A0 Acpi(000222F0,200)/Pci(1|0) 50060B00001CF2DC 1.36 (3.02.170)
A1 Acpi(000222F0,200)/Pci(1|1) 50060B00001CF2DE 1.36 (3.02.170)

You don't have the exact hardware configuration listed but this document may lend some ideas (although it's outdated for V8.3) about the rx6600.

http://docs.hp.com/en/5991-8644/5991-8644.pdf

Rgds,

[James T Horn]

Thank you all for your assistance in trying to get this working. After talking with HP today, they are replacing the EMC cards with Qlogic AB378B cards.