First time boot of second IA64 in Cluster

[Heinz W Genhart]

Hi community

I have a mixed cluster with two Alpha Server GS1280 (VMS 8.3) and currently one rx6600 with VMS 8.3-1H1. The IA64 systemdisk is a shadowset with one member, it’s a SAN disk on a XP24000.
We are planning now to add another rx6600 to this cluster and there is a point where I’m not sure that it works. It’s a production system and so there is no space for trying something.

On the second IA64 I do not have a boot entry for the FC-systemdisk. I can boot the DVD and finally I can use boot_options to create the required boot entry. My concern is the following:
Boot_options will mount the destination Systemdisk. But this disk is in a shadowset and currently mounted on the two GS1280 and on the rx6600. Can I mount the physical Disk (belonging to the IA64 Systemdisk shadowset) with boot_options without to have a problem? Is there no danger, that the IA64 systemdisk is corrupt after doing this?

Until now I had the following workaround: I shut down the running IA64 System and then I dismounted the IA64 Systemdisk on every node in the cluster. After this I booted the new IA64 from DVD and used Boot_options to create the required boot entries. But now I can’t shutdown the IA64 System because it’s a production system.

Thanks for your answers in advance

Geni

[Robert Gezelter]

Geni,

Have you carefully read "Configuring and Managing OpenVMS Booting on Integrity Servers" (http://h71000.www7.hp.com/doc/83final/ba322_90045/apbs05.html#cjhigcgi )?

In addition to the footnote on FC devices, it strongly suggests using SYS$MANAGER:BOOT_OPTIONS.COM, rather than mucking in EFI.

If I understand your question, PERHAPS the correct answer to your situation is to add the necessary system root to the I64 system disk, add a "persona" for the second machine to the BOOT_OPTIONS while running BOOT_OPTIONS.COM on the FIRST machine. Then boot the second machine into the cluster.

A key difference between Alpha/VAX and Itanium is the presence of the EFI partition on Itanium. This means certain information is stored in that disk partition (which is shadowed in your case).

Regrettably, I do not have the right in-house configuration at this instant to verify all of the above.

- Bob Gezelter, http://www.rlgsc.com

[Heinz W Genhart]

Hi Bob

thanks for the fast response.

I installed already several IA64 Systems into an existing Alpha Cluster.
After using CLUSTER_CONFIG to add a root for a second IA64 machine, I never had a boot entry for the second machine.
So the only way I see is, I have to boot the DVD on the second machine to finally add a boot option.
My problem is:
Can I boot the DVD on the second machine and use boot_options on the DVD to add a boot entry on a systemdisk, as long as this systemdisk is mounted to an existing and running IA64.

I can only assume the following two scenarios:

1.) Boot_options is accessing the SYS$EFI.SYS partition (over the GPT) to add a boot entry for the second machine and nothing else happens (I could not find confirmation for this, but I guess this is the way to do it)

or

2.) I have a corrupted systemdisk after using boot_options from the DVD, because the first and the second IA64 dont know each other and are not using the DLM

Can somebody confirm that may assumption in 1.) is right?


Regards

Geni

[Robert Gezelter]

Geni,

What I meant was running SYS$MANAGER:BOOT_OPTIONS.COM (see the citation in my first post in this thread).

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

If it's a production system, then why are you practicing on it? Get or rent or borrow a test configuration or work with HP to ensure the integration.

From BOOT_OPTIONS you should be OK. OpenVMS Alpha V8.* and OpenVMS I64 and the related disk structures are all cross-compatible. Using OpenVMS VAX BACKUP and other such won't be quite so transparent (eg: don't use those tools here with a GPT disk), but you're probably not going to deal with that.

http://64.223.189.234/node/112

Basically, BOOT_OPTIONS tosses some commands out at the EFI environment and since the disk is mounted by OpenVMS and unrelated to whether shadowing and clustering are all active, even if BOOT_OPTIONS does I/O into the partition, all users and systems remain happy. And consistent. Remember too here that you're not writing to the EFI partition when you're setting up boot aliases. (You can experiment here and prove that to yourself with the DIFFERENCES command, since this is production. The file SYS$EFI.SYS is the host view of the boot partition.)

If accessing and updating the boot partition within a a shadowset from the EFI console, you need to drop to one member. That's because EFI and multiple-volume shadowsets don't coordinate write I/O activities.

And if you're so inclined, you can do this from the EFI console, too:

http://64.223.189.234/node/786

Particularly if you're going to "experiment" in a production environment, I'd probably bring a second OpenVMS I64 system disk on-line here regardless. This particularly so you have options for upgrading and for on-going operations and for rolling your changes and patches into production.

Will I commit to this sequence working? No.

I'd run through the sequence as an experiment on your test cluster, as it's feasible to derail a production cluster even if everything is entirely and technically correct.

And get somebody in for support and escalation and troubleshooting here, whether that's HP or otherwise.

[Colin Butcher · Accepted solution]

Hi Geni,

There's a nice description of the resulting confusion here: http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1311828

You end up building a new system disc.

Well done for thinking it through up front.

What you need to do is boot the new cluster member node from another disc as a stand-alone node (eg: install DVD, copy of install DVD on local SAS drive, spare bootable SAN device, whatever).

Then mount the target cluster-common system disc containing the new member's system root /READ_ONLY/NOCACHE to let BOOT_OPTIONS see the disc, but not to write to it. All BOOT_OPTIONS needs is read access so as to find the target disc GUID and so on.

Having now used BOOT_OPTIONS to set the target disc, SAN paths, OS_FLAGS etc., re-ordered the boot options (if needed), then dismount the target system disc and shut down the new member which is temporarily booted from a stand-alone system disc.

Now boot from the new boot path / system root (which is now available to you in the EFI boot menu) and you're done.

A few suggestions for production clusters nin a mission-critical environment:

- never leave root [SYS0] on a cluster-common system disc. This prevents a node being able to boot 'by default' - you have to set the OS_FLAGS.

- with IA64, set the EFI up to leave the machine off on power-up after power-fail. So you're in control of bringing the machine back up.

- with IA64, set the boot options so that the first option is "exit to EFI". So, if anyone pounds on the , it can't boot. Again, you're in control of bringing it back up.

Cheers, Colin (http://www.xdelta.co.uk).

[R. Verkerk]

Hi,

We use following procedure:

- Add a new root for a new system
- The new system is powered on
- in the EFI prompt (shell>) we enable fibre scanning. either via drvcfg or using an OpenVMS DVD mounted via virtual media.
- we re-init the system.
- we enter the EFI (shell>) prompt again. Now we see the boottable disks.
- we go to the correct disk and directory:
fs0:
FS0:> cd efi\vms
- we boot from the correct root
vms_loader -fl 1,0 !! boot from root 1
- When vms is up we use boot_options to add a boot entry.

gr,

Robert Verkerk

[Heinz W Genhart]

Hi Colin and Robert

both of you give me a very useful answer. Thanks a lot.

Both methods are working well. I prefer the method decribed by Robert, because I dont have to boot the DVD and so, thats faster.

1. Enabling Fibre Scanning
2. Reset the system
3. With map we can see all bootable disks
4. And then using vms_loader.efi

Best Regards

Geni

[Colin Butcher]

I use Robert's method too, but it can require some further work to configure the adapters for fibre scanning in large systems / SANs. This is useful information: http://www.docs.hp.com/en/AD299-9001A/ch01s12.html

The 'boot from an alternate disc' method requires less poking around at EFI level.

The 'boot from an alternate disc' method is also useful to find out what target devices there are in a VMS context, then you can use that information to set up the HBA fibre scanning.

Another useful tool at VMS level is SYS$ETC:FIBRE_SCAN.EXE

Cheers, Colin (http://www.xdelta.co.uk).

[Heinz W Genhart]

Hi all
I could solve now the problem I had, with first time booting of second IA64 into a cluster.
I got very useful information from you. I thought that there must be a possibility to boot a further IA64 machine into a production cluster without to take down the whole cluster.
During the Itanium Migration I did the last few weeks, with 4 test-clusters and two production clusters, I encountered another problem with the quorum Disk. I will open a new discussion for this.

Thanks a lot
Regards
Geni