OVMS 7.2.1 cluster with RA3000

[Markus Waldorf_1]

Thanks. According to this the tri-link and Y-cable, both are used to put a device or controller in the middle of the bus, but only the y-cable add the term "host adapter" to it. So I don't understand, if I can use the tri-link connector to put a controller in the middle, why not a host adapter?

I was hoping to find another card which has all termination resistors, but I found only KZPBA-CY, which does not use them. From what I can gather the KZPBA-CY is differential scsi too, but obviously different. Can the KZPBA-CY be used to replace a KZPBA-CB. How is termination controlled on the CY?

[Markus Waldorf_1]

The KZPBA-CB comes with a configration utility floppy "eeromcfg.exe" which can be run from the ARC console. I'm missing the floppy disk. I cannot find a download. Someone has a copy?

[cnb]

Markus,

As long as the termination is configured and cabled correctly, you should be able to use these controllers in the middle of the chain. I've always seen these with the Y cables in this configuration though.

Correct cabling and termination are paramount!

ftp://ftp.compaq.com/pub/products/storageworks/techdoc/raidstorage/EK-SMCPO-UG-C01.pdf

FYI:

According to an old Cybrary disk I found, there appears to be several KZPBA controller versions:

KZPBA-CA PCI to UltraSCSI SE
KZPBA-CB/CC PCI to UltraSCSI Differential
(the -CC was reworked for 3.3v).
PCI TO SCSI (FAST20 DIFFERENTIAL) Adapter KZPBA-CY 401922-001
PCI TO ULTRASCSI Adapter, Single Chnl, FWSE KZPBA-CX
PCI to UltraSCSI Adapter, Dual Channel,UWD KZPBA-DY
PCI to SCSI Adapter, FWSE KZPBA-BA
PCI to SCSI Adapter,FWD KZPBA-BB


FWIW:

The CA/CX and CB/CY get mixed up all the time so make sure you have the correct KZPBA controller for your configuration and termination.

The -CX and -CY are part of the -CA and -CB *kits* respectively.

If I remember correctly, the CB/CY have the terminator sockets whereas the CA/CX do not.


hth,

[cnb]

Markus,

It is on the Alpha Firmware CD.

or here:

ftp://ftp.hp.com/pub/alphaserver/firmware/other/utility/

hth,

[Markus Waldorf_1]

Thanks a lot! I downloaded the utility - now I have to find a floppy disk ;-)

Looking closer at the card I see now that the one I found is indeed KZPBA-CX (not CY). I'm afraid now that this card could not be used, or?

[cnb]

Markus,

You'll need the -CY for this configuration that you're trying to set up.



Rgds,

[Markus Waldorf_1]

Yep. I searched a bit more info and the KZPBA-CA/CX are the ones without sockets for termination resistors. These cards are not even differencial SCSI, but SE Single Ended, unlike advertised at some places. They're not compatible with KZBPA-CB/CY. Mixing differential and SE can even cause damage. Also external Terminators and Personality modules in the BA35x shelves need to be diff SCSI too.

I'm sort of stuck now. The Tri-link H8861 adpaters don't work as BN21W Scsi-Y cable, it seems, and I cannot use them to interconnect the 2 nodes with the legs going to the controller hosts. In order to create a controller mid-bus configuration I need to terminate the nodes, but I'm missing one of the eight Termination Resistors in one card the KZPBA-CB cards. I wonder if this was actually was my initial problem.

Are those termination resistors standard or special. Could I use any other one that fits. Does it matter which way they go into the sockets - they have a mark, but nothing on the card or in the manual?

[cnb]

Markus,


The terminators have specifications(220/330?).

I can't recall what they are at the moment.
I'll see if I can locate some old specs.

This configuration guide may also help you:

http://vt100.net/mirror/mds-199909/cd3/storage/ultracgc.pdf

hth,

[Markus Waldorf_1]

Thanks again for all the info. It was quite some reading but I'm still not sure if the termination resistors need to go back into the KZPBA-CB card in a certain way - or maybe it doesn't matter.

I'm also not sure whether or not the HA8861 VHDCI Tri-link adapter could be used instead of a BN21W Scsi-Y cable - I have attached a BN38E HD68 to VHDCI adapter cable to the KZPBA-CB

One notes I found:
If you connect a trilink connector to a SCSI bus adapter, you may block access to an adjacent PCI slot. If this occurs, use a Y cable instead of the trilink connector. This is the case with the KZPSA SCSI adapters on some AlphaServer systems.

Another looks very promissing:
To be able to externally terminate a bus and connect and disconnect devices without affecting bus termination, remove the device termination and use Y cables or trilink connectors to connect a device to a shared SCSI bus. By attaching a Y cable or trilink connector to an unterminated device, you can locate the device in the middle or at the end of the shared bus. If the device is at the end of a bus, attach a terminator to the Y cable or trilink connector to terminate the bus. If you disconnect the Y cable or trilink connector from the device, the shared bus is still terminated and the shared SCSI bus is still operable.

In addition, you can attach a Y cable or a trilink connector to a shared bus without connecting the Y cable or trilink connector to a device. If you do this, you can connect a device to the Y cable or trilink connector at a later time without affecting bus termination. This allows you to expand your configuration without shutting down the cluster.

I have the following configuration now, but it does not work. None of the nodes see devices on the RA3000 (3 virutal devices).

node1 KZPBA-CB (no term) -> BN38E -> H8861
node2 KZPBA-CB (no term) -> BN38E -> H8861

The inner connectors of the H8861 trilink adpater connect the nodes together via BN37A

node1 (H8861) -> BN37A -> node2 (H8861)

The outer connectors of the H8861 connect to each of the controller host in:

node1 (ID7) -> BN37A -> RA3000 host0
node2 (ID6) -> BN37A -> RA3000 host1

Shouldn't this work?

[Markus Waldorf_1]

Today must be my lucky day!

I noticed that the 30 cm VHDCI cable I used to interconnect the 2 nodes via the H8861 trilink adapters actually is a no-name and not BA37A. I replaced it with a BN37A-05 and ... bingo it works now.

I'm able to see the all devices from node 1 and node 2 and I can also init and mount them from both nodes using the DCL prompt from the installation CD.

So the trilink adapter does work in place of the BN21W Scsi-Y cable, and it fits/looks even better than the Y-cable. Guess I really had bad luck with a wrong or bad VHDCI cable and a missing terminator resistor on one of the KZPBA-CB adapters.

While checking if I can mount the system disk on the other node I received error %PMA0 CPU0 Remote System Conflicts with Known System - Remote Node. The Memory Channel adapter is also working. I also confirmed it by running "mc_cable" simultaneously at the SRM console of each node.

Regarding the EEPROMCFG utility. I formated a MSDOS floppy drive and copied the "eepromcfg.exe". I type "arc" at the SRM console, go to utilites, run maintenance program. It shows location a: When I type eepromcfg.exe it always shows "program not found". Same on the other machine. Unfortunately the floppy disk does not have a light indicator and it's too noisy to make out any sound. Am I doing something wrong here?

Thanks,
Markus

[Jan van den Ende]

Markus,

>>>
While checking if I can mount the system disk on the other node I received error %PMA0 CPU0 Remote System Conflicts with Known System - Remote Node.
<<<

Umm.

Are you sure both systems know they are in one and the same cluster?
(if not, you risk total data corruption of drive mounted on both systems).
If so, are the volume labels of the system disks different (but I would expect "Another volume of same label already mounted" error if not).
Are the drive hardware names of that system disk as seen from both nodes equal (including $$ drive letter?
If not, what are they?

hth

Proost.

Have one on me.

jpe

[Markus Waldorf_1]

I have only booted both systems from the Install CD to see if I'm able to mount dka0 from both nodes. As such I have not set any alloclass or installed the systems yet. Guess I will see later...

[Markus Waldorf_1]

During the installation, it prompts me if I want to use the LAN for the cluster interconnect. I'm planning to use Memory Channel with LAN (decnet) fall-back. What should I answer at this prompt? Should I just go ahead and configure the cluster using the LAN, and change it to Memory Channel later?

[Jan van den Ende]

Markus,

>>>
Should I just go ahead and configure the cluster using the LAN, and change it to Memory Channel later?
<<<
Use VMS, and configure BOTH (or all, if more interconnects exist).

VMS automagically will use the optimal path, determined by speed (taking packet size in consideration).

If both systems are booted from CD, what is the "system disk" you are trying to mount? Two copies of the same CD would surely pose incompatibility!

Configure one node as cluster, then use that node to configure the second from the same system disk.
As a starter, use the same alloclass (not 0, 1, or 2) for both.
And be surprised at the ease of the process.

Success.

Proost.

Have one on me.

jpe

[Markus Waldorf_1]

I started to read in the 7.2 Installation and upgrade Manual... I found out that when I answer "yes" at the following prompt:

Will the OpenVMS Cluster use more than one system disk? (Yes/No) Yes

it will detect the Memory Channel and some more info will fly by.

I will see how it continues, but I don't quite understand the question in regard to Memory Channel. I don't need multiple systems disks, but rather one clustered system disk with [.sys0, .sys1] and so on.

[Jan van den Ende]

Markus,

>>>
Will the OpenVMS Cluster use more than one system disk? (Yes/No) Yes
<<<
>>>
I don't need multiple systems disks, but rather one clustered system disk with [.sys0, .sys1] and so on.
<<<

So,

break this off; just start fresh; and answer NO to the multiple SD question.

hth

Proost.

Have one on me.

jpe

[Uwe Zessin]

I don't know if this is documented in your version of the manual (don't have it here and I'm too lazy to search on the web...), but the question for multiple system disks is asked to prevent data corruptions in an upgrade situation:

http://h71000.www7.hp.com/doc/73final/6629/6629pro_002.html#mem_clust_mem

[Jan van den Ende]

Oh, and, Markus,

do not worry about the MC connection.
If cluster nodes "see" each other via multiple paths, they will enable them ALL for connection (unless you explicitly exclude them, but THAT would be an exercise in itself).

Sorry I forgot to mention that before...

Proost.

Have one on me.

jpe

[Markus Waldorf_1]

After the reboot I answered NO when it prompted to use the LAN for cluster interconnect. I use "1" for ALLOCLASS (host default), "2" for port alloclass for cluster PKA0 (KZBPA-CB) and "0" (nodename) for the other SCSI adapters PKB0 - PKE0. When it prompted for a quorum disk I entered "$2$DKA2".

It restarted and successfully formed a cluster. I can see it with also in "sho cluster".

I'm proceeding with the systems updates, have about 36 downloaded. The big 300 update went quite fast, less than a minute. I'm using RAID 0+1 with 6 drives.

Well I answered yes to the multiple system disk questions earlier in the installation. Is there going to be any negative side affect to it? Should I revert this, and if yes how?

Once I'm done with the updates. How will I best proceed in installing the system on the other node 2?

Thanks

[Jan van den Ende]

Markus,

>>>
Well I answered yes to the multiple system disk questions earlier in the installation. Is there going to be any negative side affect to it?
<<<

But _DID_ you configure both nodes as two roots on one system disk?

In that case, you are now already prepared if some time in the future you will wish more system disks. (maybe one day add an IA64 system -- you are prepared.

No other real effects.

Good night -- this is my last entry for today!

Proost.

Have one on me.

jpe

[Uwe Zessin]

> How will I best proceed in installing the system on the other node 2?

I assume that N2 will boot from the same system disk as you already talked about multiple system roots.

Run CLUSTER_CONFIG.COM and add the node. Remember to set the correct root on the second DS20E
>>> set boot_osflags N,0

Some reading:
http://www.openvms.compaq.com/doc/731final/4477/4477pro_013.html#build_cluster

[Markus Waldorf_1]

Hi,


I guess when all is done I will right a nice summary about this, but I'm not quite done yet.

After I installed all the updates, configured TCP/IP and before I run cluster_config to add the other node to the system I took care of the swapfile and 4 GB pagefile. Adding another member with a 4 GB pagefile would certainly exceed the HD.

I relocated the 4 GB pagefile and swapfile to the local disk. Actually I added the files to the existing configuration using "sypagswpfiles.com", and created small files in sys$system. Than after a reboot I purged the previous big files in sys$system. "show mem" looked good.

I was about to do an image backup of the system disk booting from the OS installation CD, when I thought it might be a good idea to run autogen. So I ran sys$update:autogen getparams setparams. Silly me I forgot to put swapfile=0, pagefile=0 into sys$system:modparams.dat and autogen resized the files in sys$system. So I created again small files using sysgen and rebooted the system.

The system continued to hang right after initializing the pkc0 scsi adapter. I waited 10 min, even a cold boot, no change. No disk activity but Ctrl-P and >> cont was still working though. I started the system from CD, mounted the system disk and rename old versions of the parameter files. Well, I managed to screw it up and it would go through the very initial autogen session just like it does right after a system install and it lost cluster an patch update settings. I will reinstall the system.

What went wrong? is there a problem with autogen getdata setparams? The report did not show anything beside resizing the swap and pagefiles.