- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: How to config VMScluster with 6 ES40s thru FC
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2006 10:38 AM
04-16-2006 10:38 AM
Re: How to config VMScluster with 6 ES40s thru FC
Sorry, so later to reply,
I just came back from customer's site, and still have some questions about 6_ES40 cluster. the customer's environment was formed from 6 ES40 plus MA8000 with OpenVMS/Cluster 7.3-1. For custermer's suggestion, the cluster was re_created, not roll_upgrade. ORACLE/OPS 7.3.4 was databasse supported. Node_connection were used 100MBit E_thernet, Storage_connection were used Fabric_channel, I didn't choice the QDSK fuction, every ES40 was with 1 vote, and Expected_vote was 3 votes.
and now I doubt about:
1. How many ES40 must be boot to form the cluster? and when shutdown some ES40, how many ES40 could be left in the cluster, the cluster wasn't hung.
2. How could I improve the Node_connection to aviod the cluster partition? using the second NIC port?
B+R
Charles
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2006 01:40 PM
04-16-2006 01:40 PM
Re: How to config VMScluster with 6 ES40s thru FC
just came back from customer's site, and still have some questions about 6_ES40 cluster.
[...]
I didn't choice the QDSK fuction, every ES40 was with 1 vote, and Expected_vote was 3 votes.
and now I doubt about:
1. How many ES40 must be boot to form the cluster? and when shutdown some ES40, how many ES40 could be left in the cluster, the cluster wasn't hung.
2. How could I improve the Node_connection to aviod the cluster partition? using the second NIC port?
----------
EXPECTED_VOTES should be the sum of the VOTES sysgen param for all six systems. Since each ES40 system contributes one vote, then EXPECTED_VOTES should equal 6. Given that, it will take 4 nodes to be "up" in order to achieve quorum.
If you correctly set the EXPECTED_VOTES param, you will not run the risk of a partitioned cluster.
If you do have multiple LAN adapters, it is often a good idea to set up a private LAN that is only used by SCS (the cluster communication protocol) and no other traffic.
-- Rob
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2006 03:48 PM
04-16-2006 03:48 PM
Re: How to config VMScluster with 6 ES40s thru FC
I doubt also about the EXPECTED_VOTES correctly, I must change it to 6 to aviod the cluster partition.
question:
if make the change, there are must at least 4 ES40s up to get the cluster quorum?
and,
How to set the private LAN (IE1) with SCS?
last,
May I set the difference value of param Vote, EXPECTED_VOTES on each ES40?
B+R
Charles
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2006 06:11 PM
04-16-2006 06:11 PM
Re: How to config VMScluster with 6 ES40s thru FC
To set up a private LAN for SCS communication, just connect the secondary network interfaces (e.g. EIB0) of all 6 systems together. OpenVMS will automatically start SCS communication on all LAN adapters it finds during boot. You can check with MC LANCP SHOW LAN_DEVICE, which network interfaces are used by SCS. MC LANCP SHOW CHANNEL will show you the point-to-point circuits between all the nodes and the channels/devices being used.
You must set the SAME correct values for VOTES (=1) and EXPECTED_VOTES (=6) on all members of the cluster. This is most important, as too low a value for EXPECTED_VOTES together with a network failure preventing total connectivity may lead to partitioning during boot.
Please note that a FC cluster is very vulnerable regarding partitioning, as your disk access path (FC) and your cluster communication path (LAN) are SEPARATE and can fail separately ! But all the disks are accessible locally, so cluster partitioning will immediately put your data in danger of uncoordinated access.
The way I describe the quorum scheme, which prevents partitioning, is this:
You can not divide an apple into more than 2 parts with more than one of them being greater than half of the apple.
As long as more than half of the cluster is up (4 out of 6 nodes in your case) and you've set EXPECTED_VOTES correctly on all of them, there is no risk of partitioning.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2006 09:10 PM
04-16-2006 09:10 PM
Re: How to config VMScluster with 6 ES40s thru FC
both Rob and Volker gave good answers. Particularly their EXPECTED_VOTES remarks are VITAL!
Another of your questions:
if make the change, there are must at least 4 ES40s up to get the cluster quorum?
The answer is only partially "yes".
If the nodes just leave, it is true.
However, if you REQUEST a node shutdown, AND you specify the REMOVE_NODE option, THEN you have a supported way of getting to a lower number of nodes.
This option recalculates the ACTUAL quorum value bases upon the remaining nodes.
Only, when you go from eg, 2 nodes, back again to the full cluster, as soon as one node with EXPECTED_VOTES = 6 joins again, then the cluster hangs temporarily until another node(s) booting adds sufficient votes to regain QUORUM (= total of 4 in your case).
hth
Proost.
Have one on me (perhaps in May in Nashua?).
jpe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2006 04:59 AM
04-17-2006 04:59 AM
Re: How to config VMScluster with 6 ES40s thru FC
That has not been my experience. When I (accidently) did this about 8 or 9 years ago the new node with a too-high value of EXPECTED_VOTES was not let into the cluster. However, I did not get a helpful message and the cluster was doing lots of state transitions :-(
In that case you can:
- abort the boot
- do a conversational boot
- set EXPECTED_VOTES to the desired value
- set WRITESYSPARAMS=0 to prevent an update of the parameter file
- continue with the bootstrap
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2006 05:06 PM
04-17-2006 05:06 PM
Re: How to config VMScluster with 6 ES40s thru FC
After the EXPEXCTED_VOTES changed, Is it right that I must boot at least 4 ES40s up
to get the QUORUM, and When ES40 maintenance requested(shutdown with REMOVE_NODE option), the Min Number of ES40 remained (cluster not hung)is also 4?
........
as your disk access path (FC) and your cluster communication path (LAN) are SEPARATE and can fail separately !
..........
what should I do to prevent the danger?
As all oracle backgroud processes runing on every ES40, Customer could access the data by SQL-NET thru every ES40, How could I balance the application's workload?
B+R
Charles
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2006 05:58 PM
04-17-2006 05:58 PM
Re: How to config VMScluster with 6 ES40s thru FC
As Jan said, you can shutdown/remove individual nodes from the cluster with REMOVE_NODE. The shutdown will re-calculate and re-set EXPECTED_VOTES to the new number of votes after excluding the votes of the node about to be shut down. If you use REMOVE_NODE you can shut down 5 of the 6 nodes and the remaining node will still continue to run.
The tricky part will be booting the nodes into the cluster again in such a state. A new node will NOT be allowed to enter the cluster, if the new cluster config (new EXPECTED_VOTES) would cause the cluster to loose quorum. If you are down to 1 node, you should then boot at least 3 other nodes at the same time, if you don't want your cluster to hang.
I would also suggest, that you make yourself familiar with the IPC interrupt for re-calculating quorum or the Availability Manager Fix Quorum function. Those will allow you to re-calculate quorum from the remaining members in the cluster and allow you to re-vive a cluster hung due to loss of quorum.
The best tool to distribute the incoming TCPIP traffic across the 6 nodes would be TCPIP LOADBROKER.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2006 09:50 PM
04-17-2006 09:50 PM
Re: How to config VMScluster with 6 ES40s thru FC
Have a look with SHOW CLUSTER/CONT then
ADD VOTE, EXPECTED, QUORUM and see.
Purely Personal Opinion
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2006 09:55 PM
04-17-2006 09:55 PM
Re: How to config VMScluster with 6 ES40s thru FC
http://www2.openvms.org/kparris/bootcamp2005_cnxmgr.pdf
Purely Personal Opinion