ProLiant Servers (ML,DL,SL)
1748199 Members
2998 Online
108759 Solutions
New Discussion юеВ

Re: 4-port shared storage module with 4 nodes

 
Gabriel_24
Occasional Contributor

4-port shared storage module with 4 nodes

Hi,
We are connecting 4 DL380 3G with SA5i+ to a SACS with the 4 port shared storage module.
We have upgraded to the latest firmware updates, 1.70 for the SACS and 2.38 to the SA5i+.
We are using the SSP with only one Logical Drive and it's viewed only by one Proliant DL380 3G.
First: we boot the SACS and we wait for startup complete.
Second: Boot two DL380 connected at A1 and B1 ports. It works properly
Third: Start DL380 connected at B2 port. It also works well
Fourth: When we try to start the fourth DL380 connected on the A2, it gets hanged on the 5i BIOS messages. It shows a lot of -\|/ symbols and after it gets hanged. The DL380 connected to the A1 also stop working with the SACS.

We have made a lot of tests and it's always the same: The channel A1/A2 doesn't work with two servers.
We also have proof with the SA532, but nothing new.

Any ideas?

9 REPLIES 9
Doug de Werd
HPE Pro

Re: 4-port shared storage module with 4 nodes

When you state that you are only using one Logical Drive - is that one total Logical Drive, or one Logical Drive per server?

If you are doing one total LD that is being accessed by all 4 servers, then that is your problem. Each server must have its own dedicated LD - so you need at least 4 LDs. Then you use SSP to assign access to only one server. You cannot have all 4 servers accessing the same LD.

Thanks,
Doug
I am an HPE employee
Accept or Kudo
Gabriel_24
Occasional Contributor

Re: 4-port shared storage module with 4 nodes

There's only one LD and it's accessed by one server
Doug de Werd
HPE Pro

Re: 4-port shared storage module with 4 nodes

It's possible that the 4-port module is defective. As part of the POST process, the 4-port module works with the Smart 5i controllers to make sure the SCSI ID addressing between the 5i and the SACS controller is OK. In essence, it changes one of the server's 5i's to SCSI ID 6 (like A1), leaving the other one (A2) at 7. This happens automatically and is not configurable.

The only other thing I can think of, which you may have already tried, is to change the cabling on the servers to the SACS - use different servers than you have now. That would rule out a bad 5i. I would also recheck your SSP settings - if the A2 servers has been accidentally given access to the LUN via SSP, this could cause problems as well. Another thing you could try would be to create 3 more LUNs in the SACS, and then give each of them exclusive access (via SSP) to the other 3 servers. I don't know, but maybe the A2 server wants to see a LUN on the SACS.

Thanks,
Doug
I am an HPE employee
Accept or Kudo
Dmitry_31
Occasional Advisor

Re: 4-port shared storage module with 4 nodes

I think even if you have 4 ports on your smart array cluster storage it still can handle only 2 servers. 4 ports are for multiple paths (redundancy issues). Check documentaion, I am pretty sure I've seen this limitation there.
Doug de Werd
HPE Pro

Re: 4-port shared storage module with 4 nodes

the limit with the 4-port module is:

1) 2 servers with multi-path SW (standalone or clustered)

or

2) 4 servers single path (non-clustered)

Thanks,
Doug
I am an HPE employee
Accept or Kudo
Juanjo Villaplana
Occasional Visitor

Re: 4-port shared storage module with 4 nodes

Hi Doug

I'm curious about the 4-port limitations ...

Do you know what are the hardware limitations that disallow configurations such as:

a) Two clusters of two nodes.

b) A two node cluster and two independent (non-clustered) nodes.

c) A four node cluster.

Best regards,
Juanjo
Doug de Werd
HPE Pro

Re: 4-port shared storage module with 4 nodes

The limitations for a) and b) are similar. The design of the 4-port module has some fancy circuit design where one side (the storage box side) shares a common SCSI bus/ID, but on the other side (server side) reconfigures the servers so there is no SCSI ID conflict. (This is all done behind the scenes - the SCSI IDs are not manually configurable). Any time you have a cluster on there - whether it is one cluster and 2 servers or 2 clusters - you can run into problems when a cluster failover occurs. Since a cluster failover involves doing a series of SCSI bus resets, and because of the shared SCSI bus/ID design of the 4-port module, a failover on one of the clusters can interfere with the other 2 servers. You don't have this problem with 4 standalone servers because they aren't running any cluster software to generate the bus resets.

As for c)the 4-node cluster, it is "probably" going to work OK, but it is not a tested or supported hp (or Microsoft) configuration. Because all of the nodes are in the same cluster, the bus reset issue does not matter, since being part of the cluster requires them to react to the bus resets. However, the 4-port module was not intended for use with 4-node clusters, and it has not been tested in that environment. HP would recommend that you use an MSA1000 storage box for clusters with more than 2 nodes.

Thanks,
Doug
I am an HPE employee
Accept or Kudo
Juanjo Villaplana
Occasional Visitor

Re: 4-port shared storage module with 4 nodes

So, I guess those limitations are at OS level, because the firmware (on the SACS and SA 5i/532) won't know if the 4 SCSI adapters attached to the SACS belong to 4 non-clustered servers or to 2 multipath clustered servers.

And hence, the negotiation process (adapter ID change, ...) does not depend on the configuration, that could be guessed from the SSP settings.

Being those limitations at the OS level, and primary related to SCSI bus resets, I will investigate if the clustering software I want to install (RedHat's Cluster Manager) resets the SCSI bus when fails-over. In any case I know configurations a) and b) are not supported by HP, in fact, according to http://h18004.www1.hp.com/products/quickspecs/11050_div/11050_div.html RedHat's Cluster Manager is not supported at all.

Finally, I think that the OS on a non-clustered server should also be able to react to SCSI bus resets, because even when configuring 4 independent servers on a SACS, one of them may generate a SCSI bus resets (due to a defective cable, for example) and those resets will be received by the 3 remaining servers.

Best regards,
Juanjo
Gabriel_5
New Member

Re: 4-port shared storage module with 4 nodes

Problem Solved, it was a defective 4-port shared storage module