Dual Domain - 2 x P812 with 4 x D2600

[DaveSWP]

Hi,

I am trying to configure dual domain support on 2 x p812 controllers in a DL580 G5 connected to 4 x D2600 enclosures. Unfortunately though, there seems to be very limited documentation on the configuration of dual domaining.

I have applied the latest firmware to both the p812 cards (v3.5) and the controllers in the D2600 and was led to believe by an HP storage expert that there is full dual domain support in this setup. i.e. p812 card, SAS cable, D2600 enclosure I/O module fault tolerance.

The physical setup in terms of the equipment positions in the cabinet is:

------------
server
enclosure 1
enclosure 2
enclosure 3
enclosure 4
------------

The enclosures are daisy chained (cascaded) together like so:

P812 card A --- Enclosure 4 I/O module B P1 (left)
Enclosure 4 I/O module B P2 (right) --- Enclosure 3 I/O module B P1 (left)
Enclosure 3 I/O module B P2 (right) --- Enclosure 2 I/O module B P1 (left)
Enclosure 2 I/O module B P2 (right) --- Enclosure 1 I/O module B P1 (left)


P812 card B --- Enclosure 1 I/O module A P1 (left)
Enclosure 1 I/O module A P2 (right) --- Enclosure 2 I/O module A P1 (left)
Enclosure 2 I/O module A P2 (right) --- Enclosure 3 I/O module A P1 (left)
Enclosure 3 I/O module A P2 (right) --- Enclosure 4 I/O module A P1 (left)


The back of each of the D2600s has been automatically numbered. The numbers increase sequentially from top to bottom. However, there seems to be a discrepancy with enclosure 2.

enclosure 1 (1,1)
enclosure 2 (2,3)
enclosure 3 (3,3)
enclosure 4 (4,4)

I hope I have managed to explain the setup of this. If not, then I might be able to get a photo.

I have a couple of questions.

1. Have I physically attached things correctly?
2. Where do you actually configure dual domaining support? I can't seem to find anything in the ACU?
3. Why is the automatic numbering of the enclosures as it is. If I need to, how can I change it?

I have tried creating an array on one of the cards (lets call it card 1) and seeing if any arrays were "visible" on the other card (card 2). After the array has completed initialising on card 1, then the array became visible on card 2 and looked ok. HOWEVER, the array on card 1 then said that it had failed. I have tried this twice.

Has anyone any tips or ideas?

Thanks for you time.

[Torsten.]

IMHO according to the quickspecs and the user manual

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual〈=en&cc=us&docIndexId=64179&taskId=135&prodTypeId=12169&prodSeriesId=3954787

you must connect each enclosure to a single controller only (2 cables).

See chapter 7.

[DaveSWP]

Thanks for replying. I was afraid that might be the case. However, this document seems to suggest to me that you can use two controllers in the server.

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00687518/c00687518.pdf

(bottom pg. 24)

"Dual domain SAS implementations can tolerate host bus adapter (HBA) failure, external cable failure, expander failure, or failure in a spanned disk (JBOD)."

[Torsten.]

I found this phrase on several places - it is always exactly the same, but no more details.

The problem is - like on any other storage array - the cache. In theory the controller cache needs either to be disabled (bad idea) or somehow synced between the controllers. So if one controller is going bad, the cache data must be flushed by the other controller.
This means they have to talk to each other (are you alive?), one has to have the ownership of the disk or LUN, if a write request comes in the controller has to cache the data, "mirror" the data to the other controller, send an ack and finally write the data.

I doubt if the smart array controllers can do this.

Maybe someone else can clarify.

[DaveSWP]

Thanks Torsten for your help. I just wanted to post an update.

Although the HP documentation and storage expert said that it was possible to have redundant HBA cards, it is in reality not possible.

The arrays have now been cabled using just one p812 and two ports in the p812. One chain cascading down the arrays, while the other cascades up the arrays. Everything still performs if a single cable is removed which is great and this solution provides redundancy of cables, array I/O controllers and arrays (with the RAID level we are using).

Although we can't get HBA redundancy, I have managed to get the next best thing. Two identical p812 cards remain in the server with the SAS cables being connected to just one of the cards (as above). If however the card fails, then by shutting down the server, connecting the SAS cables to the other p812 and booting back up, all of the arrays and data are still visible and accessible.

[Vlad Pintea]

hi,
we have connected 4 D2600 storage to a DL580 G7 with 4 P812 like so:
* P1 connector from I/O Module A of the HP StorageWorks D2600 Disk Enclosure
array and Port 1e of the P812 Smart Array Controller card from the back
of the
DL580 G7 server
* P1 connector from I/O Module B of the HP StorageWorks D2600 Disk Enclosure
array and Port 2e of the P812 Smart Array Controller card from the back
of the
DL580 G7 server


The server detects the disks (48 disks/12 for each enclosure) but a red
LED light is blinking on the enclosure on the P1 connector from I/O Module A