10GB Ethernet Cross Over possible (Oracle RAC Interconnect)?

[farid S]

Hi,
To enhance my Oracle RAC cluster interconnect, I'm switching from HyperFabric to 10GB ethernet, which is supported on 10g (Not HF). Since I have a couple of RP7420 Servers running HPUX 11iV1, I will buy 4 cards with P/N AB287A. I want to use cross over between the nodes, but cannot find any documentation that explains if it is supported or I have to use 10Gb switches (which are quite expensive)?
So, is it possible to connect the nodes in crossover config using these cards, and what cable should be used (LC_LC...)?
Many thanks,
F.

[Duncan Edmonstone]

Oracle don't support using crossover cables for RAC interconnect - they never have. For details see the RAC FAQ in metalink (doc id 220970.1)

HTH

Duncan

[farid S]

Hello,

I'm well aware of that, and I'm using this config only to test performance, and after validating the performance tests, I will buy a module for My Cisco 6500 to support 10GB ports.
But My question was regarding the possibility to use cross over cables with these cards? not Oracle support.

N.B. Oracle do not support CO only for availability reasons, not performance ones.

Thanks,
F.

[rick jones · Accepted solution]

It should be possible to connect a pair of AB287A's back to back and have frames flow between them.

I'm not sure that HP even still has the AB287A on the price list any longer - I would have thought it was replaced by the AD385A. But then I'm not sure if the AD385A was qualed with 11iv1/PA-RISC systems.

FWIW the AB287A is a PCI-X 1.0 133 MHz card and as such will have an upper bound of 7.X Gbit/s no matter what (no idea if an rp7420 can get there with enough concurrent connections or not). The AD385A is a PCI-X 2.0 266 MHz card and should be able to be driven rather faster - assuming of course it is placed in a PCI-X 2.0 266 MHz slot... and there is enough "oomph" in the CPUs.

You may want to consider using JumboFrames (9000 byte MTU) to help lessen the CPU overhead of data transfer through the NIC(s).

A "regular old" LC-LC cable should work. Since GbE and later, TX vs RX is figured-out automagically by the cards.

Naturally, I would suggest a ProCurve switch when you go to get a 10G switch :)

[farid S]

thanks dude...that's what I call an answer.

[rick jones]

BTW, except perhaps for bandwidth tests, connecting back-to-back will give results not achievable when you connect via a switch. In particular, the switch will add latency and so you can see a difference in say netperf TCP_RR between back-to-back and via a switch. Now, the difference may be measured in small quantities of microseconds, but it is there and it is measurable.

Just to be certain, and at the risk of preaching to the choir, there is a fundamental difference between "works" and "supported" in HPSpeak. To wit:


Supported, known to work -> warm fuzzies all around
Supported, not known to not work -> an HPite may be in trouble
Supported, known to not work -> an HPite is in trouble
Unsupported, known to work -> lucky today, unlucky tomorrow?
Unsupported, not known to not work -> there but for the grace of Turing
Unsupported, known to not work -> no, it was not deliberate ;-)

There is nothing in this case to prevent you from boldly going forth into the unsupported space, but make sure you realize you go there at your own peril :)