Oracle RAC cluster interconnect nic bonding

[mghosalk]

I am in the process of purchasing a C-class blade server solution for implementing oracl e-business suite with oracle rac. one of the issues i am facing is designing rac interconnect. i will have 16 bl460 servers, with 4 cisco catalyst 3020 switch modules for public lan and interconnect. if i have to bond two nic ports for rac interconnect, then can i mesh it across the two cisco switches and still bond them from cisco switch side. i would like to have a 2gb interconnect between oracle rac database servers. Thanks Mandar

[mkendall]

Mandar, I will have Keith Miracle on my team get you an answer. Thanks! Mike

[wonderboy]

Mandar: I've requested some support from my Cisco counterpart. We should have a response very soon. BTW, what OS are you running? Thanks ... Keith

[wonderboy]

Mandar: Here's a reply from a Cisco expert: ******************************************** Yes, it is possible to get 2x the throughput with bonding. You will have to be careful in your selection of bonding methods, some are for fault tolerance and others are for load balancing. Last time I worked with this under Linux there were numerous bugs in the various NIC and bonding device drivers. The Intel-based NICs demonstrated different behaviors than the Broadcom based NICs. The bonding device driver that was initially supplied with RHEL4 didn't function well at all. I would strongly suggest getting the latest NIC and bonding device drivers before beginning to set this up. I would advise browsing the source code looking for the developers comments, there are "undocumented" features listed in the code that helped debug the initial setup. Verify that the switch firmware (embedded OS) has been updated. Check www.cisco.com or www.procurve.com for specifics proper setup of NIC teaming in the switch. ********************************* Does this spark any thought?

[mghosalk]

thanks for the indo. didnt knew that i would have to read source code to make this work. shouldnt hp provide some kind of best practice or howto for blade-linux-nic bonding, rather than customers read source code. maybe i am used to working on hpux, than rhel.

[rh2281]

I seriously doubt that you will need the full 2gb pipe for the RAC interconnect traffic. During heavy OLTP testing with a 8 node RAC cluster, the amount of data on the interconnect was fairly light (plenty of head room left). We decided to use the failover nic bonding instead of trying to use load balancing. We used QLogic drivers for Linux RHEL 4AS