Cluster with C7000

[eric lai]

We have a C7000 with 2 GbE2c Ethernet Blade Switch modules and 8 blade servers (ProLiant BL460c G1). We are planning to cluster 2 blade servers together(MS Server 2003 Cluster). I noticed there are 2 NICs in each blade server and each NIC goes to different GbE2c switch. To install the cluster, I need to use a "Gb" link to the heart-beat connection which only requires 10Mb link. Is there a better way to setup the heart-beat connection (i.e. add the 3rd low speed NICs...) Then, I can team both Gb NICs and push nw speed to 2 Gb. (a big plus to exchange mailbox server). Thanks !!!

[Kase]

Eric, if you add a NIC mezzanine card in the first slot and purchase an additional ethernet switch, you can create a MSCS cluster as described. You would need to create a separate private heartbeat VLAN on the third ethernet switch. The setup described, however, is not an optimal MSCS HA configuration. We use a full heigth blade for MSCS clusters so we get teamed NICs for the public network (providing redundant switches) and the third internal NIC is on a private heartbeat VLAN as described. Note, MSCS guidelines state "Each cluster network must fail independently of all other cluster networks. That is,
two cluster networks must not have a component in common that can cause both to
fail simultaneously." I believe the half-heighth blade does not fit this criteria, whereas, with the NICs correctly configured in MSCS, the full-heigth blade does.

[eric lai]

Thanks for your quick reply. Do you mean I need to buy a third switch to make this heart-beat connection ? Can I just setup a private vlan in Sw1 & sw2 to make this heart-beat connection ?

[kris rombauts]

Hi Eric,

yes you could do that, i created the VLAN also this way however then you are not able to team the onboard NIC's (which is what you like to do) since, with only two interconnect Ethernet switches, you are not able to connect any additional network ports you from the server blades.

This is because the port mapping from the server blade mezzanine cards in mezzanine slot 1 i.e. map to the switch modules in the slots 3 and 4 in the back of the enclosure.

So either you build a hearbeat within the 2 switches you have now or you need to buy additional Ethernet switches and mezzanine cards to accomplish what you want.


As mentioned before, with the full height blades which have 4 build in network cards one can build a team and heartbeat lan with just 2 Ethernet switches because all those 4 network ports map to the first two interconnect swithes in the back of the enclosure.

Have a look here, this will make it more clear to you:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00816246/c00816246.pdf?jumpid=reg_R1002_USEN

figure 15 and 16 page 22

HTH

Kris

[eric lai]

Kris,

Thanks for your response. Do you think that is possible to add a new a dual ports mezzanine card in a blade server and configure those two NICs talk to blade switch 1 and 2 ? Otherwise, to buy a new blade switch just for the heartbeat connection is very pricy. But, I will go that way if I have no choice.

Thanks !

Eric Lai

[Raghuarch]

Do you think that is possible to add a new a dual ports mezzanine card in a blade server and configure those two NICs talk to blade switch 1 and 2 ?
NO Eric. Its not possible in C7000 enclosure.
If you have C3000 the Mezzanine 1 will map to Switch 2.

[Bruno J. Melo · Accepted solution]

Hi Eric,

Unfortunately, you can't change the internal port mapping of the C7000 enclosure.

As you can see in the attached file, the half-height server blade's (BL460c) integrated Gigabit Ethernet Adapters are mapped in a c7000 enclosure to interconnects 1 and 2 (Where you have your 2 GbE2c Ethernet Blade Switches).

If you want to add more Ethernet adapters to your configuration, you must place an interconnect on the back of the chassis that maps directly to your PCIe mezzanine card positioning. Mezzanine 1 will lead you to Interconnects 3 and 4. Mezzanine 2 will lead you to Interconnects 5, 6, 7 and 8 (depending on the type of mezzanine card used).

Since you're planning a cluster environment I suppose you have already occupied bays 3 and 4 with some sort of interconnect for Fibre Channel connectivity for your cluster's shared storage.
That will leave bays 5, 6, 7 and 8 free for more Ethernet connectivity.

If I was in your place, I would try to add 2 more switches to bays 5 and 6, and a dual-port Gigabit Ehernet mezzanine card to the second mezzanine slot on each server.

But if you're struggling with budget constraints you can always consider buying Ethernet Pass-throughs instead:

http://h18004.www1.hp.com/products/blades/components/ethernet/pass-thru/index.html

These are somewhat cheaper than Ethernet switches and give you direct port mapping to each ethernet port on your blade servers.

Hope this helps.

[eric lai]

Hi, Bruno,

Thanks for your suggestion. I think that is the way we will go.

Thanks for all of you again.

Eric Lai

[PJG1]

Eric,

We are looking into doing something similar. My question is, what are you using as the shared storage? A storage blade or a SAN on the network?

[eric lai]

we use 2 Brocade 4/24 SAN Switches and they both are connected to 2 HP StorageWorks 30 Modular Smart Array.

[Bruno J. Melo]

Hi PJG1,

Storage blades do not apply for shared storage... When they are inserted in a blade enclosure they are allocated directly to the server next to it.

It's Direct Attached Storage in a blade format as you can see here: http://h18004.www1.hp.com/products/servers/storageworks/c-class/sb40c/index.html

It's not possible to share it between server blades.

For cheap shareable network storage you should look into iSCSI.

[PJG1]

Thanks to all for the replies. This is my first attempt at clustering with blades and just needed some info to get started.

[PJG1]

Bruno,

Hate to hijack Eric's thread but I need help with this. So, here's what we have. We already have a full c7000 enclosure which we have been running since the beginning of the year. We just recently purchased another c7000 with 4 bl460's and 2 GbE2c switches. What are the mininum items I would need to set up a simple cluster between 2 of the blades. Obviously shared storage, probably use iScsi, etc. I know you reccommend the bl480's, but if there is a way to get it working with the bl460's, I'm sure the boss would be happier. Thanks.

[Bruno J. Melo]

Hi PJG1,

Can you please start a new thread?
Just copy and paste this and I'll answer there, I think it's best for you and the community.
As you may have noticed there's a points system to grade responses to the questions posted, which I agree is a great way of rewarding the people that spend some of their time helping others...
If you start a thread you can grade the responses.

[Bruno J. Melo]

Hi PJG1,

Can you please start a new thread?
Just copy and paste this and I'll answer there, I think it's best for you and the community.
As you may have noticed there's a points system to grade responses to the questions posted, which I agree is a great way of rewarding the people that spend some of their time helping others...
If you start a thread you get faster results and you can grade the responses.

[PJG1]

Yes. Will do. Can only the author of the thread assign points?

Thanks.

[Bruno J. Melo]

Yes, only the thread author.