BladeSystem - General
1748156 Members
4055 Online
108758 Solutions
New Discussion юеВ

Re: HP Flex-10 and blade server(c7000) planning!!

 
SOLVED
Go to solution
astra
Frequent Advisor

HP Flex-10 and blade server(c7000) planning!!

Hi All,

Currently we are using RX2660 server and Cisco catalyst switches for interconnection.

We are using the following softwares :
- HP-UX B.11.23V2
- HP Serviceguard/Serviceguard extension for RAC
- Oracle RAC 10Gr2
- HP-APA for port aggregation (manual mode)

Now, we are planning to migrate the whole system to Blade servers and HP-Flex 10 (10GBe) technology. Considering the system usages as mentioned above, what are the things i should take care while implementing it. I am also raising some related questions :-
- Do we still need HP-APA?
- Can we use FlexNICs for HP-Serviceguard and Oracle RAC?
- How the HA of lan connections and Oracle RAC VIP will be ensured?
- Do i need to create VLANs on server side/VC module?
- Can i use each FlexNiCs for different traffics?

If there is any related guides please share the links.

I shall be grateful to you all.

Thanks in advance,

Amit
9 REPLIES 9
Adrian Clint
Honored Contributor

Re: HP Flex-10 and blade server(c7000) planning!!

Most important question first - Which blades are you looking at to replace RX2660?
As the BL860c and BL870c do not support flexible NICs on their onboard NICs.
I believe what you get if you use VC-Flex-10 is 4 NICs at 2.Gb.
You would need to add a 10Gb dual port NIC card and another pair of Flex-10 modules to get Flexible NICs. $$$$$$$$$$$

How many VLANs/Networks do you have and how much bandwidth do you want on each and do they need to have HA on each network?
astra
Frequent Advisor

Re: HP Flex-10 and blade server(c7000) planning!!

Hi,

We are going to use BL860c. We will aslo use Flex-10 ethernet port.

Do we also need to have VC-ethernet module?

Currently we have 4 VLans and all are in HA (HP-APA).

Do we need to have Hp_APA if we use FLEX-10 ethernet ports? How the HA of Lans will be achieved?

Thanks in advnace,

Regards,
Amit




As the BL860c and BL870c do not support flexible NICs on their onboard NICs.
I believe what you get if you use VC-Flex-10 is 4 NICs at 2.Gb.
You would need to add a 10Gb dual port NIC card and another pair of Flex-10 modules to get Flexible NICs. $$$$$$$$$$$

How many VLANs/Networks do you have and how much bandwidth do you want on each and do they need to have HA on each network?
The Brit
Honored Contributor

Re: HP Flex-10 and blade server(c7000) planning!!

Adding on to the comments from Adrian, above.

If you are using bl860's, then FlexNIC's are not an option for the LOM ports, however you do have 3 Mezzanine slots so extra NIC's can be added (Flex or otherwise)
Network redundency within the enclosure is achieved by installing your interconnect modules in horizontal pairs and then configuring independent connectivity from "Left" modules to NETWORK/SANFabric, and from "Right" modules to NETWORK/SANFabric.
At the server level, Network redundancy would be by setting up NIC "Teams" with one member of the team connected to the "Left" side, and the other to the "Right" side.

Will you be splitting your RAC cluster between enclosures?? If it is all in one enclosure, then there will be no need for your RAC/ServiceGuard communication to leave the Enclosure, it can all happen across the mid-plane (which is 10Gb). If it is between two enclosures, then the VC modules in each enclosure can be stacked into a single VC domain, and if the stacking is done using CX4 cables, then you will have a 10Gb path between your servers, which does use the external network.

Can you use different NIC's for different traffic ?? Yes.

Dave
astra
Frequent Advisor

Re: HP Flex-10 and blade server(c7000) planning!!

Hi Dave,

Thanks for your valuable info.
If i understand correctly, this should be the configuration (correct me if i am wrong)

Server1(Flex-10 port1)--->VC-Enet module (Left)
Server2(Flex-10 port1)--->VC-Enet module (Left)
Server1(Flex-10 Port2)--->VC-Enet module (Right)
Server2(Flex-10 Port2)--->VC-Enet module (RIght)

Similarly,

VC-Enet module1---->Switch1
VC-Enet module1---->Switch2
VC-Enet module2---->Switch1
VC-ENet module2---->Switch2

If you have any reference documents please provide me with the same.

Will you be splitting ur RAC clusters between eclosures?

Ans>Yes, we can. But we will try to keep it in same enclosure.

Thanks in Advance,

Regards,
Amit

The Brit
Honored Contributor
Solution

Re: HP Flex-10 and blade server(c7000) planning!!

Amit,

First a correction to my earlier post: I said,

"then you will have a 10Gb path between your servers, which does use the external network."

This was a typo, it should have said "...does NOT use the external network."

Anyway, your configuration looks to be fine, however just to be clear, If you are determined to use Flex10, then, assuming that you put your Flex10 cards on the bl860c Mezz 1 slot; Your Interconnect modules setup needs to look something like this (considering Ethernet only)

Bay 1 - VC ENet module of some type.
Bay 2 - VC ENet module of some type.
Bay 3 - Flex10 VC Module
Bay 4 - Flex10 VC Module

Should also include CX4 stacking links between Bay 1 -> Bay 3, and Bay 2 -> Bay 4, i.e. vertically between the left-side modules, and similarly for the right-side modules.

Note:: there MUST be VC modules of some type in Bays 1 & 2 because this is where Virtual Connect Manager (VCM) runs.

If you place the Flex10 Card on one of the other Mezz ports, then the Flex10 VC modules will have to move to different IC bays.

Remember also, as pointed out by Adrian, you can put Flex10 modules in Bays 1 & 2, however you will not be able to configure FlexNIC's on the bl860c LOM NICS.

There are some benefits which go with this configuration however. If your datacenter Network infrastructure is NOT 10Gb, (i.e. if like most people the main network is still 1Gb or less) then you can run your uplink cables directly from the 1/10Gb modules in bays 1 & 2, to the Main network switches without any additional cost. If you want to run uplinks for the Flex10 modules, and the main network is not 10Gb, then you will have to purchase additional 1Gb SPF Options and appropriate cables for each port you want to uplink through.

Dave.
astra
Frequent Advisor

Re: HP Flex-10 and blade server(c7000) planning!!

Hi Dave,

Thanks for detailed information and its quite useful for me to get going with this.

Our data center network is 1GB. So, in that case as per your explanation, i need to have 2 VC-ENet modules (of some type), and 2 Flex-10-VC modules..right? If i dont use 2 VC-Ent module of some type and i use 1GB SPF, then where the VCM will run? Isnt FLex-10 VC module support VCM?

What is the difference between VC-Enet module of some type and Flex-10 VC module? Is it only to use to connect to the network where the data speeed is less than or equal to 1GB?

Apart from that, how do we achieve the complete bandwidth use of 10GB, if our datacenter network is 1GB?

Once again many many thanks.

Regards,
Amit
The Brit
Honored Contributor

Re: HP Flex-10 and blade server(c7000) planning!!

Amit,
Keep in mind that this discussion is based on your earlier statement that you are going to use bl860c blades.

The bl860c has 4 LOM NICs which are hard mapped to Interconnect bays 1 & 2. These NIC are NOT FlexNICs, although they can talk through a Flex10 VC module in Bay 1 or 2, but only at (officially) 1Gb, (unofficially closer to 2Gb) each.

Flex10 modules ARE VC modules, and can run VCM when installed in Bays 1 & 2. The point you must grasp however, is that the LOM NICs are not FLEXNICS, and they only talk to IC bays 1 & 2. Because of this, (if money is a constraint), you can get away with having 1/10Gb VC ENet Modules in Bays 1 & 2, and matching up your Flex10 Modules with a 10 Gb Mezzanine Card in your servers.

If you get a suitable 10Gb Mezz Card and place this card on Mezz 1, then you need to have Flex10 modules on IC bays 3 and 4. The location for the Flex10 modules will depend on which Mezz slot you use for the 10Gb Mezz card.

If place the 10Gb NICs on the Mezz 1 slot on your servers, then your Interconnect bays should look like.

Bay 1 = 1/10Gb VC ENet Module, or Flex10 VC Module.
Bay 2 = 1/10Gb VC ENet Module, or Flex10 VC Module.
Bay 3 = Flex10 module.
Bay 4 = Flex10 Module.

with this config you will have

4 NICs / Server to Bays 1 & 2 at 1(2)Gb.
8 NICs / Server to Bays 3 & 4 at (total 2x10Gb)

Now add CX4 Stacking Links from Bay 1 --> Bay 3, and Bay 2 --> Bay 4.

With this done, you now have 8 x 1Gb uplinks to your external network from either Bay 1 or Bay 2, depending on which is Active.

Dave
Rthompson_2
New Member

Re: HP Flex-10 and blade server(c7000) planning!!

I'm needing the same information. We have a Flex-10 enclosure and only Flex-10 modules in port 1 and 2 no other VC modules aside from the san modules. I was under the impression that I could setup our heartbeat link without having to go outside the enclosure (uplink) but have yet to figure out how to do this, and with 11g RAC you need SCAN addresses as well.
Rthompson_2
New Member

Re: HP Flex-10 and blade server(c7000) planning!!

I have one bl860c in slot 1/9 and a bl460c in bay 8 and bay 16 these are the two blades that will be running 11G RAC