BladeSystem Virtual Connect
cancel
Showing results for 
Search instead for 
Did you mean: 

ESX(i) Flex-10 and Blades - what the best networking setup?

julianwood
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

We're working on our design for migrating our ESX environment to Blades and are keen to go HP Flex-10. How are you setting up your networking? We use NetApp NFS for all storage. We would have multiple VLANs for our production networking. We would have a single 10GbE uplink per Flex-10 module. We wouldn't stack together chassis. Do we have shared uplinks or just dedicated uplinks? Do we just pass through VLANs to ESX or actually set up the VLANs in VConnect? Do we get the ESX hosts to do the failover or get VConnect? Are you carving up your FlexNics or just using the single 10GbE? Do you separate LAN / NAS physically or with VLANs or not at all? Do you separate out Service Console and VMotion? Do you use portgroups to direct traffic over particular uplinks but allow failover? What Blades do you use? The 490c can take more memory but doesn't have hot swappable disks. Does it matter? You cannot currently mirror the internal solid state disks. The 460C has less memory but HSS disks, so is that not a single point of failure. 8GB Dimms are very expensive so we're thinking of going 4GB dimms and getting more blades which works out cheaper. Do you run ESX or ESXi installable or ESXi Embedded? Anyone running ESXi off the internal SD cards on a 490C? Would love to see some reference architecture or at least get everyone's thinking for how to set up vSphere with Blades.
12 REPLIES
chopper3
Frequent Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

If I were you I'd just present 2 x 10Gbps VC trunk nics (although enable all 8 nics but with nothing assigned to nics 2-4 and 6-8 so you can change things later without NIC enumaration issues), let ESX do the faiover and vlan tag management. I do have different vlans for SC and VMotion. We use 490c's with a single SSD for ESX and 460 G6's for everything else (like windows and other baremetal builds), always go with 4GB mem modules, the 8's aren't worth it. I use ESX over 'i' as the management agents in 'i' are no way near as good as the regular ones. I have ran 'i' off SD, they're quite nice for that really as the slot is right at the front on the top but much harder to get to on the 460, worked ok but prefer regular esx off the SSD. I'm happy to help with anything else in this arena, I'm in the UK and I'm phil at buckley-mellor.com ok.
Pascal
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

Hello, We use ESX Host as well and the flex-10 technology in the same way as chopper3 does (vmware is doing the vlan tagging stuff). All ESX host are installed on BL460G6 blades. We use also Netapp filer and NFS to store vm datastore. We did create several volume (per application or per server type) to be able to isolate activities and performance on the filer. A dedicated volume is also created to store only vm's swap files. All other vm's files are put on deduplicated volumes using SIS (40% to 70% disk saved at the moment with around 160 vm depending on wich volume you look at). We do not run (i) because it is very limited if you want to use scripts and cli commands. Failover is done by the flex-10 and the mode you choose for your ulink (auto/failover) and transparent for the host and vm.
julianwood
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

chopper3 / Pascal do you then have only 1 x vSwitch and multiple VLANs for SC/VMotion/VM/NFS. Are your 2 uplinks setup with active / passive or do you route traffic from some portsgroups over uplink1 and other portgroups on uplink 2.
chopper3
Frequent Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

Yes, we only have a single vswitch with multiple vlans, we currently use active/passive throughout but you can make them active/active if you use Cisco's VSS.
Pascal
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

We use several vswitchs on our side though it is for compatibility reason with some other non hp hardware. Active/Passive mode is also our choice, despite the fact that we have a strange behavior on the passive flex (we get some inbound traffic on it and we dont know why yet). You can use active/active with cisco'vss and 6500 series, also with Stacked 3750 switches, or with VPC and nexus 5000 (for the last one ..get some time before going to it ..the paint is not dry).
julianwood
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

So, what do you do with your 6 unused Flex-Nics. I really do think HP should have a way that you can disable these. If you don't allocate anything to them HPSIM sees them as down, previously we used to fool HPSIM by plugging in a loopback adapter set to 100Mb and the link would be sees as up. What do you do with blades?
chopper3
Frequent Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

It's good that you can see them all, that way their enumeration won't change if you reconfigure the VC profile, they'll stay the same.
julianwood
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

Well other than not actually wanting to see them, I Don't want to have unused Nics being presented but if I have to live with them being present how can I get HPSIM to ignore them so they are not reported as down?
mboanta
Occasional Advisor

ESX(i) Flex-10 and Blades - what the best networking setup?

Hey guys, I was actually referred to this site from an HP friend of mine. I have a SR Systems Engineer position open here in Orange County, CA and I am looking for someone that has strong experience with the following: HP Proliant series servers, c-Class blades/enclosures, enclosure interconnects/switches HP Proliant Virtual-connects (including Flex-10 VC) HP Insight Control environment for Windows (including HPSIM, RSP, RDP, VCEM) VMWare ESX 3.5, vSphere4, Virtual Center, Update manager, VMWare Convertor Please give me a call at 949-417-4818 if you or anyone you know is qualified and interested in a new opportunity. Thanks again for your time and help! Sincerely, Joe 949-417-4818 jojhenry@teksystems.com
Mustafa Sayla_1
Occasional Advisor

Re: ESX(i) Flex-10 and Blades - what the best networking setup?

Hi we are in the process of setting up flex 10 in our environment also. We have a C7000 chassis with two flex10 modules in bay 1 and bay 2 each with 4 10GB ports. On the Cisco VSS side we have created a 8 port trunk. On the flex 10 side we want to do something similar; that is one big 80GB trunk. We have been told by HP that it is not possible because you cannot aggregate ports from bay 1 and ports from bay 2. Not sure it that is a true statement. Do you have any input on this? This environment is mainly for our new VMWare ESXi Farm and we are trying to come up a config that can give us the maximum bandwidth. Any input is appreciated.

 

Bart_Heungens
Honored Contributor

Re: ESX(i) Flex-10 and Blades - what the best networking setup?

Hi

 

First I wouldsuggest to create to trunks of 40Gb, so 1 for module 1 and another for module 2...

Inside VC you need to create also 2 Ethernet networks...

This is to be able to create a active/active environment... If you bundle ports from multiple modules in 1 trunk, you will get active/passive connections...

Please check which setup you exactly want in the VC Ethernet Cookbook: http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01990371/c01990371.pdf

 

Further on VC you do not need to create anything, on Cisco side you need at least an EtherChannel...

Best practices of configuring the CIsco switches can be found in the Virtual Connect for the Cisco Administrator document: http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01386629/c01386629.pdf

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
jorgemgonz
Occasional Visitor

Re: ESX(i) Flex-10 and Blades - what the best networking setup?

Hi there. This is question for anyone out there with experience installing ESX(i) 4 or 5 on HP Blade Systems.

 

I have experience with BL460's but I have a questions regarding the Full height Blades. We are currently deploying HP BL685c G7 Blades - 8 per c7000 chassis. And plan to run ESXi-5-U1 on the blades. We realize that the BL685's come with 2 dual port CNAs -  which yields 4 LOMs per blade. If we carve out all of the possible bandwiths 4 per LOM then we are left with 16 Flex NICs. 2 Flex Nics will be used for FC.

That leaves 14 Ethernet Flex NICs that can be presented to the ESXi5 OS.

 

If we present all 14 Flex NICs to ESXi5 as VMNICs are we in violation of the VMware-ESXi5 Configuration Maximum of 8 10Gbe NICs allowed to be presented to ESXi5?

Would we be in an VMware Un-supported configuration?

 

Is it best to just carve out the required bandwith and only present 8 ethernet Flex NICs Max to the ESXi5 OS?

 

Thank you inadvance for your help. - Jorge