ESX(i) Flex-10 and Blades - what the best networking setup?

[julianwood]

We're working on our design for migrating our ESX environment to Blades and are keen to go HP Flex-10. How are you setting up your networking? We use NetApp NFS for all storage. We would have multiple VLANs for our production networking. We would have a single 10GbE uplink per Flex-10 module. We wouldn't stack together chassis. Do we have shared uplinks or just dedicated uplinks? Do we just pass through VLANs to ESX or actually set up the VLANs in VConnect? Do we get the ESX hosts to do the failover or get VConnect? Are you carving up your FlexNics or just using the single 10GbE? Do you separate LAN / NAS physically or with VLANs or not at all? Do you separate out Service Console and VMotion? Do you use portgroups to direct traffic over particular uplinks but allow failover? What Blades do you use? The 490c can take more memory but doesn't have hot swappable disks. Does it matter? You cannot currently mirror the internal solid state disks. The 460C has less memory but HSS disks, so is that not a single point of failure. 8GB Dimms are very expensive so we're thinking of going 4GB dimms and getting more blades which works out cheaper. Do you run ESX or ESXi installable or ESXi Embedded? Anyone running ESXi off the internal SD cards on a 490C? Would love to see some reference architecture or at least get everyone's thinking for how to set up vSphere with Blades.

[chopper3]

If I were you I'd just present 2 x 10Gbps VC trunk nics (although enable all 8 nics but with nothing assigned to nics 2-4 and 6-8 so you can change things later without NIC enumaration issues), let ESX do the faiover and vlan tag management. I do have different vlans for SC and VMotion. We use 490c's with a single SSD for ESX and 460 G6's for everything else (like windows and other baremetal builds), always go with 4GB mem modules, the 8's aren't worth it. I use ESX over 'i' as the management agents in 'i' are no way near as good as the regular ones. I have ran 'i' off SD, they're quite nice for that really as the slot is right at the front on the top but much harder to get to on the 460, worked ok but prefer regular esx off the SSD. I'm happy to help with anything else in this arena, I'm in the UK and I'm phil at buckley-mellor.com ok.

[Pascal]

Hello, We use ESX Host as well and the flex-10 technology in the same way as chopper3 does (vmware is doing the vlan tagging stuff). All ESX host are installed on BL460G6 blades. We use also Netapp filer and NFS to store vm datastore. We did create several volume (per application or per server type) to be able to isolate activities and performance on the filer. A dedicated volume is also created to store only vm's swap files. All other vm's files are put on deduplicated volumes using SIS (40% to 70% disk saved at the moment with around 160 vm depending on wich volume you look at). We do not run (i) because it is very limited if you want to use scripts and cli commands. Failover is done by the flex-10 and the mode you choose for your ulink (auto/failover) and transparent for the host and vm.

[julianwood]

chopper3 / Pascal do you then have only 1 x vSwitch and multiple VLANs for SC/VMotion/VM/NFS. Are your 2 uplinks setup with active / passive or do you route traffic from some portsgroups over uplink1 and other portgroups on uplink 2.

[chopper3]

Yes, we only have a single vswitch with multiple vlans, we currently use active/passive throughout but you can make them active/active if you use Cisco's VSS.

[Pascal]

We use several vswitchs on our side though it is for compatibility reason with some other non hp hardware. Active/Passive mode is also our choice, despite the fact that we have a strange behavior on the passive flex (we get some inbound traffic on it and we dont know why yet). You can use active/active with cisco'vss and 6500 series, also with Stacked 3750 switches, or with VPC and nexus 5000 (for the last one ..get some time before going to it ..the paint is not dry).

[julianwood]

So, what do you do with your 6 unused Flex-Nics. I really do think HP should have a way that you can disable these. If you don't allocate anything to them HPSIM sees them as down, previously we used to fool HPSIM by plugging in a loopback adapter set to 100Mb and the link would be sees as up. What do you do with blades?

[chopper3]

It's good that you can see them all, that way their enumeration won't change if you reconfigure the VC profile, they'll stay the same.

[julianwood]

Well other than not actually wanting to see them, I Don't want to have unused Nics being presented but if I have to live with them being present how can I get HPSIM to ignore them so they are not reported as down?

[mboanta]

Hey guys, I was actually referred to this site from an HP friend of mine. I have a SR Systems Engineer position open here in Orange County, CA and I am looking for someone that has strong experience with the following: HP Proliant series servers, c-Class blades/enclosures, enclosure interconnects/switches HP Proliant Virtual-connects (including Flex-10 VC) HP Insight Control environment for Windows (including HPSIM, RSP, RDP, VCEM) VMWare ESX 3.5, vSphere4, Virtual Center, Update manager, VMWare Convertor Please give me a call at 949-417-4818 if you or anyone you know is qualified and interested in a new opportunity. Thanks again for your time and help! Sincerely, Joe 949-417-4818 jojhenry@teksystems.com