topic Using 10Gb uplinks in BladeSystem - General

Using 10Gb uplinks

The Brit — Mon, 19 Jan 2009 20:00:59 GMT

We are considering abandoning the 1Gb uplinks from our VC modules to the network, and using instead, the CX4 10Gb uplinks.

Are any of you out there using this configuration, i.e. CX4 only. We would be interested in any observations you might have. Were there any limitations or considerations which had to be taken into account?

Is anyone mixing VMWare with other more general use servers in the same enclosure. I am given to understand that VMWare has very specific requirements regarding vLAN tagging. Is anyone running such an environment and can throw any light on any observations or restrictions they have encountered.

thanks.

Dave.

Re: Using 10Gb uplinks

karim h — Tue, 20 Jan 2009 04:37:17 GMT

hey dave,
i can't speak specifically to the coax CX4 connections but we use the 10gb XFP connections and they are only good to about 3.6Gbps maximum under observed load testing. CX4 can cover a maximum for 300meters from memory.

We are mixing ESX with general servers in a couple of enclosures and they do need their own physical interfaces. At a minimum you will need your shared uplinks for your normal app servers and direct bindings to physical interfaces for your ESX hosts - they can't be part of the shared uplink because of the way VMWare utilises VLAN tagging.

To be honest, we only run dev/test ESX hosts in our chassis because of the limitation of the number of physical NICs you can present to a blade. Add to that the need to seperate DATA/VMKERNEL/MGMT onto physically seperate chassis NIC interfaces makes it all the more complicated.

The new Virtual Connect Flex-10 modules appear to overcome these problems and would be perfect for ESX.

rgds,

Re: Using 10Gb uplinks

Adrian Clint — Tue, 20 Jan 2009 09:27:56 GMT

The HP quickspecs say only lengths of upto 15m are supported for the CX4 cables.

They only provide part numbers for 0.5,1,3,15m lengths of CX4 cable. Even though previously they used to also show the part numbers for the 5,7 & 10m
And I dont think these parts have been discontinued so I wonder why they are not listed anymore.

For Support I'd stick with -
HP .5m 10GbE CX4 Cable 444477-B21
HP 1m 10GbE CX4 Cable 444477-B22
HP 3m 10GbE CX4 Cable 444477-B23
HP 15m 10GbE CX4 Cable 444477-B27

Re: Using 10Gb uplinks

Julian Stenning — Tue, 20 Jan 2009 14:52:38 GMT

I'm using the 10Gbe links and having blades running Windows2003 and VMware ESX in the same IP space.

If I remember correctly, we have VC1 & 2 running a pair of CX4's providing our 'front' interfaces, bundles 1Gbe connections in VC5 & 6 we use for iSCSI/NFS and another pair of CX4's in VC7 & 8 providing interfaces for virtual machines only. Blades are BL460c's with a quad port mez card.

Again, if I remember correctly, we use a shared uplink for the VC1/2 connectivity and present a VLAN directly to the vmware servers, so no vlan tagging is needed server side.

Re: Using 10Gb uplinks

Bill.Clark — Wed, 18 Feb 2009 16:04:59 GMT

Just an FYI. If you want to get past the 3.5 GB limit you are at you must enable Jumbo Frames on the server and switch. That should get you up into the 7 - 8 GB range.

Re: Using 10Gb uplinks

The Brit — Thu, 20 Aug 2009 12:32:17 GMT

Thanks for the information.