Using 10Gb uplinks

[The Brit]

We are considering abandoning the 1Gb uplinks from our VC modules to the network, and using instead, the CX4 10Gb uplinks.

Are any of you out there using this configuration, i.e. CX4 only. We would be interested in any observations you might have. Were there any limitations or considerations which had to be taken into account?


Is anyone mixing VMWare with other more general use servers in the same enclosure. I am given to understand that VMWare has very specific requirements regarding vLAN tagging. Is anyone running such an environment and can throw any light on any observations or restrictions they have encountered.

thanks.

Dave.

[karim h · Accepted solution]

hey dave,
i can't speak specifically to the coax CX4 connections but we use the 10gb XFP connections and they are only good to about 3.6Gbps maximum under observed load testing. CX4 can cover a maximum for 300meters from memory.

We are mixing ESX with general servers in a couple of enclosures and they do need their own physical interfaces. At a minimum you will need your shared uplinks for your normal app servers and direct bindings to physical interfaces for your ESX hosts - they can't be part of the shared uplink because of the way VMWare utilises VLAN tagging.

To be honest, we only run dev/test ESX hosts in our chassis because of the limitation of the number of physical NICs you can present to a blade. Add to that the need to seperate DATA/VMKERNEL/MGMT onto physically seperate chassis NIC interfaces makes it all the more complicated.

The new Virtual Connect Flex-10 modules appear to overcome these problems and would be perfect for ESX.

rgds,

[Adrian Clint]

The HP quickspecs say only lengths of upto 15m are supported for the CX4 cables.

They only provide part numbers for 0.5,1,3,15m lengths of CX4 cable. Even though previously they used to also show the part numbers for the 5,7 & 10m
And I dont think these parts have been discontinued so I wonder why they are not listed anymore.

For Support I'd stick with -
HP .5m 10GbE CX4 Cable 444477-B21
HP 1m 10GbE CX4 Cable 444477-B22
HP 3m 10GbE CX4 Cable 444477-B23
HP 15m 10GbE CX4 Cable 444477-B27

[Julian Stenning]

I'm using the 10Gbe links and having blades running Windows2003 and VMware ESX in the same IP space.

If I remember correctly, we have VC1 & 2 running a pair of CX4's providing our 'front' interfaces, bundles 1Gbe connections in VC5 & 6 we use for iSCSI/NFS and another pair of CX4's in VC7 & 8 providing interfaces for virtual machines only. Blades are BL460c's with a quad port mez card.

Again, if I remember correctly, we use a shared uplink for the VC1/2 connectivity and present a VLAN directly to the vmware servers, so no vlan tagging is needed server side.

[Bill.Clark]

Just an FYI. If you want to get past the 3.5 GB limit you are at you must enable Jumbo Frames on the server and switch. That should get you up into the 7 - 8 GB range.

[The Brit]

Thanks for the information.