ESX Vmotion network and BL460c G6 Blades

[Lash J]

Infrastructure is as below:

1 x C7000 Enclosure
12 x BL460c G6 Blades
Active + Standby OA Modules
2 x VC Enet Modules
2 x VC FC Modules
OA Firmware at 2.60
VC Firmware at 2.12


Blades running either ESX or XEN.

Currently we've got only 1 Shared Uplink network configured connected to a 1GB Switch.

We want to create a Vmotion network to be used by the blades, but use the bandwidth available between the blades through the chassis, i.e not making the Vmotion network go through the 1Gb Shared uplink.

(a) Is this possible?

(b) Anyone know the specifics of making such a connection in Virtual Connect so as to use the bandwidth of the chassis only the Vmotion Network?

(c) Keep in mind that currently, there is only 1 shared uplink connected to the Virtual Connect, and no other connections can be made. [ Limitations at Networking Dept. level side ]

Cheers for any help.

[JKytsi]

a) yes

b) create a network without any uplinks.

c) hope you have flex-10 vc modules, because You need more NICs than two.

[Lash J]

Hi Jarko,

I actually thought about something ALMOST similar to what you suggested. and yes they are Flex-10 VC modules.

So I've already got my one Shared Uplink with Vnets being used by ESX and XEN for networking.

I created another Network, but when I was at the part of selecting the uplink, I actually selected an Uplink that didn't have any cable physically connected to it.

I didn't know that I could create a network in VC without actually having to specify an uplink.

So, in my case, I got a Red Cross error saying it's not linked and is unavailable.

That Unavailable network was allocated to a Vmotion network in ESX, and there it showed the connection.

However when we tried to PING between 2 servers on same Vmotion network, there was no reply.


SO, tomorrow, I will try and create the network without any uplinks and see if it takes it.

Thanks for the help.

Will assign you points once I test it out.

[Michal Kapalka (mikap)]

hi.

check this document :

http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

page 6 ==> Uplinks

mikap

[Adrian Clint]

Abdool
1. Make sure you DO NOT select "private network" in the VC network you create without uplinks.

That will guarantee you will not have VMOTION traffic

2. Make sure that you dont have SFPs in all the uplink ports 7&8 on both modules. With SFPs in those there is no local communication within the blade enclosure.
You must have a 7/8 port with no SFP in each module.(or an external SFP cable linking them both - but that would be silly and unrequired)

[Lash J]

Update this morning.

The setup we did in Virtual Connect last night actually is working. Even if in VC it shows the link as unavailable, and the profile on blades as degraded.

Withing VMWare, it detects the connection as connected, and the ESX guys confirmed that they have been able to make use of the Vmotion network as it is.

---

Now we will try the option proposed by Jarko, i.e: creating another network, this time WITHOUT selecting an uplink for it, and then try out.

Hopefully, with this setup, there won't be the 'degraded' sign next to the VC Device Profiles, although it is actually working as it should now.

Thanks for helping.

[JKytsi]

I do it always without uplink, so there is nothing to be degraded =) When You map the vMotion network for one/two flexNICs your ESX does not know the difference.