StoreVirtual Storage
1748274 Members
4014 Online
108761 Solutions
New Discussion

Re: Setup vSphere ESX w/ p4000 and separate Switches

 

Setup vSphere ESX w/ p4000 and separate Switches

Folks,

I want to do the following: I have two p4500G2 storage nodes with 2x 1Gb NIC each. I also have three DL380G7 with 8x 1Gb each. Last I have 2x ProCurve 2910 switches only for iSCSI. I want to create a fully redundant setup with two separate Layer1/2 networks, as I did with e.g. p2000G3 iSCSI storage:



- Network 1:

NIC1 of each p4500 node
NIC3 of each ESXi
Subnet: 10.0.32.0/24

- Network 2:

NIC2 of each p4500 node
NIC4 of each ESXi
Subnet: 10.0.33.0/24



I want to access the volumes using round-robin path policy for performance and resilience. The servers' LOMs are HW iSCSI capable and I can assign different IP addresses to each of the storage nodes' NICs.



Some of the documentation I read says, that I should bond the p4500's NICs and keep all iSCSI traffic on one subnet. But that would mean, that I have to create one Layer1/2 network instead of separating them.



Does someone have experience in doing so?
Murphy was an optimist.
9 REPLIES 9
Bart_Heungens
Honored Contributor

Re: Setup vSphere ESX w/ p4000 and separate Switches

Hi

First check this reference document from HP:
http://h41112.www4.hp.com/promo/master_ESS-Trilogy-4/uk/en/pdf/Documentations/Virtualization/Running%20VMware%20vSphere%204%20on%20HP%20P4000%20SAN.pdf
Did 10's of installations like this, works like a charm...

Both 2 NICs from all storage nodes should be split over the 2 network switches for high availability.

Then create a bond accross the 2 NIC's on the storage nodes in the CMC. Bond settings should be ALB Adaptive Load Balancing (which is actually Transmit Load Balancing)

Create 2 VMkernel interfaces inside the ESX server with 2 separate IP addresses and link them to the vmhba (follow the mentioned document)
Untill now I got the best performance with the software iSCSI adapter in ESX, compared to the hardware iSCSI capable NICs...

Finally detect the iSCSI volumes inside ESX and select Round Robin in the Path Selection of that volume...

Looks complicated but it isn't at all...
--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !

Re: Setup vSphere ESX w/ p4000 and separate Switches

Hi,
This standard setup requires that my switches are connected to each other, right? That's what I want to avoid.
Murphy was an optimist.
Bart_Heungens
Honored Contributor

Re: Setup vSphere ESX w/ p4000 and separate Switches

Indeed, the 2 switches must be interconnected with each other, best with multiple cables in a trunk.

The setup of the trunk is really easy, you don't have to be a network expert for that.
First connect the switches with one cable so that you can manage them both, create the trunk (fixed, no LACP) on both switches and add the additional cables...

Why do you not want to interconnect the switches? Had no issues with it untill now...
--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !

Re: Setup vSphere ESX w/ p4000 and separate Switches

I actually have four switches, two in each room. I have two OM3 cables between the rooms, over which I will connect two of the switches with 10Gb.
In standard setup I need to connect the 'local' switches one to another and I need to run STP and so on. I would really like to keep the 'fabrics' separate, but this seems not to work.
Murphy was an optimist.
Bart_Heungens
Honored Contributor

Re: Setup vSphere ESX w/ p4000 and separate Switches

What do you mean with: this does not seem to work?
The LAN switches and SAN switches (which is actually also a LAN switch but just to avoid confusion) should not be interconnected with each other for functionality.

2 (or more) NICs of the servers are connected to the LAN switches for regular LAN traffic, 2 (or more) NICs are connected to the SAN switches for iSCSI trafic.

Both LAN switches must be interconnected with each other, also both SAN switches.

Now, for management reasons I have seen some setups where the SAN switches had 1 uplink connection to the LAN network, with routing enabled, to do management of the SAN switches from the LAN network. If you are interested in this, let me know but that will take a little longer to explain/type...
--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
vmcreator
Occasional Advisor

Re: Setup vSphere ESX w/ p4000 and separate Switches

Hi Bart,

 

With vSphere ESXi 4.1 Update 1 and ALUA, there does seem to be some confusion, and the document referenced below is primarliy vSphere 4.0 circa 2009.

 

vSphere ESXi 4.1 Update 1 brings along ALUA and also a big improvement with iSCSI. Now HP [Lefthand] does not support ALUA, so what is the best practice for this configuration?.

 

Also the document says HP P4000 Multi-Site SAN configurations with more than one VIP canot se multipathing!

 

I think HP need to write a new white paper for this with vSphere ESXi 4.1 etc

 

What do you think.

 

Best Regards

 

Bob

 

 

Bart wrote:

 

Hi

First check this reference document from HP:
http://h41112.www4.hp.com/promo/master_ESS-Trilogy?-4/uk/en/pdf/Documentations/Virtualization/Runnin...
Did 10's of installations like this, works like a charm...

Both 2 NICs from all storage nodes should be split over the 2 network switches for high availability.

Then create a bond accross the 2 NIC's on the storage nodes in the CMC. Bond settings should be ALB Adaptive Load Balancing (which is actually Transmit Load Balancing)

Create 2 VMkernel interfaces inside the ESX server with 2 separate IP addresses and link them to the vmhba (follow the mentioned document)
Untill now I got the best performance with the software iSCSI adapter in ESX, compared to the hardware iSCSI capable NICs...

Finally detect the iSCSI volumes inside ESX and select Round Robin in the Path Selection of that volume...

Looks complicated but it isn't at all... ....

Uwe Zessin
Honored Contributor

Re: Setup vSphere ESX w/ p4000 and separate Switches

Bart,
vmcreator wants to emulate the "two Fibre Channel fabrics" rule. He wants to avoid interconnecting all iSCSI LAN switches, but, as you already said: that is a requirement for the P4000.


vmcreator,
ALUA does not apply, because the P4000 does not work like an asymmetric active/active fibre channel storage array.  And support for ALUA/RoundRobin is available in ESX 4.0 already - the big improvement was going from 3.5 to 4.0.

.
vmcreator
Occasional Advisor

Re: Setup vSphere ESX w/ p4000 and separate Switches

Ahh thanks Uwe, I understand now.

 

Best Regards

 

vmcreator

RHC1
Occasional Visitor

Re: Setup vSphere ESX w/ p4000 and separate Switches

 

Spoiler
 

 

 

Bart Heungens_1,  Could you please put the entire link to the doc you are refering to? I am in the same boat as Oliver and would love to see that setup. When I click on the link it says "document not found"

Thank you.