P4500G2 -- NIC bonding

[George Vardikos]

Hello,

In another post, a user stated that "If you Bond the nics and use ALB bonding you should be able to achieve 2Gbps read speeds and 1 Gbps write speeds.".

I would like to know if there is a way (and if anyone has done it) to achieve 2Gbps read and 2Gbps write speeds too. Is there anything I'm missing here?

Also we have 2 units which are on network raid 10 with their 1Gbit links bonded (2 bonds, one for each unit). When testing with various VMs along the hypervisors, we are able to get up to 4Gbps of read speeds (total) but can't exceed 1Gbps write speed (total)

Sincerely,

George Vardikos

[oikjn]

Th limitations with ALB are with how ALB functions.  Do some searches on ALB and LCAP and you can see the difference and understand the advantages/disadvantages of ALB.  If you really want true 2/2 you have to go LCAP.

As for the read/write performance limitation, keep in mind that a write action in raid one is actually two writes and a write confimation so there is more overheard than a read.  A read on NR10 can be read from one of two nodes whcih means your read capacity is double, while a write has to occur on two nodes which means the data has to be written twice and confirmed between the nodes before a write is completed which is a significant overheard.  Its a design decision on the way the SAN was designed to get the reliability it has.  Basically for NR10 you get the read performance of raid0, but the write performace of raid1 and the capacity of raid1.  In addition, the latency penalty for raid1 is magnified because the network latency is added on top of the disk-disk latency so its a bit slower on write than traditional hardware raid1.

[Sbrown]

Can you explain more on this.

Would the client actually know its locality? I thought a read request would go to 1 specific node only? (assuming no MPIO/RR).

If you had 3 nodes with network raid-1, would every node have a copy of the data (no?) what happens now?

Same scenario but writing, if the data is striped on node1/2 and node3 does the write, what happens?

Without the client having knowledge of the data location, you would have a scaling issue with network raid-1 above 2 nodes.

With 2 nodes, you would always talk to a node that has the data with network raid-1.

Now abstract this to a VSA where hardware acelerated 10/40gbe nic's can do vswitch loopback.

IE. VSA01 and VSA02 - network RAID-1. Vm's running on each host as well. With two VSWITCH (one for ISCSI software client) and one for the VSA01, typically with a standard nic the data would have to go out to the switch and back.  Many modern 10gbe (some gigabit) have acceleration in some cases where it can loopback the data without hitting the network.

In that case it would go VM1 -> VSWITCHiscsi -> VSWITCHvsa -> VSA01 with 100% locality.

Without Cache Coherency (think NUMA), you waste a ton of bandwidth talking to the wrong node.

[oikjn]

read the literature on network raid there is mroe than I can put into a post. 

Basically, network raid 10 stores two copies of the data, if you have three nodes, your data is split across will look something like this (AB) (AC) (BC).  Effectively each node you add (assuming node size is the same on all) will increase your effective NR10 storage by 1/2 the node capacity.

That said, if you use the HP DSM, it knows what nodes the data should be located on and will talk to the nodes directly which makes the gateway only for administrative connections and means performance pretty much scales in line with the number of nodes you add to the cluster.  ESX I believe forces the traffic through the gateway which load balancing is really only at the LUN level for each ESX host and then the bandwidth of the acting gateway node for the LUN/initiator can become a limiting factor in scalability.   The ESX implimentation IS inefficient in its use of both storage space and bandwidth, but that is the price you pay for the resilliancy you get from the VSA system.