StoreVirtual Storage
Showing results for 
Search instead for 
Did you mean: 

Terrible iSCSI VSA performance?

Occasional Contributor

Terrible iSCSI VSA performance?

The SAN IQ 8.0 VSA is on a dedicated vswitch that also connects a vmkernel and service console.
The VSA has 2TB of storage composed of 16 external 146GB 2.5 10k SAS hard drives in a RAID 50, VSA has been dedicated a 2.4GHz processor core and 1GB on a lightly loaded 24 core Dell R900.

A) The initial situation consist of copying a 600GB virtual machine to the 2TB VMFS storage (before it is used to house the VSA vmdk) at the console with the "cp -a" command. The 600GB is being copied from slower 7.2k SAS drives. The rate of transfer is at 140MB/s which takes about 75 minutes and I believe is at the maximum read rate of the vm source.

B) The poor performance situation occurs when similarly copying the 600GB at the console to the same 2TB volume but as an iSCSI export by a VSA. THE TRANSFER TAKES 11.5 HOURS with the write rate never exceeding about 50MB/s and alternating frequently to 0MB/s.

Attached is the network usage performance chart for the VSA during this time.

A) What can I do to speed IO throughput up?
B) What does the network chart typically look like for a sustained transfer?
c) What is the typical IO throughput performance of a VSA on same vswitch as vmkernel?

Thanks for your assistance.
Occasional Contributor

Re: Terrible iSCSI VSA performance?

Hi Guys,
I really like the VSA environment and easy of use, but I must be doing something wrong. One of the problems is a lack of data to compare the observed performance against.
In general, I wouldn't be concerned about sequential write performance, but the difference in rate from 140MBps to 25MBps was so startling. We purchased 10 VSAs and I have TeraBytes of data to transfer, but I'm quite concerned about performance.
I'm looking forward to some helpful tips or insights. Thanks.
Trusted Contributor

Re: Terrible iSCSI VSA performance?

Trouble shooting VSA performance requires a little more data first.
Are there other VMs running off the same disks?
Other VMs running off the same datastore?
Is this just one VSA?
Or a cluster?
Cluster of how many?
Replication level of the volume?
Is there a battery backed write cache on the controller used behind the VSA?
You say the CPU and Memory are "dedicated" do you actually mean "reserved" as the VSA requires?
Adam C, LeftHand Product Manger
Occasional Visitor

Re: Terrible iSCSI VSA performance?

My 2 cents

You should probably consider running your VSA on a dedicated server, or at the very least a dedicated set of disks and controller. If other VMs are accessing the same disk then you will not get optimal performance. It won't matter whether or not you reserve CPU and memory, because there is contention for disk resources.

In my experience, the VSA doesn't require much in the way of CPU cores or memory in order to perform well. Even the full on NSM modules use a single dual core CPU and only a couple GB RAM.
Occasional Visitor

Re: Terrible iSCSI VSA performance?

Might be a little late but I had the same issue.

It turned out to be a networking issue relating to the VMware Nic Teaming setup.

If you have the 'Network Failover Detection' set to 'Beacon Probing' then performance sucks!

Change it to 'Link Status only' and see how you go.

BTW I am running ESX 3.5 U4 on a four node Dell Blade Chassis (1955)with three VSA's.