Major Performance Issues within Fiber Channel Networks

[RBruce]

I have what I believe is a bottleneck within my SAN Switch Infrastructure. I cannot find where it is and both HP and Brocade sees no problem because they are only looking for errors.

If you can help me, that would be great.

Here is what I am seeing.

ESXTOP DAVG is much higher than 25 and 50...
SAN Switch stats never go above 100MB/s on any port.
When trying to cause a fiber storm by turning on all VMs at the same time stats never go above 100MB/s.
When backing up data from VMs or local hosts (ie. Windows 2003 Server) I never see performance increase above 100 MB/s.

Datastores are all 1 to 1. I do not share more then one VM per datastore.

My environment:

HP C-7000
4/24 fiber switch in chassis

4 BL460c G1 (32GB Memory each)
4 ESX 4.0 hosts (30 VM total)

5 BL460c G1 (4 GB Memory each)
5 ESX 4.1 hosts (8 VM total)

1 BL460c G1 (4 GB Memory)
1 ESX 4.1 Virtual Center (VC)

1 BL460c G1 (4 GB Memory)
1 Windows 2003 Server
1 Symantec Backup Exec 2010 R2

Connected to Fiber Switch
2 Xyratex SANs (Certified for ESX 3.5)
Active-Active Configuration
Each SAN has 16 disks per RAID set.
R10 SAS and R50 SATA

1 Virtual Tape Library
1 Physical Tape Library
2 fiber based LTO4 drives

Anything you can suggest would be a great help.

[Jan Soska]

Hello,
- what is your SAN speed (4Gb?)
- have you tried run some benchmark on san luns atached to your hardware vCenter server?

jan

[Rob Buxton]

What problems are you seeing?

I'm not familiar with the Xyratex, but I'd check things like the right policy has been set for this kind of array. E.g. does it support ALUA and are you using Most Recently Used or Round Robin or is it fixed.
If fixed then ensure you're going to the right controller.

[Anubhav_Jain]

it seems you are using LTO drives for VMs, which is itself can be bottlenecks. They are usually preferred to have SAN device or iSCSI. You can try to create multipath for SAN. Also, do not place al VMs in same datastore, create multiple data store else access might get slow

[Jags_21]

Hi,

Cross check if there is any issues at the SAN Switch zoning which can be causing the bottleneck as there are fiber based LTO4 drives, also the number of datastores created with conjunction of other devices connected to same fiber path could be also an bottleneck.

Click here for the mulitpath policies which can be configured on the hosts.

Thanks,

-----------

If you like this post please click on "KUDOS White Star" on the left pane of this post.