StoreVirtual Storage
1748123 Members
3304 Online
108758 Solutions
New Discussion юеВ

Re: Poor performance P4500 Campus SAN with 4 x switches

 
SOLVED
Go to solution
Paul Large
Occasional Contributor

Poor performance P4500 Campus SAN with 4 x switches

We have a P4500 campus SAN that was originally connected to 2 x 3Com GB switches with 2Gb trunk connection using VMware Round Robin and two GB connections per ESX host - 1 switch per site. To provide switch redundancy at each site, we added a second switch to each site - Site 1 now has 2 x switches with 2Gb trunk, site 2 has the same and we have a separate 2GB trunk linking the sites. when we move P4500 so that one GB connection goes into each switch the performance decreases significantly. If we move both GB connections to the same switch then performance is much better.

any ideas?
7 REPLIES 7
Cajuntank MS
Valued Contributor

Re: Poor performance P4500 Campus SAN with 4 x switches

Have you read the best practice for performance article dealing with jumbo frames, adaptive load balancing, and flow control? http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01750150/c01750150.pdf?jumpid=reg_R1002_USEN
Paul Large
Occasional Contributor

Re: Poor performance P4500 Campus SAN with 4 x switches

Yes, I have LACP for 2GB connection (2 x 1GB) between the switches and Flow Control enabled on all switch ports. Does flow control need to be enabled anywhere else - on the P4000 for example?.

Have not enabled Jumbo frames as I have read that it rarely makes any difference.
Cajuntank MS
Valued Contributor
Solution

Re: Poor performance P4500 Campus SAN with 4 x switches

Yes. Flow control must be an end to end implementation. NICs on SAN nodes, ports on switches, and NICs on servers.
You said you have LACP enabled between your switches but did not say what you were doing on your SAN...ALB? LACP?

On the Jumbo Frames part, some switches don't play nice with Jumbo Frames and Flow Control turned on at the same time. I have HP Procurve switches which I have not had an issue with myself.
Paul Large
Occasional Contributor

Re: Poor performance P4500 Campus SAN with 4 x switches

OK, I did not have Flow Control enabled on the P4000 which I have now enabled. Initial tests show a great improvement in performance :)
Jitun
HPE Pro

Re: Poor performance P4500 Campus SAN with 4 x switches

Here is a small list that I check...
Please check on the Node, Switch and Host Server
1.) Enable Flow Control on network switches and adapters. Flow control ensures a receiver can make the sender pace its speed and is important in avoiding data loss. (Recommended)
2.) Ensure Spanning Tree Algorithm for detecting loops is turned OFF. Loop detection introduces a delay in making a port become usable for data transfer and may lead to application timeouts.
3.) Segregate SAN and LAN traffic. iSCSI SAN interfaces should be separated from other corporate network traffic (LAN).

Networking best practices
├в ┬вUse Non-Blocking switches and set the negotiated speed on the switches.
├в ┬вDisable unicast storm control on iSCSI ports. Most switches have unicast storm control disabled by default. If your switch has this enabled, you should disable this on the ports connected to iSCSI hosts and targets to avoid packet loss.
├в ┬вServers should use dedicated NICs for SAN traffic. Deploying iSCSI disks on a separate network helps to minimize network congestion and latency. Additionally, iSCSI volumes are more secure when├в ┬ж Segregate SAN & LAN traffic can be separated using port based VLANs or physically separate networks.
├в ┬вConfigure additional Paths for High Availability; use either Microsoft MPIO or MCS (multiple connections per session) with additional NICs in the server to create additional connections to the iSCSI storage array through redundant Ethernet switch fabrics.
├в ┬вUnbind File and Print Sharing from the iSCSI NIC ├в on the NICs which connect only to the iSCSI SAN, unbind File and Print Sharing.
I work for HPE
--------------------------------------------------------------
How to assign points? Click the KUDOS! star!

Accept or Kudo

Snakehead
Occasional Visitor

Re: Poor performance P4500 Campus SAN with 4 x switches

Hi,

 

I got P4500G2 x 2, ESXi5 x 2, H3C switch.  All running @ 10GB.

P4500G2 network raid 10, (802.3ad bonding).

ESXi5  x 2 with software iscsi, storage path (RR).

H3C switch using (IRF=Stack).

 

We have 2 guest windows to copy data and delete from each another.  It just a simple test, however I see the disk queue length over 6 on the CMC monitor.  Besides we found the error message of the disk lantency increased come out frequency.

 

Please help...

David_Tocker
Regular Advisor

Re: Poor performance P4500 Campus SAN with 4 x switches

A disk queue lenght over 6 is not probably something to worry about. We have one cluster in our setup (P4300 g2 X 2) that can hit upto about 80 on the queue lengh and the customers do not seem to notice. Real world testing is always going to better than numbers in the CMC. We have found that we can run upto about 300 RDS users with their SBS servers on a single pair of P4300s without the customers reporting any performance issues. Throughput seems to be much less important than latency, make sure you have flow control enabled on the iSCSI ports on your VM hosts and the P4000 ports on your switches. Jumbo frames appear to cause issues on the procurve 2810 switches, i dont know about the 2910 als?

Regards.

David Tocker