topic Very slow performance on p4500 Nraid in StoreVirtual Storage

Very slow performance on p4500 Nraid

Prasanth_VA — Wed, 30 Nov 2011 09:24:39 GMT

We have three new p4500 storage nodes in which i want to configure Nraid5 and Nraid10 volumes.

But i noticed that, the performance reduce when i add these 3 nodes in same cluster.

We have existing single node clusters which are giving throughput of 60 MB/s

But the new cluster with these 3 nodes giving me throughput of 900KB/s !!!!

When i remove 2 nodes from this cluster and made it as singlenode cluster, it start perform well !!

I tried all possible ways of Nraid ( 0 , 10 and 5 ).

Note:- We use 10G Network cables for Storage as well as blade servers.

I wonder, what can be the issue?? ... all suggestions are much appreciated

Rgds

Prasanth

Re: Very slow performance on p4500 Nraid

M.Braak — Wed, 30 Nov 2011 22:27:25 GMT

Wow! Two way replication slows things down a lot but not so much.. could it be that one of the two additional nodes has an hardware problem? (Cache disabled) Please look at the performance counters of the nodes and compare the values especially latency and iops.

Re: Very slow performance on p4500 Nraid

Robert Lennie — Thu, 08 Dec 2011 21:40:26 GMT

Network RAID will NOT slow down the nodes by any means measurable to most users. Certianly not to the extent that Prasanth is speaking to.

Yes there is going to be a big difference in performance between NR5 and NR10 - but that so much between NR0 and NR10.

Are all the nodes the same disk type? MDL SAS and SAS should not be mixed in the same cluster for example. Otherwise it is likely a networking (are all the NICs on all the nodes configured the same? flow control on, for every node?) or hardware problem

Re: Very slow performance on p4500 Nraid

lpoggio — Tue, 07 May 2013 10:41:42 GMT

HI there,

I'm having a similar problem.

We have four P4500 lefthand and with syncronous replication enabled we get just about 900Kb.

How did you solve this problem ? Did you find which was the root cause ?

Thanks

Re: Very slow performance on p4500 Nraid

oikjn — Tue, 07 May 2013 15:06:34 GMT

I would check the following:

1. your nodes for a problem

2. your management group bandwidth. is it set at default?

3. check your switches to make sure flow control is on

4. check all nics/ports to make sure you are matching your frame sizes through the nodes/switches/initiators

5. look for network transmit errors on your switches.

Re: Very slow performance on p4500 Nraid

Hermanbrood — Wed, 02 Apr 2014 17:34:29 GMT

Same problem here, got a brand new P4500 series Storevirtual SAN. Hooked it up to two 10 Gbit switches, one server node: 750 KB/sec write performance. No matter what I did, it kept performing very very slow. Driving me nuts at the moment.

HP support looked into it today. I'll let you know what they found.

Re: Very slow performance on p4500 Nraid

Hermanbrood — Tue, 15 Apr 2014 14:12:21 GMT

HP support came with a solution: Buy new SFP+ cables. We support 1 meter versions and 15 meter versions of your cable type, but not the 3 meter versions. Also, it needed to be active SFP+ according to HP Support. I called the distributor of HP, which was very surprised, because an active SFP+ cable is never being used with a StoreVirtual P4530.

So I did not go for this solution, because it would have cost me over 3500 euros of cabling to accomplish. I solved the problem by setting all switch and server ports to 10000/10000 full duplex, without auto negotiation. On the switch-side, I set the ports to auto-10g. A lot of improvement after that. Got 850 MBps read and 200 MBps write.