StoreVirtual Storage
1748003 Members
4687 Online
108757 Solutions
New Discussion юеВ

Re: Copying data from LeftHand to a client system abnormally slow

 
Andrew Kaplan
Super Advisor

Copying data from LeftHand to a client system abnormally slow

Hi there --

 

We have a LeftHand system set up where for the past several months application data has been written to it. We are still adding data to the server, but we are now also using that data in the application on client workstations. We have discovered that retrieving the data from the system has been very slow. The server that is connected to the LeftHand is running Windows 2008R2 64-bit, and has SQL Server 2008R2 64-bit as its database. The database itself runs several instances of databases as part of the application. The client systems run Windows 7 64-bit as their respective operating system. The server itself recently had two of its NIC's teamed up to provide the equivalent of a dedicated two-gigabyte connection.

 

A series of tests were run to determine where the fault lies. These tests involved copying a two gigabyte .iso file, and include the following:

 

1. Copying the file from a client system to a network drive on the LeftHand.

2. Copying the file from the LeftHand to a client. This test was done on several clients.

3. Copying the file from one client to another client.

 

The results of the tests were as follows:

 

1. The transfer rate from the client to the LeftHand was witnessed at a 450Mpbs transfer rate.

2..The transfer rate from the LeftHand to the client was seen at a 15Mpbs speed.

3. The transfer rate from client to client was at a speed comparable to that from a client to the LeftHand system.

 

The LeftHand appears to be configured to optimize writes to it, rather than reads from it. We need to have a configuration

where the reads are done at a much faster rate. If that is not the case, what other possible causes are there to this problem?

 

Thanks.

A Journey In The Quest Of Knowledge
5 REPLIES 5
oikjn
Honored Contributor

Re: Copying data from LeftHand to a client system abnormally slow

Andrew, you are going to need to provide a ton of additional information.

 

1.  exactly what lefthand system are you running and what raid/network raid levels?

2.  are the windows 7 machines connecting using iSCSI or through some other source?

 

 

IF the P4000 system is the problem, you will see it very easily by monitoring the system through your CMC console.  Watch the performance monitor there ans check each node and the overall cluster for CPU, latency, IOPS, throughput.

 

Side note:  copying a large ISO is absolutely not a good way to test a system performance for a database application.  databases are typically lots of small random transfers while an ISO transfer is a large sequential transfer... those are two totoally different beasts.  Also, if you are connecting with a windows 2k8r2 machine, the windows performance monitoring default profile as seen through the task manager is a real easy way to watch disk activity, throughput and latency as seen by the windows server.

M.Braak
Frequent Advisor

Re: Copying data from LeftHand to a client system abnormally slow

Writing to the san is optimized because of the write cache of the array controllers. About the read performance, saniq does a very bad job. HP also acknowledges this. Saniq is highly optimized for random io. Saniq effectively limits sequential read performance!!!! So this is why you get very slow throughput when reading large files, doing backups etc. Every single io stream is limited by saniq. You can test this by yourselve. Now you can read the iso at 15 MB/s. Please start two read sessions simultainously. You shall see you can read at 2 x 15 MB/s. When you start three sessions you will get 3x15 MB/s. This is why we decided to move off vmware templates off the san to other storage fir example.
oikjn
Honored Contributor

Re: Copying data from LeftHand to a client system abnormally slow

not that I don't doubt that the system design is read-crippled, I'm watching my really under-powered software VSAs pulling 50MB/s on each of three snapshots used to backup to my DPM server.  Its all a function of IOPS/latency/CPU and as long as the disks can provide the data stream and the CPU isn't maxed out the system should keep moving faster and faster until one of those stats becomes the limiting factor.

Andrew Kaplan
Super Advisor

Re: Copying data from LeftHand to a client system abnormally slow

Hi there --

 

Thanks for everyone's reply. To answer the questions posed by our colleague:

 

1. The LeftHand system that is being used is the P4500G2 series with RAID5 using SAS disks. The version of software is the 9.5.00.1215.0 release. We have network bonding set up for two of the cards to provide, in theory, two gigabytes of network bandwidth.

 

2. The Windows 7 workstations are not directly attached to the LeftHand system. They have client software which connects over the network to the server application.

A Journey In The Quest Of Knowledge
oikjn
Honored Contributor

Re: Copying data from LeftHand to a client system abnormally slow

your answers will be in the CMC perfmon and the windows performance monitoring on the windows server. 

 

I just tried a 2GB file transfer on file server that has its LUN connected to the lefthand system.  It took about a minute to completel and averaged a bit over 50MB/s as seen on both the CMC and on my windows7 desktop.

 

 

 make sure you are using the HP DSM and it is setup correctly on the windows server that is directly connected to the lefthand system.

 

I'm guessing you might have a network setting issue such a network flow control configuration problem.  My suggestion would be to read the manual since it goes through all the configuration suggestions and how to test/throubleshoot.