topic Re: EVA 3000 performance too slow in Disk Enclosures

EVA 3000 performance too slow

medali.benchaabane — Thu, 17 Aug 2006 12:04:53 GMT

Hi all,

I have an EVA 3000.
My applications are too slow.
when I have run
sar -d 2 10 , I saw that some disks are 100% busy.
My backup speed is running between 700KB/s and 2000KB/s which is too slow for an EVA.
I saw on technical documentation that the EVA can deliver up to 300MB/s when I'm under 2M/s.

My EVA is 4 enclosure 56 disks 146G 15K tours/m

Could you please help me

Re: EVA 3000 performance too slow

Ivan Ferreira — Thu, 17 Aug 2006 14:11:49 GMT

Well, is hard to tell where is the problem because the application "may" be doing a lot of small I/O access, that increases teh I/O wait and reduced the throughput. The backup process is not described.

You "could" do some performance test by creating a vdisk, presenting to the host and do "dd" to read and write data, for example:

time dd if=/dev/ of=/dev/null bs=8192 count=131072

This is a destructive test:

time dd if=/dev/zero of=/dev/ bs=8192 count=131072

In this way, you can check the performance counters again and view the time it takes to read and write the data by using a larger block size and fast I/O.

Re: EVA 3000 performance too slow

medali.benchaabane — Fri, 18 Aug 2006 05:29:39 GMT

Hi,

Thanks for your answer, I got the following output:
# time dd if=/dev/test_perf/lvol1 of=/dev/null bs=8192 count=131072
131072+0 records in
131072+0 records out

real 0m55.81s
user 0m0.14s
sys 0m12.73s
# time dd if=/dev/zero of=/dev/test_perf/lvol1 bs=8192 count=131072
131072+0 records in
131072+0 records out

real 1m30.71s
user 0m0.15s
sys 0m15.95s
#

how can you qualify it ?
I don't know how to interpret it

Re: EVA 3000 performance too slow

Ivan Ferreira — Fri, 18 Aug 2006 10:41:59 GMT

In this command:

# time dd if=/dev/test_perf/lvol1 of=/dev/null bs=8192 count=131072
131072+0 records in
131072+0 records out

You READ from the VDISK and write to "nothing", that is /dev/null. You perform 131072 operations with a 8k size, that is 1 GB. The operations took ~ 56 Secs. So, you are reading with a performance of about 19 MB/s.

In the second command:

# time dd if=/dev/zero of=/dev/test_perf/lvol1 bs=8192 count=131072
131072+0 records in
131072+0 records out

real 1m30.71s
user 0m0.15s
sys 0m15.95s

You read from /dev/zero (special file that delivers only zeros) to your VDISK. Again you write 1 GB in 1m 30 secs. That is, your performance is more or less 11 MB/s.

EVA can deliver 300 MB/s, but is not specified under what conditions and tests.

As you can see, the EVA can deliver more that you are getting from your application. But now the question is, is your performance good enough? Well, this test demostrates that can perform better, you can run the test with 64 k block size to increase the performance. You can find more information in this thread:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=979480

Re: EVA 3000 performance too slow

Cavailhe — Sun, 20 Aug 2006 10:49:16 GMT

Hi

If your EVA3000 has one DG with 56disks 15KRPM, so EVA can give 8400 IO/s (56*150IO/s) when the IO is very short. When the IO is more sized, so EVA cache plays and EVA gives a big thruput.

When you does backup, IO size is important.
For example I seen from 180MB/s to 220 MB/s during 4hours on one EVA port during a backup window from EVA 16disks*10KRPM in a DG to LT02 drive.

So, normally EVA performance is not the issue, it's other thing.

Best regards.
Olivier Cavailhe

Re: EVA 3000 performance too slow

Michael Steele_2 — Sun, 20 Aug 2006 20:44:08 GMT

You have a disk bottleneck on these four disks.

09:31:08 device %busy avque r+w/s blks/s avwait avserv
Average c27t2d2 100.00 5.21 1264 42590 9.54 6.75
Average c27t4d2 7.44 0.50 83 8243 5.05 1.18
Average c27t4d5 5.44 0.58 311 8534 5.11 0.44
Average c27t0d6 0.15 0.50 1 10 4.92 0.47

Can you paste in 'strings /etc/lvmtab'?

Re: EVA 3000 performance too slow

medali.benchaabane — Mon, 21 Aug 2006 02:42:05 GMT

hi, here is my lvmtab file
bscs2 root:/ #strings /etc/lvmtab
/dev/vg00
/dev/dsk/c0t6d0
/dev/dsk/c2t6d0
/dev/vgt_06
/dev/dsk/c27t0d0
/dev/vgt_07
/dev/dsk/c27t0d6
/dev/vgt_08
/dev/dsk/c27t1d3
/dev/vgt_09
/dev/dsk/c27t1d4
/dev/vgt_10
/dev/dsk/c27t2d5
/dev/vgt_11
/dev/dsk/c27t0d1
/dev/vgt_12
/dev/dsk/c27t0d2
/dev/vgt_13
/dev/dsk/c27t0d3
/dev/vgt_14
/dev/dsk/c27t0d4
/dev/vgt_15
/dev/dsk/c27t0d5
/dev/vgt_20
/dev/dsk/c27t1d5
/dev/vgt_21
/dev/dsk/c27t1d6
/dev/vgt_22
/dev/dsk/c27t2d0
/dev/vgt_29
/dev/dsk/c27t0d7
/dev/vgt_30
/dev/dsk/c27t1d2
/dev/vgt_31
/dev/dsk/c27t2d6
/dev/vgt_33
/dev/dsk/c27t3d1
/dev/vgt_35
/dev/dsk/c27t1d1
/dev/vgt_36
/dev/dsk/c27t1d0
/dev/vgt_39
/dev/dsk/c27t1d7
/dev/vgt_42
/dev/dsk/c27t3d7
/dev/vgt_44
/dev/dsk/c27t2d2
/dev/vgt_47
/dev/dsk/c27t4d3
/dev/vgt_32
/dev/dsk/c27t3d2
/dev/vgt_41
/dev/dsk/c27t2d4
/dev/vgt_48
/dev/dsk/c27t3d6
/dev/vgt_49
/dev/dsk/c27t4d2
/dev/vgt_50
/dev/dsk/c27t4d4
/dev/vgt_51
/dev/dsk/c27t4d5
/dev/vgt_52
/dev/dsk/c27t2d3
/dev/vgt_53
/dev/dsk/c27t2d7
bscs2 root:/ #

Thanks

Re: EVA 3000 performance too slow

Michael Steele_2 — Mon, 21 Aug 2006 08:59:10 GMT

When you map these volume groups to the sar report, notice that avwait is always greater then avserv.

/dev/dsk/c27t4d5
/dev/vgt_52

/dev/dsk/c27t4d2
/dev/vgt_50

/dev/dsk/c27t0d6
/dev/vgt_08

/dev/dsk/c27t2d2
/dev/vgt_47

09:31:08 device %busy avque r+w/s blks/s avwait avserv
Average c27t2d2 100.00 5.21 1264 42590 9.54 6.75
Average c27t4d2 7.44 0.50 83 8243 5.05 1.18
Average c27t4d5 5.44 0.58 311 8534 5.11 0.44
Average c27t0d6 0.15 0.50 1 10 4.92 0.47

Refer to /etc/fstab to map the file systems for these vgs and run defrag.

/dev/vgt_47, /dev/vgt_08, /dev/vgt_50, /dev/vgt_52

Here's defrag:

# fsadm -F vxfs -D -d -E -e /mount_point

Here's are reference about disk bottlenecks to refer to, it reccommends increasing your cache.

http://www1.itrc.hp.com/service/iv/docDisplay.do?docId=prodITRC/DE_SW_W2K_performance_EN_01_E/Performancenoptimization/Disk_bottleneck_analysis/disk_bottleneck_analysis.htm

In this case, the controller on the server side is your fibre channel adapter (HBA). Your HBA's can be either 1G Hz or 2G Hz. Use this command to determine.

# ioscan -fnC td (* write down the /dev/td#

# tdutil /dev/td#

Start a new thread in the HP-UX forum, paste this information in and ask for comments about changing the memory cache. One more thing, note the HW address of the HBA from ioscan and check logtool for errors:

STM > TOOLS > UTILITY > RUN > LOGTOOL > FILE > RAW SUMMARY

Please include this report if you see a lot of errors. Important to include the start and end date and time at the top of the report.

Thanks!

Re: EVA 3000 performance too slow

medali.benchaabane — Tue, 22 Aug 2006 10:58:52 GMT

I've opened new thread on HP UX forum,

I think the problem is located on the server
have a look at the test.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1053539