Local disk much faster than EVA8400

[Manfred Schilling]

BL460 G6 + LPe1205 8Gb + EVA8400 32xFC450
Win2003 x64 SP2 + KB945119
Emulex FC storport 2.20.006 (cp010982)
MPIO 4.00

IO-Performance: 220 MB/sec (read), 90 MB/sec (read+write) on SAN-LUN

Unbufferd write-IO, recsize = 2k: local disk 50 MB/sec; SAN-LUN 15 MB/sec !!!

any idea ???

[Víctor Cespón]

If you're writting 2 KB packets and get 50 MB/s, then you're writting 25000 packets per second.

Those are random or sequential writes?
You cannot get 25000 IOPS in the local disk of a blade unless it has a SSD.

[Manfred Schilling]

Hi,
I think you're right. "spotlight" shows this, perhaps it's the cache-modul in the blade...
Still the question, why is SAN slower than local disk on small recsizes ?

(sustained) measurements are made by

iozone -i0 -t1 -r2k -s40m -+U

data local disk:
...
Record Size 2 KB
File size set to 409600 KB
Unbuffered Windows API usage.
...
Throughput test with 1 process
Each process writes a 409600 Kbyte file in 2 Kbyte records

Children see throughput for 1 initial writers = 14647.51 KB/sec
Parent sees throughput for 1 initial writers = 14629.57 KB/sec
Min throughput per process = 14647.51 KB/sec
Max throughput per process = 14647.51 KB/sec
Avg throughput per process = 14647.51 KB/sec
Min xfer = 409600.00 KB

Children see throughput for 1 rewriters = 33914.07 KB/sec
Parent sees throughput for 1 rewriters = 33803.10 KB/sec
Min throughput per process = 33914.07 KB/sec
Max throughput per process = 33914.07 KB/sec
Avg throughput per process = 33914.07 KB/sec
Min xfer = 409600.00 KB

SAN-disk:

Children see throughput for 1 initial writers = 1901.24 KB/sec
Parent sees throughput for 1 initial writers = 1900.86 KB/sec
Min throughput per process = 1901.24 KB/sec
Max throughput per process = 1901.24 KB/sec
Avg throughput per process = 1901.24 KB/sec
Min xfer = 409600.00 KB

Children see throughput for 1 rewriters = 1896.13 KB/sec
Parent sees throughput for 1 rewriters = 1895.70 KB/sec
Min throughput per process = 1896.13 KB/sec
Max throughput per process = 1896.13 KB/sec
Avg throughput per process = 1896.13 KB/sec
Min xfer = 409600.00 KB

[Víctor Cespón]

You're creating a file writting 2 KB sequential blocks. This way you will be limited by the buffer size on the HBA. There are parameters on the Emulex drivers to configure the send window size.

Try iozone -i2 -t8 -r2k -s400m -+U

with MPIO set to SQST load balancing.

Also, investigate what kind of read/write pattern your applications will need. It's not the same an EVA hosting an Exchange server than one hosting VmWare virtual machines.

Using 2K I/O size is very unusual and not representative of a real world application.

[Manfred Schilling]

It's a Sybase dump-utility used for backup (to SAN-disk). I've been told, recsize depends on 2k DB-blocksize and couldn't be changed.
Backup-dump takes 4,5 h when utility is run in compressed-mode. With compression turned off, backup takes only 1 h (needing double disk space). Funny: double size of data speeds up backup by 4,5 x !!! (looks like streaming on tapes).
MPIO is set to SQST (we've already tried all other load balance policies).
Can I change "send window size" of the Adapter in HBAnyware, and to what value ?