Help! Very slow Disk I/O on G1610T Gen8 and ESXi

[Gavin123]

Hi,

I just bought and configured a G1610T Gen8 Microserver, I upgraded it with:

16 Gb

Xeon 1240V2

And added 4x disks WD Red 3Tb each

Configured a Raid 10

Installed ESXi (HP custom) vSphere 6.0 on a 16Gb SD card

Installed a fresh Ubuntu VM, installed VMware tools

Everything seemed to be good... but then I noticed things were slow...

I did probide 4Gb of Ram and 1 Virtual Socket with 2 Core to the VM.

There is no other VM and every settings are default.

Running the following command on the VM to test writting speed:

sudo hdparm -Tt /dev/sda

gives me the following:

Timing cached reads:   19800 MB in  2.00 seconds = 9907.62 MB/sec

Timing buffered disk reads: 150 MB in  3.11 seconds =  48.15 MB/sec

On an old iMac running VMware fusion 7 a similar VM gives me the following results for the same command:

Timing cached reads: 10594 MB in 2.00 seconds = 5298.41 MB/sec
Timing buffered disk reads: 1152 MB in 3.01 seconds = 383.12 MB/sec

It looks like my Buffered Disk read is having some issues on my Gen8 server...

Worse.. trying the following test on The Gen8 and iMac:

time sh -c "dd if=/dev/zero of=tmp1 bs=8k count=1000000 && sync"

on my iMac it completes in 16s

On my Gen8 it completes in over 4 minutes!

Granted, I am new to the microserver, and also new to ESX!

My experience is with vmware for desktop.

This setup is not for work/production but for a lab at home I wanted to run.

I might have missed something very obvious!

But after batteling a whole weekend learning how to set all this up (iLO, Raid, ESX), I am just about to give up and send everything back.

So disapointed! I seem to have come so close but I just can't find out what is wrong with my setup so access/write to the disk is so slow on my Gen8.

I understand there could be issues with either my Raid setup (just using the B120i) but I am using all default settings or with my ESXi config (but then again, all default settings).

If anyone can help that would be greatly appreciated. I really want this to work, but I seem to have hit a wall!

Thanks

Gavin.

[ArnisR]

It is because "new" hpsva driver (versions 090 and 092).  Downgrade to 088 solves low I/O problems.

Read further here (from 13 April) - http://homeservershow.com/forums/index.php?/topic/9141-low-io-performance-with-esxi-60/page-3

[Gavin123]

Thanks so much ArnisR!

That really helped!!

I think there is still an issue... installing the older driver really did improve the performance but it is still slower than my old computer which doesnt make sense..

I am now getting 

> sudo hdparm -tT /dev/sda

/dev/sda:

Timing cached reads:   20618 MB in  2.00 seconds = 10317.72 MB/sec

Timing buffered disk reads: 428 MB in  3.00 seconds = 142.59 MB/sec

That went from 50 MB/sec to 142 Mb/Sec

And creating the 8Gb test file went from 4 min to 2 min...

but the iMac still does it in 16s!

If you have any idea on how to improve further I am listening :)

I will also check on the forum from the link you provided.

If not completly fixed, it is already a great improvement and the wall is starting to crack :)

Thanks so much.

[ArnisR]

iMac is cheating (if it have HDD not SSD)    Even 4 fastest HDD in RAID0 cannot create 8 GB of 8k blocks in 16 sec. 1 million I/O in 16 s = 62 kIOps, it is not real for HDD.
Try /dev/urandom instead of /dev/zero. And check when really disk finished activity.

I also enabled physical write cache of disks on B120i (I have UPS).

There is also newer HP ESXi image from 30 March (hpsva driver should be reverted after upgrade)