DL 380G7 - Problem with disk performace

[branko rekic]

Hi

We bought 4 Proliant DL 380G7 servers with this configurations:

4x HP Proliant DL380 G7

2 x Intel Xeon Quad Core E5620 2,40 GHz, 6 x 4 GB REG, Smart Array P410/512MB BBWC (RAID 0/1/1+0/5/5+0), 2 x 300 GB 10k rpm SAS HDD, DVD RW, 4 x 1 Gbit NIC, 8Gbit Single Port FC HBA, iLO 3 mngmnt,2 x 460 W RPS.

On that servers we installed ESXi4.1,witch are compatible for that server.(I've check on VMware and HP site).Now we have problem with disk performace...when we tried read or write something from disk we had bad result;about 50-70mb/s,what is very bad.I also increase write cache on servers but we still have same performance.Is'it necessary to configure something on that controller before use?I know that performance need to be better so I think maybe we have patch problem.I found on HP page that exist a new patch for storage controller,but I'm not sure whether this will solve our problem.Current patch is 3.52.Is it good idea to put a new version of patch 3.66?Here I found that patch:
http://bizsupport1.austin.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=4091412&prodNameId=4091432&swEnvOID=4091&swLang=8&mode=2&taskId=135&swItem=MTX-2bc3d0d5a4db40549ee6c97485

Regards

[Michael A. McKenney]

DL380 G7 is reported alot of having poor RAID performance. Is it RAID 1? 50-70 MB/s on RAID 1 is not bad. You should be only supporting ESXi 4.1 on it. Your applications should be in a SAN or on another set of RAID 10 drives.

[Terry Hutchings]

I would also recommend not overlooking the firmware for the drives. I would recommend updating both the controller and drive firmware. I don't see any fixes for any performance issues on the P410, but I do know that some of the recent drive firmwares fix some issues which might result in some poor performance.

How is the performance being measured? What is the read/write ratio set on the logical drive you have configured?

I would also recommend going into the BIOS on the server and changing the power profile to "static high performance". Sometimes when the CPU's are changing C-states it is not able to respond fast enough to interrupt requests.

[Michael A. McKenney]

I would do firmware and Option ROM on controller, drives, and server board. Backup first.

[branko rekic]

Hi

Thx for replay.
Yes,I google it about that server and many people are very dissatisfied.I was also looked all that patches witch HP provide for DL380G7 but no one told something about performace.

We tested read and write in many ways:
1.copy same data from client on datastore
2.copy same data from client on vitual machine
3.put something from datastore on vm
4.tried to read some files...(also tried all that in opposite direction)and result was bad.As I told before;about 50-70 mb/s.These are disk with 10000rpm so I think that it would be better to do.

" What is the read/write ratio set on the logical drive you have configured? "what do you think under that??
I will tried this with CPU,and I will tell you results.

If I must to updeate firmware;witch are the backup procedures?and also what consequences that can we have if something goes wrong?

[Frank Gruber]

You are running 2 x 300GB 10K disks as RAID1. The perfomance you see is what you can expect from this config within ESXI 4.1.

[Michael A. McKenney]

ESXi performance on RAID 1 is just booting it. Once loaded on the server, it does not really access the drives. It is like saying RAID 1 for Windows 2008 is slow. Once 2008 is loaded it is writing small logs to the RAID array. 50-70 MB/s won't slow it down.

[Frank Gruber]

Sorry but I can't follow what you are saying.

From what I have read so far I assumed your datastore is also located at the local RAID1.

If this is the case the figures you stated are what you could expect from that configuration.

Describe exactly from where to where you copied the data and what you measured.

If network was also involved in the copy process watch the CPU utilization of the VM while copying. There is a good chance that you max out one vCPU then.

Cheers.

[Michael A. McKenney]

I only considered VMWare ESXi on a C7000 blade center. I would not do it on 8 cores. The DL380 G7 is too limited for VMWare.

[Frank Gruber]

Runs great on DL380 G7.
IMHO such a general statement isn't valid anyway. From my experience number of CPUs is the bottleneck only in special cases. Most often storage performance gets you into trouble first. I just built a new environment for a customer consisting of three DL380 G7 in a vSphere cluster having 12 cores each which runs great. Here again we reach performance limits of the SAN before we are running out of cores.

Just my 2 cents.

Cheers.

[Michael A. McKenney]

You add the overhead of VMWare on 8 cores. Windows 2003/2008 overehead, multiple paging files degrading disk IOs. I rather run a DL380 D7 as a single file server. You can't put SQL or Exchange in VMWare on that box without really degrading performance. From a hardware standpoint, your disk IOs are probably bottlenecked worse than your CPUs with 24GB of RAM handling multiple OS loads. I talked with VMWare about this issues last year, they said less than 16 cores on a server with 64GB of RAM would not be a good plan for mutliple production servers. Then you are sharing 4 Gbps of NICs. Each of my production servers has 4 Gbps each. I guess your idea of performance vs cost saving on hardware does not meet my expectations for performance. RAID 1 would definitely get bottlenecked with two or more VMWare sessions.

[Erdogan Temur]

Hi branko,

Accelerator is the Ratio? ( Read / Write)
Please attach adu report.

[Frank Gruber]

Sorry but agian I have to disagree completely.

>>You add the overhead of VMWare on 8 cores.

Not true, 12 cores for each node having 36 cores altogether.

>>Windows 2003/2008 overehead, multiple paging files degrading disk IOs.

Sorry but paging files are located on SAN storage.

>>You can't put SQL or Exchange in VMWare on
>>that box without really degrading >>performance.

We run both on them without problem, but sure it depends on the individual customer situation. In the case mentioned we only have 580 users so situation may be different if you have to handle multiple thousand users.

>>From a hardware standpoint, your disk IOs
>>are probably bottlenecked worse than your
>>CPUs with 24GB of RAM handling multiple OS >>loads.

We are ruuning 72 GB on each node having 216 GB alltogehter.

>>I talked with VMWare about this issues
>>last year, they said less than 16 cores on
>>a server with 64GB of RAM would not be a >>good plan for mutliple production servers.

Such a general statemend isn't valid anyway. It depends on the individual stiuation.

>>Then you are sharing 4 Gbps of NICs. Each >>of my production servers has 4 Gbps each.

Wrong. Each node has 8 Gbps of NICs so having 24 Gbps alltogether. Again most of the data pulled via LAN has to be delivered via SAN. Not sure what SAN infrastructure you are using but customer's is not able to saturate 24 Gbps LAN bandwith.

>>I guess your idea of performance vs cost
>>saving on hardware does not meet my
>>expectations for performance.

That may be true for that special customer's situation.

>>RAID 1 would definitely get bottlenecked
>>with two or more VMWare sessions.

Here I have to fully agree with you. But I never said anything different. Here we only have RAID 1 to boot ESXi from. All datastores a located at the FC SAN (8 x 8 Gbit FC + 72 x 300 GB SAS SFF 10K).

Cheers.