HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

ProLiant DL380 G3 I/O Bottleneck

 
Andrew_235
Occasional Advisor

ProLiant DL380 G3 I/O Bottleneck

Hi,

Hoping someone can help. I have a ProLiant DL380 G3 with 1GB RAM and 3 x 18.2GB SCSI Ultra320 15k drives in a RAID 1 configuration.

We are experiencing massive i/o bottleneck problems. Using performance monitor our average % Disk Time is 80+ percent (over a 12-24 hour period).

The system is running SQL server 2000. It is also set as a Domain Controller, however the domain is very small with only a handful of users and a max of 15 connections at any one time.

Any ideas/suggestions?

Andrew
23 REPLIES
Prashant (I am Back)
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi,

Is the read operation is more or write operation on HDD. We can set these paramtre under ACU.

Regards,
Nothing is impossible
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi,

It appears as if we are doing more writing than reading.
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

as prashant indicates if you believe there is more write operations than read on the server, you can set a higher write cache ratio than read on the array accelerator. So here is how you do that:
from windows, fire up ACU (generally found under HP system tools -> Array Configuration Utility. Select the SA5i controller in embedded slot, then on the right hand side click on the "Controller Settings" and under "Cache ratio" specify appropriate read/write cache ratio.

If you do not have ACU installed on the server, download it from the link below:
http://h18023.www1.hp.com/support/files/server/us/download/14185.html

hope this helps and don't forget to assign points:)

regards,
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

how do you monitor the disk time counter, are you just using perfmon on windows or a system management software?

You could also run HP PAT (performance assessment utility) and test the disk read/write/sequential read/etc on the various drives/volumes on the system.

do you have the latest system BIOS and the latest firmware on the drives?

Have you enabled disk caching from within Windows on OS level disks that Windows sees?

hope it helps and don't forget to assign points:)

regards,
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Saket,

When running the ACU, i get no other option than 100% read, 0% write. Is this normal? How do i change this?

I have the latest BIOS and firmware updates.

Will check on the OS side of caching and let you know.

Regards,
Andrew
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

that would be the case if your SA5i did not come with BBWC (battery backed write cache).

Note, without BBWC, write caching is too dangerous and hence you could only set 100% read. Are you sure you are not using Proliant DL380 G2?

P.S.Could you please assign points to the replies.

Thanks,
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

No doubt, i have a DL380 G3.

What other options do I have? Is there anything else I can do to improve our i/o performance?

Andrew
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi again,

Would I be better off to use RAID 1+0? Can this be done 'on the fly' while the system is online?

What performance will I see over RAID 1 using this method?

How do I configure RAID 1+0 under ACU?

Thanks for your help,
Andrew
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

So, why do you have three disks in a RAID 1 set? Is this your system partition and data both?

I very much doubt that you will gain much by going from RAID 1 to RAID 1+0. I am also not sure if this RAID migration is supported on SA5i.I will check it for ya.

hope it helps and dont forget to assign points:)

regards,
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

Do you know if your disks are fragmented? you could run a fragmentation anaylsis from within Windows and let us know what report you get.

Do you have a large number of small files on the NTFS volume? What about CPU and memory utilisation on the box? I know SQL can chew away massive amounts of memory, have you actually gone through SQL tweaking?

please reply to the above and see how we go...

hope it helps and dont forget to assign points:)

regards,
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Saket,

I have some file fragmentation but nothing too serious.

I have been through the tweaking of SQL server also.

We are not experiencing any paging so I have assumed memory is not an issue.

Would setup of cabling etc be an issue for through put? Is there a way of determining i/o transfer rates?

Andrew
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Another person has suggested upgrading from the onboard 5i controller to a 6402 controller. It appears that the 5i does not run at Ultra320 speeds which is what our SCSI drives are capable of.

The 6402 also has 128MB read/write cache.

Is anyone able/willing to try and determine a performance increase from the 5i to a 6402 controller? :-)


Andrew
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

you could use the good old IOmeter tool to isolate I/O performance issues.

Below is the link for it:
http://sourceforge.net/projects/iometer/

hope it helps and dont forget to assign points:)

regards,
Rob Leadbeater
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

The BBWC is an installable option on the DL380 G3, so if you don't get the option to turn on the relevant cacheing it more than likely isn't installed.

You can find out for certain by taking the lid off, and looking for the battery holder which slots in to the front right hand side of the server.

The picture here:
http://h18004.www1.hp.com/products/servers/proliantstorage/arraycontrollers/smartarray5iplus/index.html

shows the battery module.

Regards,
Rob
kcpant
Trusted Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

hi andrew,

there will be sure performance increase if you migrate to SA 6402, bacause, not only you will have optimum speed handelling ( UW320) for youe HDDs, but you will be able to set read /write chaching as desired.one thing I want to know is , what is the array configuration in your system? ( I'm sorry , you have written RAID 1, but raid 1 can only have 2 disks, so, how you are using the third one?).

regards,
PreSales Specialist
SAKET_5
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

I would recommend you take a look at the following link:
http://h18000.www1.hp.com/products/quickspecs/11063_na/11063_na.HTML

regards,

Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi All,


Saket - Have just downloaded IOmeter, will give it a shot and get back to you all with the results.

Rob - Thanks for that info, now deciding if we are better off just upgrading to the 6402 controller instead of getting a BBWC enabler.

kcpant - It would be nice to know what kind of performance increase we are likely to see, has anyone upgraded from a 5i to a 6402? If so, what was the difference?

Our RAID is a RAID 1 with an online spare. A more expensive option, however we run a critical application and redundancy is a big priority.

I may make a post in another forum to see if anyone has upgraded from a 5i to a 6402 controller to try and measure the improvement. Please continue this thread however.


Andrew
Gus Kwong
Respected Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew,

DL380g3 uses Simplex setup as standard --- Just want to check whether you have already set it to Duplex configuration for you drive cage?

SA6402/4 is such high-end controller it will certainly give you performance boost... however, people will use that mostly to connect to things like StorageArrays30DB... But I don't think it is good value for money, as Duplexing the 5i should already give you max 320MB/s (160MB/s per channel).

However, if you can easily afford such controller (hey, SA641 could be ideal too), I would also recommend you to buy another hard drive (to make it up to 2 sets of RAID1). Then, set the DL380g3 to PCI Duplex so that you'd get 480MB/s max throughput (160MB/s in slot 0,1 controlled by the 5i for OS usage, and 320MB/s shared on slot 2,3,4,5 controlled by SA6402/SA641 for MSSQL usage)... Of course, you can use PCI Simplex and let the SA6402 to handle all 6 slots at shared max 320MB/s --- that will depend on your environment.

Hope this helps.

Cheers,

Gus
Learning never ends
Doug de Werd
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

just to throw in a couple more things...


All of the suggestions about upgrading controllers and RAID sets is good, but I am convinced that the single most important thing you could do is to add enough drives to physically separate your OS, data, and transaction logs.

Ideally you would have 3 arrays, each with their own single disk volume, and each of them set up as a 2 drive RAID 1. Another option wuold be to combine the OS and data onto a single RAID 1 array (2 drives) with 2 volumes.

However, the MOST important thing for database performance - especially with heavy writes - is to dedicate a physical array (2 drives in RAID 1) with a single volume.

Think about it this way... the ACU allows you to set up multiple volumes within an array of physical drives. This is nice for convenience of assigning LUNs, but it can be a performance killer for databases.

OS and data volumes are characterized by random read/write I/O. So combining them (with 2 volumes) on a single physical 2 drive array is no big deal - it's going to do random I/O anyways.

But the transaction logs are characterized by sequential writes - that's it. By combining the volume containing the logs on the same physical array as the random I/O of the OS and data, you really reduce the write performance of the transaction log (which is already very busy because you are in a write intensive environment). It all comes down to head movement in the physical drives themselves. With random I/O the heads are moving all over the place, back and forth. With sequential writes, the heads IDEALLY only move in teeny little increments to do the next sequential write. By combining them, you have 2 opposite performance requirements - the large amount of head movement back and forth across the disk platters is in direct competition with the teeny little movements of the sequential writes.

As a side note, having a Smart Controller with a relatively large BBW Cache will also do wonders for performance, but this will have less of an effect than separating the log files (you should really do both!)

Check out these white papers for more info:

http://h71019.www7.hp.com/ActiveAnswers/Render/1,1027,5108-6-100-225-1,00.htm

http://h71019.www7.hp.com/ActiveAnswers/Render/1,1027,5264-6-100-225-1,00.htm

http://h71019.www7.hp.com/ActiveAnswers/Render/1,1027,5265-6-100-225-1,00.htm


Thanks,
Doug
Expert in ProLiant Clusters
Andrew_235
Occasional Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi,

Guy - To be honest, I am not sure. I believe the drives may be in simplex mode. What is the easiest (and least painful) way to check? If it is not, how do I go about setting it up? The documentation is very thin. With Ultra320 drives, running at 320Mb/s, will duplexing allow through put of 320Mb/s (2 x 160MB/s)? How does this work with multiple drives on the channel?

Doug - Will have a think about adding some more drives and sepearting things over 3 x RAID 1 sets.

Can someone explain the simplex/duplex method and how it will alter our performance?


Andrew
Prashant (I am Back)
Honored Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi,

See the snap shot for conenction. You are using two cotnrller to incrase the performace.

PCI duplex - All internal hard drives are split between two SCSI controllers. The embedded Smart Array 5i Plus Controller is connected up to two hard drives on one SCSI bus and the optional PCI array controller installed into an available PCI slot controls up to four hard drives on the other SCSI bus.

Regards,
Prashant S
Nothing is impossible
Gus Kwong
Respected Contributor

Re: ProLiant DL380 G3 I/O Bottleneck

Hi Andrew

>What is the easiest (and least painful) way to check?

The way I'd check is to see if the SCSI terminator is in place at the back-plane --- if not, then it is simplex. If you bought the server new, then it should come with a terminator and a spare SCSI cable.


>how do I go about setting it up? The documentation is very thin.

Unfortuantely, I don't know if there is an easy way to migrate the RAID set from simplex to duplex... not unless you Ghost the server up-the-way, recreate the RAID set in duplex, and Ghost it down again. This document should be useful (do a find on Simplex and duplex) http://h20000.www2.hp.com/bizsupport/CoreRedirect.jsp?targetPage=http%3A%2F%2Fh200007.www2.hp.com%2Fbc%2Fdocs%2Fsupport%2FUCR%2FSupportManual%2FTPM_303130-003%2FTPM_303130-003.pdf


>With Ultra320 drives, running at 320Mb/s, will duplexing allow through put of 320Mb/s (2 x 160MB/s)?

Since hard drives cannot really reach 160MB/s, a dedicated SCSI channel (at 160MB/s) will be more than adequate for just one drive. So if you mirror 2 disks on 2 separate SCSI channels, then you don't have to worry about hard drives fighting for the SCSI channel (not until you have more than 2 drives on one channel).


>How does this work with multiple drives on the channel?

Quite simply, the Drive Cage uses the shared channel for all disks --- so in Ultra3 mode, if all 6 disks are writing at the same time then each disk could be hit as little as 26MB/s.

In duplex mode, the drive cage is physically separated so that slot 0,1 is in one isolated area, and slot 2,3,4,5 in another area. They will simply be assigned with different SCSI ID as well as the controller ID. But the DL380g3 doesn't care, and you can simply select appropriate disks when you create your RAID sets.

Gus
Learning never ends
Martin Forster_1
Frequent Advisor

Re: ProLiant DL380 G3 I/O Bottleneck

you do not need to joke arround with the bus performance to disk. event fastest Harddisk i know does not need that performance. Its more for 14 Drive boxes or so.
You will see the most performance gain from the BBWC.
Next Step is to separate the transaction logs. 1st logicall 2nd on a own raid.

And make sure that memory is not your bottleeck, because when you limit the amount of memory for the SQL Server. Paging lost its indication for a memorybottleneck.

Regards.