CPU Wait Time issue

[Stafford Hyacinth]

I have been through just about all the threads concerning high cpu wait time but can't find anything to explain my issue.

Sar indicates high i/o wait times but queue lenght is always zero. I have an 8420 with 4 hbas connected to an EVA 5000. I am trying to figure out where could this wait time come from. I am running Oracle 9i. All ideas are welcomed...Sar -d attached

[Luk Vandenbussche]

Hi,

You have a quick high IO on you EVA5000 LUNS
Perhaps you can spread the load by changing the secure path paramters.

What application is running on your servers?

[Stafford Hyacinth]

I am running Oracle 9i. Secure path's preferred path/mode is set to No preference. The max I a pushing on the HBAs is 68MB/s so I am trying to figure out where my wait time is.

[Sameer_Nirmal]

Hi,

As you are aware the "wio" is CPU wait state for physical I/Os (block/raw). The most probable causes for high "wio" are non-optimised entities like database layout on the EVA, Oracle configuration , Host/ Storage I/O setup.

EVA is quite sensitive to I/O pumped into it. Oracle which has invariably different types of I/Os , the matter becomes to concern about. It is always good idea to ensure optimised DB layout on EVA to get max throughtput from it. Oracle indexes,data, redolog,archive logs should be on separate Vdisks.

Secondly knowing the capability of EVA , more SCSI I/O could be queued in from host by increasing "scsi_queue_depth" parameter.
Zoning is must. You should ensure to equally distribute the Luns across two HSV controllers .Although EVA tries to load balance the HSVs ( when preferred path = no in CV-EVA), but it always good idea to pre-define the load balancing by setting preferred path ( controller A or B )thus releaving the EVA from such excercise contributing good throughput.
This will give you good control over I/Os being put on controllers. But you should know the I/O patterns well.

As far as the "sar -d" report is concerned , the ave service time looks little high with those I/Os. It is required to take a look at above aspects to conclude the cause of "wio"

[Joseph.KL Teo]

Hi,

For issues on high CPU wait time, the explaination can be quite vague as we are looking more into performance tuning at this point which requires the expertise of a trained personnel in that field.

You would probably have to do lots of trial and error then monitor on the stats over and over again...

For eg, 8420 are Keystone servers which can be hard partition in 4 virtual servers. You can look into your how npar and vpar are configured first.

For SAN, do you have zoning implemented? You may also have to look into the area on your zoning design.

For EVA5000, it uses active-passive controllers by default and you may want to look into how the SecurePath is being configured as SP allows load-balancing as well.

[Stafford Hyacinth]

I am not queueing any I/O. I did have to increase the queue_dept to achieve this.

By not setting a preferred path the EVA alternated the luns on the two controllers. Thanks, I will review for the optimum configuration based on i/o needs.

I am thinking my issue is either on the switches or SAN since I am not queueing i/o on the server side.