getting very high write complete waits and i/o wait times

[John Goetz]

Running Oracle applications and getting very high write complete waits and i/o wait times.
Please advise if any kernel parameters setting may help this type of issue.

[Jose Mosquera]

Hi,

In the past I've posted a similar case and solution was set dbc_max_pct kernel parameter. So pls check that issue solution link:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=32385

I hope that be useful for you.

Rgds.

[Steven E. Protter]

My max_dbc_pct is 7 or 8 on most systems
my min_dbc_pct is 5 on all managed systems.

Other things to check:

vxinodes, you may want to set that manually instead of letting the system decide. It decides too much and impedes performance.

Also, see this doc.

http://www1.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&search.y=8&search.x=28&mode=id&admit=-682735245+1086795823425+28353475&searchCrit=allwords

You might actually have an issue with disk performance. It its local, probably not. If its SAN, san configuration can have a huge impace.

We have three Unix systems and several major windows systems striped over 30 disks raid 5. The various systems have an inordinate effect on one and other when activity gets high on our largest oracle data filesystem.

We're still trying to figure out why.

SEP

[Navin Bhat_2]

Before making any kernel changes I would advise you get top the root cause of the problem. Are you seeing this wait times as part of Oracle's stats? or from sar -d output?

Do you have a disk array in SAN config? Has the SAN/array been checked out for h/w errors/warnings?

Get to the root cause before you make changes to the kernel.

[Sridhar Bhaskarla]

Hi,

Check if the system stats are supporting the issue. Run "sar -d 2 20" and observe the 'avwait' and 'avserv'. If they are high numbers with 'avwait' more than 'avserv', then there is a good chance that the bottleneck lies on either storage (mostly) or the hardware components in between.

You should also see some devices that are 100% '%busy' in sar output with 'avqueue' with somenumber other than .5.

If you have glance, it will also show better numbers of queue built on the corresponding disks.

There may be a good chance that one/few of the disks (LUNs) is heavily accessed than the ohter in which case you may have to spread the load.

-Sri

[Fred Ruffet · Accepted solution]

What Oracle recommends in such cases is to bypass the OS IO cache for FS dedicated to Oracle's DB files. It can be done by mounting with options like this :

nodatainlog,largefiles,mincache=direct,convosync=direct

regards,

Fred