sar vs perfview

[Asad Malik]

Hi
sar -d displays a column %wio. which metric in glance and specially in perfview shows this information.
%wio shows values greater than 60%. I want to investigate further disk bottleneck.

Any suggestions?

Thanks

[Jim Welch]

excerpt from the old "Glance vs SAR" whitepaper:

some users have also expressed interest in sar -u's
"wio" metric, and the fact that this metric is not in glance or
measureware. Sar's wio metric is an estimate of cpu time "idle, but
waiting for I/O to complete". The instrumentation that glance uses (the
KI/MI) provides more accurate cpu statistics in general because they are
trace-based not sample-based, but idle cpu time is not broken down.
Glance's idle cpu time will be roughly equal to sar's idle plus wio.

[CHRIS_ANORUO]

Use "sar -ud" report activity for each disk and "sar -b" for buffer activity informations.
Also check out these documents: S3100002312A,B and C (Sys Adm: determining the cause of system performance problems).

[Stefan Farrelly · Accepted solution]

The %wio from sar is a very useful global performance metric. I wish PerfView/glance had it also.

Its very valuable in determining your systems overall disk i/o performance. I can give you a guide, anything over 20 and your system is completely i/o bound, <10 is ok, < 5 is good. At HP we ensure all our important servers run a %wio <5.

If yours is indeed 60 I would begin a plan to add faster disks/stripe everything etc. once youve used sar or Perfview to determine which disks/lvol is causing the large bottleneck.

[Dave Wherry]

I can not tell you specifically how yet, but, high %wio also relates to buffer cache. It goes deeper than just slow disks or i/o bottleneck.
I had a K570 running with 2 FC30 arrays. %wio was consistently in the 50% range during peak hours. We then moved to a V2500 and XP256. %wio is now in the 30% range. Better, but, still bad. However, i/o performance is good. If we check the access times from our applications they are sub-second.
When the system was installed the kernel parameter dbc_max_pct was left at the default of 50. This means that the system can use up to 50% of RAM for buffer cache. This is typically too high. I decreased dbc_mac_pct to 30 and %wio shot up to 70%. I needed that buffer cache.
A second part of this is that our system is an Oracle database server. There is a performance hit called double buffering. Oracle caches writes in the SGA. Then depending on how the file systems are mounted, it can also write to your buffer cache. Two i/o's. If you mount the file systems with I believe it is the "mincache=direct" option it will not use buffer cache. Your database writes to the SGA which then writes directly to the disk, by-passing the system buffer cache. this is much faster.
When I tried this %wio again jumped up to 50%. Obviously I did something wrong and I have not gone back to work with it again. What it does show is that buffer cache usage plays a large role in the %wio metric. And, while ours is still high the system performs well. While slow disks will create an i/o bottleneck there is often times more than one problem.