topic Re: improving i/o performance and max depth queue diskes with slow users in Operating System - HP-UX

improving i/o performance and max depth queue diskes with slow users

Max5 — Fri, 22 Dec 2017 15:30:25 GMT

HI All

for improving i/o disk performance modifing max depth queue , beeing 8 at the moment as the default !,

i would like to suggest to tune it at 32

there are 8 physical diskes for application data , every vg one disk , 136 GB eachone

old os hp-ux more or less 15 years a go

vendor Compaq , product SAS BF14684970

problem is very slow sas performance for the users

would be very good to put 32 for the maximum queue depth for improving the performance ?

which is your opinion ?

thanx in advance, best regards Max

Re: improving i/o performance and max depth queue diskes with slow users

Bill Hassell — Wed, 27 Dec 2017 03:00:51 GMT

I don't think you'll see anything better with a deeper queue depth. The queue depth allows more requests to pile up but there is no performance change since HP-UX is running as fast as it can. Your layout (8 disks, one disk per VG) has several performance issues. The most important is that it sounds as if the disks are not mirrored. That means complete loss of the data on the disk(s) that fail, production downtime while repair is made and data is restored from backups.

There is very little that can be done to improve a single disk VG performance. The recommendation is to replace all of the disks with a modern disk array where several disks are used in striped mode to reduce overalll access time and improve the data transfer speed.

Note: The system hardware and version of HP-UX must be considered when looking at adding modern disk arrays. Older PARISC systems running HP-UX 11.11 or earlier cannot connect to more recent arrays such as the MSA2040. An alternative would be to replace the entire system with an rx2800 where all your storage will now be internal with an array controller maximizing performance. This can be done with HP-UX Containers.

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Wed, 27 Dec 2017 11:22:20 GMT

Hi Bill

thanx for your feedback.

even if is useful what you write, i think instead that is a good test to extend from 8 to 16 the queue depth

8 is very low default

the only mode for trying , it is to organize the change and test the performance

thanx for your infoes

best regards Max

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Wed, 27 Dec 2017 13:06:07 GMT

can i ask you, with kmtune -u -s scsi_max_qdepth=16 , i can get new value on all the disk devices in dynamic way , can i ?

only this command, and no other, from what i know

no reboot is necessary

thanx in advance

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Wed, 27 Dec 2017 17:02:56 GMT

     Or.. in more i could look at these values , for the fs tuning :  maybe would you know suggest some values ?  now is default

      default_indir_size
         

      discovered_direct_iosz
         

      hsm_write_prealloc

 thanx, best regards !

Re: improving i/o performance and max depth queue diskes with slow users

Bill Hassell — Wed, 27 Dec 2017 18:03:18 GMT

I would not change the VxFS parameters without repeatable work loads and careful analysis.
I think you'll fid that the default settings are the best for single disks.
If you have Glance installed, use it to provide details on the system's current workload.
You need to determine if the system is actually disk-bound or has a large CPU load.
Without Glance, you have to use sar to monitor current OS status.

For CPU load, use top.
For disks, use sar -d 2 20

There is no point in changing disk or filesystem settings if the CPU is 100% loaded. For any meaningful recommendations, you need to provide:

System model, HP-UX version, RAM, processor count

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Thu, 28 Dec 2017 12:02:19 GMT

Hi ,

cpu is not 100% or saturaton , but i/o disk has some strong peaks

thanx inadvance

# sar -d 5 5

HP-UX B.11.31 U ia64

11:36:03 device %busy avque r+w/s blks/s avwait avserv
11:36:08 c1t3d0 0.20 0.50 0 0 0.00 10.55
c3t2d0 100.00 54.92 918 109799 60.71 8.70
c3t4d0 0.20 0.50 0 0 0.00 7.75
disk1 0.60 0.50 1 10 0.00 5.08
disk19 0.20 0.50 0 0 0.00 10.55
disk24 100.00 54.92 918 109799 60.71 8.70

Re: improving i/o performance and max depth queue diskes with slow users

Bill Hassell — Thu, 28 Dec 2017 20:43:01 GMT

Is there a reason that you did not paste all of the lines from sar?
This is just a single picture of a few of your disks.
There should have been 5 groups of 5 second snapshots plus a summary.
What little it shows seems to point out something unusual for your disk assignments:
c3t2d0 and disk24 appear to be the same physical disk.
It is also the only disk that is busy.
Based on this sparse information, your system is not busy at all.
Changing the queue depth will likely have no measureable effect.

How about the requested information about your server?
And program or database is running slow on this server?

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Fri, 29 Dec 2017 07:44:16 GMT

Hi Bill

ye, i have put only one part of the output,

ye at the moment nog big load and no big work on the machine is running

but when works immediately there are big peaks in i/oand ram usage ...

you have seen sometime 100% disk usage

i'll post more shortly

thanx

Best Max

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Fri, 29 Dec 2017 16:58:43 GMT

# swapinfo -m
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 0 8192 0% 0 - 1 /dev/vg00/lvol4
reserve - 4226 -4226
memory 71887 12851 59048 18%

Re: improving i/o performance and max depth queue diskes with slow users

Max5 — Fri, 29 Dec 2017 17:14:06 GMT

this is the bottleneck, for exemple , just 2 rows is visible .. some 100 % cpu's (not all)

and avwait values 59 or 54 > then avserv values 8 or 7 ...

disk07 99.80 55.19 936 111218 59.88 8.52

c3t2d0 100.00 53.28 1016 121446 53.22 7.85

sorry for the few exemples.

Re: improving i/o performance and max depth queue diskes with slow users

Bill Hassell — Fri, 29 Dec 2017 20:27:37 GMT

...sorry for the few exemples....

Analyzing and resolving performance concerns requires a lot of measuremnts and information about what is using the system resources. There is no magic switch or parameter to make your system run faster. You can make any disk 100% busy with the dd command, or make all the CPUs busy with a simple script. The application (database perhaps) is causing the load. Stop the application and performance will be excellent.

Is the application something that can be changed, perhaps rewritten to be more efficient?

The latest measurement shows two different disks that are very busy. The queue depth for both disks is over 50, so changing the SCSI driver depth to 32 will produce no change for that specific moment in time.

Changes for various parameters are meaningless if you do not have a repeatable load. For this simple example, you would have to change all your external disks for solid-state disks. In that case, you would see immediate performance gains.