RE: High disk I/O.

[Henry Manda]

I'm ruuning COB(close of Business in T24 and jBase) and the CPU and disk IO hits 100%.This makes the system too slow and the COB process takes too long even after modifying fs_async and create_fastlinks kernel parameters.
I'm running on HP-UX 11.31 on rx8640 and EVA4400.Kindly help me to resolve this issue.

[Jineesh]

Hi Henry,

Can you please paste the following command outputs....?

sar -Mu 2 2

Average value in( Ofcourse we should not reply on the average)
sar -d 3 3
top

Are you using asynchronous I/O ...?

[Bart Paulusse]

Hi Henry,

a high cpu load and high disk IO could very well be caused by expensive or ineficient database access, in which case you need to look into your T24 application.

Swapping also causes high disk io. If you run 'top' or 'glance' do you see vhand in the processlist? Your memory is exausted then, probably caused, again, by the T24 application. Use glance to identify the biggest memory consumers.

good luck,

Bart

[Michal Kapalka (mikap)]

hi,

use glance or gpm ( gui ),

the gpm will inform you if some bootleneck happen.

mikap

[Bill Hassell]

> I'm ruuning COB(close of Business in T24 and jBase) and the CPU and disk IO hits 100%.

Unless there is a misconfiguration for these processes, that's what they require. You can't fix a bad application with HP-UX changes.

> This makes the system too slow and the COB process takes too long even after modifying fs_async and create_fastlinks kernel parameters.

fs_async=1 guarantees that you *will* have data loss in case of a powerfail or system crash. You gain a small improvement in speed only when your application opens, writes and closes files very rapidly (many times per second). This is an unuausual situation and as you have see, it made no difference. See the warnings in the man page:

http://www.docs.hp.com/en/B3921-90010/fs_async.5.html

create_fastlinks only applies to HFS filesystems and you may have one filesystem (/stand) that is this type which is virtually unused except during bootup and building a new kernel -- in other words, it has no benefit at all.

You need to contact the vendor(s) of the applications and ask about patches and/or proper configuration. If there is no fix from the vendor, you'll need to add more processors and use much faster disks with many more alternate paths and a bigger cache in the disk array.

[Mel Burslan]

Since you have not mentioned anything about your system properties, I am wondering if you have sufficient memory and your kernel is properly configured to take advantage of all your memory. Your system may be suffering from excessive and unnecessary swap.

If you have glance installed, please run glance and check the top processes and post them here when you get a chance.

As Bill already said, if the application was written poorly by the vendor, there is no magic that you can do on the operating system to make it run faster and with java development workshops popping everywhere in the cheaper workforce employing third world countries, I would not be surprised you have a piece of software developed by someone getting paid $20/day. If the app is poorly coded, even throwing more hardware resources (more memory, more CPUs) to it, will most probably not help, in any significant way.

[ManojK_1]

Hi Henry,

I am also working in an T24 environment but using Oracle 10G as Database.

Can you please give me the Version details of Jbase and T24.Also the Server H/W Details and transcation volume including number of Branches and Accounts.

Thanks and Regards,
Manoj K

[Henry Manda]

sorry, i could reply on time because the COB has finished and its good to have the actual results as below.

sar -Mu 2 2

16:17:03 cpu %usr %sys %wio %idle
16:17:05 0 28 72 0 0
1 31 69 0 0
2 87 13 0 0
3 86 14 0 0
system 58 42 0 0
16:17:07 0 33 67 0 0
1 37 63 0 0
2 86 14 0 0
3 94 6 0 0
system 62 38 0 0

Average 0 31 69 0 0
Average 1 34 66 0 0
Average 2 87 13 0 0
Average 3 90 10 0 0
Average system 60 40 0 0

sar - d 3 3
16:22:23 device %busy avque r+w/s blks/s avwait avserv
16:22:26 disk6 0.33 0.50 1 4 0.00 0.19
16:22:29
16:22:32 c7t0d2 47.00 10.26 958 7660 5.23 3.65
c7t0d3 47.33 10.36 966 7729 5.80 3.63
c7t0d4 47.00 9.97 951 7609 5.12 3.64
c7t0d5 47.00 10.23 961 7688 5.56 3.64
c12t0d2 47.67 10.47 958 7662 5.45 3.69
c12t0d3 48.00 10.43 966 7727 5.93 3.68
c12t0d4 47.33 10.08 951 7611 5.24 3.67
c12t0d5 47.33 10.32 961 7688 5.69 3.67
disk4 0.33 0.50 2 23 0.00 2.11
disk6 94.67 10.36 1916 15322 5.34 3.67
disk8 95.00 10.40 1932 15456 5.87 3.66
disk10 94.00 10.02 1903 15219 5.18 3.65
disk12 94.67 10.28 1922 15376 5.62 3.65
Average disk6 31.63 10.36 638 5103 5.34 3.67
Average c7t0d2 15.65 10.26 319 2550 5.23 3.65
Average c7t0d3 15.76 10.36 322 2573 5.80 3.63
Average c7t0d4 15.65 9.97 317 2533 5.12 3.64
Average c7t0d5 15.65 10.23 320 2560 5.56 3.64
Average c12t0d2 15.87 10.47 319 2551 5.45 3.69
Average c12t0d3 15.98 10.43 322 2573 5.93 3.68
Average c12t0d4 15.76 10.08 317 2534 5.24 3.67
Average c12t0d5 15.76 10.32 320 2560 5.69 3.67
Average disk4 0.11 0.50 1 8 0.00 2.11
Average disk8 31.63 10.40 643 5146 5.87 3.66
Average disk10 31.30 10.02 634 5067 5.18 3.65
Average disk12 31.52 10.28 640 5120 5.62 3.65

top

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
2 ? 25893 live 152 20 87532K 7424K run 1727:32 99.89 99.71 jpqn
0 pts/tp 14245 live 152 20 99088K 8552K run 4:36 69.11 68.99 jpqn
1 pts/td 17099 jbase 152 20 180M 133M run 2748:56 62.46 62.36 jpqn
1 pts/tB 16005 jbase 152 20 133M 56420K run 0:27 58.55 53.49 jpqn
3 pts/tC 15265 live 152 20 87864K 7496K run 0:45 40.55 40.48 jpqn
0 pts/tz 15974 live 152 20 86644K 7404K run 0:06 11.62 10.88 jpqn
2 pts/tc 15740 kibet 152 20 54032K 5332K run 0:01 3.58 3.57 jpqn
0 tty0p0 14653 root 162 10 40348K 4372K run 0:29 3.23 3.23 glance
1 ? 8 root 152 20 1008K 896K run 71:04 2.96 2.96 ksyncer_
3 pts/tj 14686 live 152 20 87688K 7508K run 0:37 2.72 2.72 jpqn
2 ? 137 root 152 20 4608K 4096K run 0:15 1.25 1.25 pagezero
3 pts/tl 15768 live 152 20 86644K 7428K run 0:06 1.05 1.05 jpqn
2 pts/tn 16048 live 152 20 76036K 6612K run 0:00 1.56 0.86 jpqn