RE: Disk IO shooting to 100%

[Henry Manda]

Every time I run COB in T24, the disk IO shoots to 100%.I created a new file system with PVGs to enable load balancing, disk stripping is also enabled, but still Im experiencing the same problem.COB takes 12 hrs.Kindly give me an idea about creating a memfs and use it with T24 application.or is there anything else i can do to reduce the IO??
Im running HP-UX 11i V3, T24R06.005, jbase 4.1,15branches, 2000 trxn per branch per day.

[Sunny123_1]

Hi

Refer following link

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01917506/c01917506.pdf (BSC link updated by admin)



Regards
Sunny

[ManojK_1]

Hi,

Please provide the following details.

1) server Model
2) vgdisplay -v
3) Disks are local disk or disk presented from
storage?
4) Whether it is an application Server or DB
Server?
5) out put of the command "sar -d 2 20"
6) out put of the command "swapinfo -tam"

Manoj

[Viktor Balogh]

FYI: T24 is a special banking application from company Temenos.

http://www.temenos.com/Software/Core-Banking-Solution-System-T24/

and COB must be referring to "Close of Business", a routine at the end of the day.

Could you confirm it, Henry?

[Henry Manda]

Server Model: rx8640
vgdisplay output:

--- Logical volumes ---
LV Name /dev/vg00/lvol1
LV Status available/syncd
LV Size (Mbytes) 1792
Current LE 28
Allocated PE 28
Used PV 1

LV Name /dev/vg00/lvol2
LV Status available/syncd
LV Size (Mbytes) 32768
Current LE 512
Allocated PE 512
Used PV 1

LV Name /dev/vg00/lvol3
LV Status available/syncd
LV Size (Mbytes) 10240
Current LE 160
Allocated PE 160
Used PV 1

LV Name /dev/vg00/lvol4
LV Status available/syncd
LV Size (Mbytes) 5120
Current LE 80
Allocated PE 80
Used PV 1

LV Name /dev/vg00/lvol5
LV Status available/syncd
LV Size (Mbytes) 30720
Current LE 480
Allocated PE 480
Used PV 1

LV Name /dev/vg00/lvol6
LV Status available/syncd
LV Size (Mbytes) 10240
Current LE 160
Allocated PE 160
Used PV 1

LV Name /dev/vg00/lvol7
LV Status available/syncd
LV Size (Mbytes) 20480
Current LE 320
Allocated PE 320
Used PV 1

LV Name /dev/vg00/lvol8
LV Status available/syncd
LV Size (Mbytes) 33280
Current LE 520
Allocated PE 520
Used PV 1


--- Physical volumes ---
PV Name /dev/disk/disk4_p2
PV Status available
Total PE 4785
Free PE 2525
Autoswitch On
Proactive Polling On



VG Name /dev/vgt24
VG Write Access read/write
VG Status available, exclusive
Max LV 16
Cur LV 1
Open LV 1
Max PV 8
Cur PV 4
Act PV 4
Max PE per PV 32000
VGDA 8
PE Size (Mbytes) 16
Total PE 51196
Alloc PE 51136
Free PE 60
Total PVG 1
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0
VG Max Size 4000g
VG Max Extents 256000

--- Logical volumes ---
LV Name /dev/vgt24/lvt24
LV Status available/syncd
LV Size (Mbytes) 818176
Current LE 51136
Allocated PE 51136
Used PV 4


--- Physical volumes ---
PV Name /dev/dsk/c14t0d2
PV Name /dev/dsk/c7t0d2 Alternate Link
PV Name /dev/dsk/c9t0d2 Alternate Link
PV Name /dev/dsk/c12t0d2 Alternate Link
PV Status available
Total PE 12799
Free PE 15
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c7t0d3
PV Name /dev/dsk/c14t0d3 Alternate Link
PV Name /dev/dsk/c9t0d3 Alternate Link
PV Name /dev/dsk/c12t0d3 Alternate Link
PV Status available
Total PE 12799
Free PE 15
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c9t0d4
PV Name /dev/dsk/c7t0d4 Alternate Link
PV Name /dev/dsk/c14t0d4 Alternate Link
PV Name /dev/dsk/c12t0d4 Alternate Link
PV Status available
Total PE 12799
Free PE 15
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c12t0d5
PV Name /dev/dsk/c7t0d5 Alternate Link
PV Name /dev/dsk/c9t0d5 Alternate Link
PV Name /dev/dsk/c14t0d5 Alternate Link
PV Status available
Total PE 12799
Free PE 15
Autoswitch On
Proactive Polling On


--- Physical volume groups ---
PVG Name pvg01
PV Name /dev/dsk/c14t0d2
PV Name /dev/dsk/c7t0d3
PV Name /dev/dsk/c9t0d4
PV Name /dev/dsk/c12t0d5
PV Name /dev/dsk/c7t0d2
PV Name /dev/dsk/c9t0d2
PV Name /dev/dsk/c12t0d2
PV Name /dev/dsk/c14t0d3
PV Name /dev/dsk/c9t0d3
PV Name /dev/dsk/c12t0d3
PV Name /dev/dsk/c7t0d4
PV Name /dev/dsk/c14t0d4
PV Name /dev/dsk/c12t0d4
PV Name /dev/dsk/c7t0d5
PV Name /dev/dsk/c9t0d5
PV Name /dev/dsk/c14t0d5


--- Volume groups ---
VG Name /dev/vgt24b
VG Write Access read/write
VG Status available
Max LV 2047
Cur LV 1
Open LV 1
Max PV 2048
Cur PV 3
Act PV 3
Max PE per PV 230391
VGDA 6
PE Size (Mbytes) 4
Total PE 230391
Alloc PE 230391
Free PE 0
Total PVG 1
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 2.1
VG Max Size 921564m
VG Max Extents 230391

--- Logical volumes ---
LV Name /dev/vgt24b/lvt24b
LV Status available/syncd
LV Size (Mbytes) 921564
Current LE 230391
Allocated PE 230391
Used PV 3


--- Physical volumes ---
PV Name /dev/disk/disk43
PV Status available
Total PE 76797
Free PE 0
Autoswitch On
Proactive Polling On

PV Name /dev/disk/disk44
PV Status available
Total PE 76797
Free PE 0
Autoswitch On
Proactive Polling On

PV Name /dev/disk/disk45
PV Status available
Total PE 76797
Free PE 0
Autoswitch On
Proactive Polling On


--- Physical volume groups ---
PVG Name pvg1
PV Name /dev/disk/disk43
PV Name /dev/disk/disk44
PV Name /dev/disk/disk45

Disk is presented from EVA4400 storage
Its a database server.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 32768 0 32768 0% 0 - 1 /dev/vg00/lvol2
reserve - 893 -893
memory 15520 5859 9661 38%
total 48288 6752 41536 14% - 0 -
#
YES, ITS FROM TEMENOS AND YOU ARE 100%CORRECT,

[Pete Randall]

100% of what? Where are you seeing this output? And what does it really mean? Is it really a problem or does the high number (that we have no frame of reference for) concern you?


Pete

[Tim Nelson]

A few thoughts:

Just because a device is at 100% utility does not neccesarily mean there is a performance issue. If a disk is being accessed then what more is expected ?

a RAM disk is certainly ok, but look at a couple things first.

Is there really an issue ?

Keep in mind how the performance gathering works, take a sample wait, take another sample and average. If every time a sample is taken the device is busy then 100% is the number. ( this is usually the case with single devices leading to a SAN).
When devices are busy you are more interested in the response time of the requests ( e.g. disk service time from either sar or glance ).
IF these are poor then sure, start trying to make the device faster by any number of methods, faster HW, striping, data layout, contention reduction etc..

if responses are around 3-5ms on SAN storage then you are not going to get much faster except moving to solid state disk or your RAM disk solution.

Keep in mind there are trade offs with everything. A RAM disk has a very high risk of data loss in the event of a system crash. And, a is a RAM disk that much different from the fs buffer cache mechanism.

Maybe some tuning in the right places could be a better solution ? Maybe moving the data around to reduce contention ? Maybe splitting up of processes ? Maybe ...

Just some things to think about.

[Henry Manda]

To me there is a problem because the COB process takes longer than usual.
with 15 branches and around 2000 trxns per day and a database size of 200GB, COB should take at most 4 hours, instead its taking 13hrs, and infact Temenos consultants have done all that pertains to performance tuning on the database.
Please Assist me.

[Shibin_2]

Can we think something out with respect to the OS and application?

T24 released long ago, before HP-UX 11.31 comes out. Even, I worked on it.

Could this be an issue with the database version of jBASE that you are running?

[Torsten.]

You should always use the new device files (e.g. /dev/disk/diskx) instead of legacy.
Consider to convert!