topic Wait Reason: SYSTM in Operating System - HP-UX

Wait Reason: SYSTM

Gordon Fong — Wed, 21 Aug 2002 09:31:25 GMT

Hi all,

Right now doing performance tuning, is it normal / what cause this behaviour, that a process spend > 70% of time waiting SYSTM?

I sure CPU & Memory has enough resource ( > 3G memory free ) , for disks, sometimes it get to 100 spikes ( not last long, maybe 2 sec ), not obvious disks queue observer.

Bgds,
Gordon

PC : 0.0 Cache : 0.0 Wait Reason: SYSTM
Job Control: 0.0 CDROM IO : 0.0
Message : 0.0 Disk IO : 0.0
Pipe : 0.0 Graphics : 0.0
RPC : 0.0 Inode : 0.0
Semaphore : 0.0 IO : 0.0
Sleep : 0.0 LAN : 0.0
Socket : 0.0 NFS : 0.0
Stream : 0.0 Priority : 2.6
Terminal : 0.0 System : 75.9
Other : 0.0 Virtual Mem: 0.0

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Wed, 21 Aug 2002 11:12:08 GMT

Gordon

When this is at 70%+ what is the server doing - how busy?

Paula

Re: Wait Reason: SYSTM

Gordon Fong — Thu, 22 Aug 2002 00:02:38 GMT

Hi Paula,

Since it's Sybase replication server process, I have no idea on what that time this rep. server doing...

Bgds,
Gordon

Re: Wait Reason: SYSTM

harry d brown jr — Thu, 22 Aug 2002 00:30:25 GMT

(1) What OS revision?

(2) What kind of hardware?

(3) Do you have glance installed? if not, purchase it and install it.

(4) is ems running?

(5) what are the top processes running?

(6) what does your swapinfo look like?

(7) what are your kernel parameters?

(8) What kind of disk sub-system?

live free or die
harry

Re: Wait Reason: SYSTM

Gordon Fong — Thu, 22 Aug 2002 00:40:11 GMT

Hi Harry,

Thx for yr address of Q.

1. It's running 11.00
2. N-class 8 CPU and 9GB memory
3. Yes have glance install, above screen dump for the suspect process is capture from Glance, and showing Wait reason : SYSTM on the sybase replication server process ( Reason I am interest in this process is coz we are experencing serious sybase replication delay )
4. Yes, we have EMS running
5. Top process is the Sybase server
6. sh: swapinof: not found.
:/#swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol2
dev 3072 0 3072 0% 0 - 1 /dev/vg00/swap2
dev 4312 0 4312 0% 0 - 1 /dev/vg01/swap3
dev 4312 0 4312 0% 0 - 1 /dev/vg01/swap4
reserve - 4122 -4122
memory 7133 4498 2635 63%
total 19853 8620 11233 43% - 0 -
7. Which kernel parms? There are so many..
8. It's using EMC Symm4 class.

Thx.

Bgds,
Gordon

Re: Wait Reason: SYSTM

Bill Hassell — Thu, 22 Aug 2002 01:11:35 GMT

SYSTM means the program(s) are waiting for system calls. This is completely meaningless however since this is the way the program works and you have little control over that. A system call might be a query for the date or a request to open a file, etc.

Since the Rep-process is taking too long, has it ever run as you expect? Is there anything to compare with? This may be perfectly normal and only a redesign of the database will improve things. Without any 'norm' there's not much you can do to speed up a slow process.

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Thu, 22 Aug 2002 06:10:41 GMT

Hi Gordon

As said without a baseline to go by it is difficult to see if this is a problem.

If the server is very quiet the 70% of not very much is very little, but if busy then that 70% could be 70%b of a lot - therefore a problem.

Do a :-

grep -e KC_PARAM_NAME -e KC_PARAM_STATUS /var/sam/boot.config > /tmp/kern.info

Which will extract the kernel parms and please post the kern.info

Paula

Re: Wait Reason: SYSTM

Gordon Fong — Thu, 22 Aug 2002 09:20:40 GMT

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Thu, 22 Aug 2002 10:21:20 GMT

Gordon

your :-

KC_PARAM_NAME = "dbc_max_pct"
KC_PARAM_STATUS = "20"
KC_PARAM_NAME = "dbc_min_pct"
KC_PARAM_STATUS = "5"

How much physical memory do you have?

A target for dbc_max_pct is circa 400 meg and dbc_min_pct circa 70 meg

use sar ???b

and watch the %rcache column for figures below 100% -

That is:- the system was not 100% successful in finding what it wanted in the cache and so therefore user the slower medium of disk.

If this happen regularly then increase slightly and try again.

Can you post a ps -ef and top output ?

Paula

Re: Wait Reason: SYSTM

Gordon Fong — Thu, 22 Aug 2002 10:29:59 GMT

HI Paula,

Below is the output of Glance instead

Read Cache Hits 11161 100.0 168349 100.0 100.0
Write Cache Hits 260 87.0 5990 79.5
DNLC Hits 14566 90.1 271976 90.6 98.8
DNLC Longs 0 0.0 78 0.0 0.1

Confirm the cache util. is quite good in both read / write, our machine is 9G in memory.

Top output is like this:
Load averages: 0.74, 0.61, 0.58
219 processes: 196 sleeping, 15 running, 8 stopped
Cpu states:
CPU LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
0 0.60 19.8% 0.0% 14.9% 65.3% 0.0% 0.0% 0.0% 0.0%
1 0.71 16.0% 0.0% 14.0% 70.0% 0.0% 0.0% 0.0% 0.0%
2 0.61 29.0% 0.0% 24.0% 47.0% 0.0% 0.0% 0.0% 0.0%
3 1.08 94.0% 0.0% 2.0% 4.0% 0.0% 0.0% 0.0% 0.0%
4 0.68 14.0% 0.0% 13.0% 73.0% 0.0% 0.0% 0.0% 0.0%
5 0.81 99.0% 0.0% 1.0% 0.0% 0.0% 0.0% 0.0% 0.0%
6 0.65 14.0% 0.0% 12.0% 74.0% 0.0% 0.0% 0.0% 0.0%
7 0.80 12.0% 0.0% 9.0% 79.0% 0.0% 0.0% 0.0% 0.0%
--- ---- ----- ----- ----- ----- ----- ----- ----- -----
avg 0.74 36.9% 0.0% 11.7% 51.5% 0.0% 0.0% 0.0% 0.0%

Memory: 458324K (364156K) real, 445652K (366608K) virtual, 3263240K free Page#
1/44

CPU TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
5 ? 8245 sybase 241 20 17472K 4380K run 1350:40 98.91 98.74 dataserv
3 ? 8226 sybase 241 20 17472K 4380K run 1380:32 98.88 98.71 dataserv
0 ? 8225 sybase 168 20 17472K 4380K sleep 758:21 55.43 55.33 dataserv
7 pts/tn 18726 sybase 152 20 48136K 44976K run 4:51 33.01 32.95 repserv

Thx.

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Thu, 22 Aug 2002 10:57:40 GMT

HI

From this info you do not have many users on this system. and it is failry happy.

Your dbc_max_pct is allocating 1.8 Gig of approxx 1.4 gig could be released by changing it to 5 and dbc_min_pct to 1.

The attached doc will help.

The :-

? 8245 sybase 241 20 17472K 4380K run 1350:40 98.91 98.74 dataserv
3 ? 8226 sybase 241 20 17472K 4380K run 1380:32 98.88 98.71 dataserv

Seem very busy although the server is quiet.

Have you tried a reboot in order to try to resolve the delay?

Paula

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Thu, 22 Aug 2002 11:02:14 GMT

Hi

In the Doc ignore the :-

"I would not be happy with a straight adjustment to 10% and feel that the safest option is to initially bring the setting down to 25% and if OK then readjust the size downwards to this "Just Enough" setting."

As I wrote this in respect of a particular server.

Paula

Re: Wait Reason: SYSTM

Paula J Frazer-Campbell — Thu, 22 Aug 2002 11:07:27 GMT

Gordon

From top:-

8 stopped

What are these ?

Paula