topic Re: CPU sys:users:idle in Operating System - Tru64 Unix

CPU sys:users:idle

jack wu_2 — Mon, 02 Aug 2004 03:47:37 GMT

Hi all
I meet a problem on GS80 V5.1A+PK6.
the CPU used sys:user:idle is 39:22:39.
and the pages fault is too much, and system performance is very poor.
Is there a tools on Tru64 unix to analyze which process consume the CPU sys resources?
I list the output of vmstat 2.

Virtual Memory Statistics: (pagesize = 8192)
procs memory pages intr cpu
r w u act free wire fault cow zero react pin pout in sy cs us sy id
8 405 142 226K 457K 85K 5G 697M 3G 922 1G 0 6K 40K 13K 22 39 39
7 401 146 226K 457K 85K 10568 1327 5379 0 3027 0 8K 16K 16K 34 35 31
9 401 144 226K 457K 85K 13531 1418 6106 0 3095 0 8K 16K 16K 39 41 20
13 401 141 226K 457K 85K 13777 1566 7043 0 3470 0 8K 20K 16K 35 42 22
8 400 146 226K 457K 85K 14054 1504 7387 0 3465 0 9K 25K 16K 44 44 13
14 396 142 226K 457K 85K 13365 1437 6802 0 3280 0 8K 18K 15K 43 42 15
13 396 143 226K 457K 85K 11411 1463 6114 0 3198 0 8K 38K 14K 41 52 7

Thanks
Jack

Re: CPU sys:users:idle

Ralf Puchner — Mon, 02 Aug 2004 04:14:54 GMT

as written so much time: collect

Re: CPU sys:users:idle

jack wu_2 — Mon, 02 Aug 2004 04:22:04 GMT

Thanks
But collect can't tell me the CPU sys cause by which process?

#### RECORD 1 (1091397664:200) (Mon Aug 2 06:01:04 2004) ####

# CPU SUMMARY
# USER SYS IDLE WAIT INTR SYSC CS RUNQ AVG5 AVG30 AVG60 FORK VFORK
15 75 9 1 11959 24770 18202 3 8.20 5.18 4.28 39.03 0.05
# SINGLE CPU STATISTICS
# CPU USER SYS IDLE WAIT
0 12 82 6 0
1 17 73 10 1
2 15 74 10 1
3 18 70 12 0

Jack

Re: CPU sys:users:idle

Michael Schulte zur Sur — Mon, 02 Aug 2004 04:30:28 GMT

Hi Jack,

when you post an outpout, please do it as attachment otherwise all extra spaces good for reading get cancelled out.

thanks,

Michael

Re: CPU sys:users:idle

jack wu_2 — Mon, 02 Aug 2004 04:40:27 GMT

Thanks
The attached include output of collect and top.
I hope to know how to analyze the which process cause system page fault and CPU sys.

Thanks
Jack

Re: CPU sys:users:idle

Hein van den Heuvel — Mon, 02 Aug 2004 09:27:01 GMT

WAG you have Oracle statistics enabled and statistics_level set to 'typical'. Try switch statistics off, and/or switch to basic level.

Use DCPI or at the very least KPROFILE to get an idea where the system tims is spend.

Looks like you are doing an unreasonable number of system calls, probably from Oracle. How about using system call TRACE for a few minutes for an oracle slave, and summary counts by call? read+write should dominate, but I suspect excessive 'getrusage'.

What Oracle version?
You have a lot of page faults, yet plenty of free memory. How about giving Oracle some more SGA and reboot system with a enough gh (granulariy hints region, gh_chunks / rad_gh_regions / gh_fail_if_no_mem = 0 )

You have a good few interupt / second also. Oracle network traffic? Tune ipc / SQl*net?
Is Oracle 'statspack' showing a lot of 'more_data' from/to sql?

Good luck,
Hein.

Re: CPU sys:users:idle

jack wu_2 — Tue, 03 Aug 2004 00:02:36 GMT

Hein
Thanks you very much.
The oracle is 8.1.7.4 and timed_statistics = true, but I mknod /dev/timedev.
Sorry , I don't know how to use KPORFILE.
I use trace one process, there is a lot of syscall_224 syscall_225 gettimeofday...., but I don't know how to summery and analyze the collect data.
About SGA, if I set ipc:ssm_threshold=0, then set gh_chunks/rad_gh_regions[0,1,2,3...] to allocate the SGA IPC memory.

Thanks
Jack

Re: CPU sys:users:idle

Hein van den Heuvel — Tue, 03 Aug 2004 01:22:33 GMT

>> The oracle is 8.1.7.4 and timed_statistics = true

Ok, The excessive use I was afraid about is is 9i problem, not 8i.

Still, since timed_statistics is dynamic you may want to switch it on and of over controlled period to doublecheck effects (should be less that 3%).

>> but I mknod /dev/timedev.

Great.

>> Sorry , I don't know how to use KPORFILE.

It's not too hard. Just 'man kprofile'.
Note, you might need to create a 'pfm' device: # sysconfig -c pfm

But it may be preferable to spend an extra hour and try to learn/use dcpi.
Start at: http://h30097.www3.hp.com/dcpi/ ?

>> I use trace one process, there is a lot of syscall_224 syscall_225 gettimeofday...., but I don't know how to summery and analyze the collect data.

I'm thinking about a small perl/awk script to post-process your trace data, adding up how many time you've seen a call. I oculd not find an examle quickly. I can write one if needed.

>> About SGA, if I set ipc:ssm_threshold=0, then set gh_chunks/rad_gh_regions[0,1,2,3...] to allocate the SGA IPC memory.

Sounds good. Is it working? Check my recent post here on 'big pages' for a tool that might help display '4mB page mapping usage.

thanks fro the feedback!
Hein

Re: CPU sys:users:idle

Ralf Puchner — Tue, 03 Aug 2004 04:04:24 GMT

And if all fails - open a call within the HP support center. Help us to save our jobs ;-)

Re: CPU sys:users:idle

jack wu_2 — Tue, 03 Aug 2004 04:40:25 GMT

Thanks everyone.

Jack