topic Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours in Operating System - HP-UX

shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

wvsa — Wed, 29 Sep 2004 15:55:21 GMT

Greetings fellow admins

We have oracle apps environment, have noticed from time to time the f60webmx jobs which are basically oracle forms processes going to a runaway state consuming 100% of a cpu. Is there anyway to write a script or ? that would detect these processes and alert us.

Thank you for input,it is truly appreciated.

wvsa

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

Prashant Zanwar_4 — Wed, 29 Sep 2004 16:00:43 GMT

Just run top and observe for CPUTIME...No need to do anything extra.

Thanks
Prashant

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

Sridhar Bhaskarla — Wed, 29 Sep 2004 16:10:33 GMT

Hi,

The best way is to configure 'alarmdef' of measureware and get alerted on it. It does a good job of keeping track of the history as well reporting.

Otherway is to use the grand old 'ps' and gather the data. For ex.,

UNIX95= ps -C "sshd" -o "pcpu args" |sort -n

Should give you the %utilization of the process sshd. You can take samples of these utilizations for every 15 mins or so and keep track of processes that are at 100% and report them after 2 hours. Bit scripting is involved but that's not difficult to write.
You can specify multiple processes with -C option such as -C "sshd,inetd' etc.,

-Sri

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

Sundar_7 — Wed, 29 Sep 2004 16:21:33 GMT

WVSA,

It is relatively easy and accurate to implement this using Measureware alarmgen.

Do you have measureware agent installed in the system ? -

If you do

1) # vi /var/opt/perf/alarmdef

threshold=100
PROCESS LOOP
{
if (PROC_PROC_NAME == "f60webmx") then
{
if (PROC_CPU_TOTAL_TIME > 7200) and (PROC_CPU_TOTAL_UTIL_CUM > 100) then
{
exec "/usr/local/bin/run-away.ksh", PROC_PROC_ID
}
}
}
#

Create the script /usr/local/bin/run-away.sh and process the PID provided as the command line parameter to the script.

Once the script is ready, you need to restart the MWA alarmgen

# mwa restart alarmgen

You might have to do some tweaking to the measureware alarm definition.

- Sundar.

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

wvsa — Thu, 30 Sep 2004 13:25:43 GMT

Sundar,

Thank you for your response, going to test first by using mailx to mail the procid to me.

Thanks to all who responded

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

Sundar_7 — Thu, 30 Sep 2004 13:42:57 GMT

Hi Wvsa,

Sorry, there are couple of blunders in my post...

1) # vi /var/opt/perf/alarmdef

threshold=100
PROCESS LOOP
{
if (PROC_PROC_NAME == "f60webmx") then
{
if (PROC_CPU_TOTAL_TIME > 7200) and (PROC_CPU_TOTAL_UTIL_CUM >= threshold) then
{
exec "/usr/local/bin/run-away.ksh", PROC_PROC_ID
}
}
}
#

and

# mwa restart alarm

Good luck :-)

- Sundar.

Re: shell script or ? to find all processes that have used 100% of cpu for more than 2 hours

wvsa — Thu, 30 Sep 2004 14:01:25 GMT

Sundar,
Thanks for getting back to me and pointing out the modification.

wvsa