Operating System - OpenVMS
1748213 Members
3023 Online
108759 Solutions
New Discussion

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

 
Edmundo T Rodriguez
Frequent Advisor

Help! - Performance Problem - Can't find 4 CPU are been loaded!

Hi

 

I been leading with a system tuning for a while and cant find a way to determine why

few specifics type of process are ocerloading the 4 CPU on this OpenVMS Alpa ES47

 

The application is Centricity Business with Cache (previously name IDX-FlowCast)

Normally when some specific type of processes which generated reports in background

even they are running on low priority they take high CPU. This system is normally loede with

800 + interactive users with NO issue until these type of processes run, where the monitor

mode chart show a continous steady User-Mode of 365% (4 CPU) If they are not running

it stays around 230 %

 

Attached is a file with all information I believe is pertinent to obtain an idea

how the system has been setup (parameters, etc) and its behavior.

 

Can anybody notice anything weird that aI may be overlooking?

 

        Please, dont reply if you don't take time to analyze the attachement.

 

Thank you.

 

 

 

 

14 REPLIES 14
Volker Halle
Honored Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

Edmundo,

 

those jobs are cache jobs. You need to find out, what they are doing. Maybe looping for something to happen instead of waiting ?

 

Volker.

Edmundo T Rodriguez
Frequent Advisor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

 

I been monitoring them and they are having a lot of IO because they are actually

searching-writing. Theya are the type process we call report attached processes.

 

 

Volker Halle
Honored Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

Edmundo,

 

so these processes are searching in the database and writing reports - o.k. With an OpenVMS priority of 0, they are not taking CPU cycles aways from others processes with higher priorities, they are just consuming all the - otherwise idle - CPU cycles.

 

What is the perceived problem ? Is there any ? Except the system being at 365% CPU utilization ?

 

Volker.

Edmundo T Rodriguez
Frequent Advisor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

Hi Volker

 

Yes, there are two problems

 

1. the actual reports are taking too long to be produced

2.  sometime users are complaining about system been slow

 

   I know as you may notice in the attachement that some of the disk-volumes are quiet fragmented

  some of them hold huge Caché (MUMPs) databases and duw to this a 24x7/365 system is very difficult

  to obtain time to work in disk or database de-fragmentation/compation. This may be causing high IO

  hence CPU but I have not been able to directly relate this type of specific reports run with that effect.

 

I may have some fog in my eyes!

 

Volker Halle
Honored Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

Edmundo,

 

severe disk fragmentation is only a noticable problem, if there is signifcant IO on those fragmented disks. As the Window Turn Rate is very low and the most fragmented disks (except maybe DSA450:) have a very light IO rate, I don't think disk fragmentation is a problem here.

 

MONITOR may not be the right tool to investigate such a performance problem, especially if the 'slow performance' only show up 'sometimes'. Consider using T4, it will give you much more detailled performance data at 1-minute intervals.

 

Volker.

 

Mike Kier
Valued Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

I agree with Volker.

 

T4 with TLVIZ can help spot correlations that are not otherwise readily apparent.  Also you can add application-specific data items to the analysis.


I wouldn't be too quick to rule out an application-related cause to the perceived slow downs.  I didn't see any OS locking statistics in your post, but contention for a shared resource, like a shared counter, can often cause applications to spin in user mode (I don't know how cache internally handles such things or what paradigms are available to the programmer to deal with them; its conceivable they may not even use the distributed lock manager and therefore be invisible to Monitor).

Practice Random Acts of VMS Marketing
abrsvc
Respected Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

I too agree with the previous 2 replies. I would be looking for resource contention around record locking/access. With the rather large number of reports running, you increase the chance of a record being locked. Also, can you check to see whether or not the reports do any updates? I have seen similar problems when reports that should be read-only for data access are not set that way.

Dan
John McL
Trusted Contributor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

What length are the I/O queues to your disks?

 

$ MONITOR DISK/ITEM=QUEUE_LENGTH

 

Even if the queue is 0.3 on average, that means that a disk I/O will have to wait 1 time in 3 because there's an I/O request  ahead of it. 

 

Also, is this a recent problem on what was a happily running system? If so, what's changed?

Edmundo T Rodriguez
Frequent Advisor

Re: Help! - Performance Problem - Can't find 4 CPU are been loaded!

HI again

 

I am sorry for takin too long to foolow your replies. Thank you all for the insight.

 

No doubt about it in connection with Monitor vs. T4... I do run T4 for long time

I tought it was much simple to provide that Monitor chart instead of embeading T4 charts.

 

Now, take a look at the attached presentation based on T4 for the hole month of may-2014

and you are going to notice more clearly the difference whn I have those type of reporting processes

and those that are not.

 

Look at the beginning of the month when we run the closing and all Mondays (2, 12, 19)

What I noticed is that that CPU load doesn't correlate with CPU acticity in other modes

like Direct IO, Buffere IO, Interrupt or MP Sync. Nor with Monitor Compute state.

 

What I do notice is that seems like the overall cache, specifically Cache reads (High) and Free cache (Low)

do correlate some how. The only thing I did not long ago was to increase some of the cache seen on

"File System Caching" like Dir-Data, File-Hdr, File-ID connected with ACP_* quotas and then run autogen.

But only see a little bit improvement.

 

Some how is my believe that Disk IO (Files (db) sear and writes) are affecting).

 

Anyhow, please take a look at the T4 *comp.VSC (if you have TlViz) for Monday-Tuesday

and let me know if you can see anything disturbing.