Operating System - OpenVMS
1828478 Members
2989 Online
109978 Solutions
New Discussion

Re: Bad performance Openvms

 
SOLVED
Go to solution
geir_2
Super Advisor

Bad performance Openvms

Hi,

The system performance is decreasing the last month.

The show process command list all process on the system with belonging state and priority.
Is there some special state or columns that i must be aware on? Other suggestions?

The system have one CPU, no NFS.

Rgds Geir
21 REPLIES 21
Robert Gezelter
Honored Contributor

Re: Bad performance Openvms

Geir,

Many things can cause performance problems that appear to steadily get worse. The problems can range from poorly designed applications to steadily worsening hardware problems.

The SHOW SYSTEM command will show you the process states and the priorities, but that will not necessarily lead you to a result. That report is based upon the exact instant in time that the command is processed. MONITOR PROCESS/TOPCPU would be more helpful in identifying if there is a process consuming all available CPU time.

However, as I mentioned earlier, that is but one of the possible underlying causes of decreasing performance. There are many others.

Do you have some additonal information about the system workload?

- Bob Gezelter, http://www.rlgsc.com
Wim Van den Wyngaert
Honored Contributor

Re: Bad performance Openvms

Ian Miller.
Honored Contributor

Re: Bad performance Openvms

What are you measuring to know the system performance is decreasing? User reports or actual figures of reduced application thoughput?

Do you have records of performance data (e.g from MONITOR) which cold be compared?
____________________
Purely Personal Opinion
Karl Rohwedder
Honored Contributor

Re: Bad performance Openvms

Geir,

1st question: has something changed, e.g.
number of users, new version of software (system or application), hardware, network,...?

2nd: has something gone bad, e.g. memory, disk?

Check Errorlog, SHOW MEMORY, MONITOR *.
SHOW SYSTEM -> processes with xxxO (outswapped, e.g. COMO)

Do you have hard data for the performance degradation or is this more a feeling/user complaint?

regards Kalle
geir_2
Super Advisor

Re: Bad performance Openvms

Hello,

Everyting seems to be normal:

1) Normal traffic on the network according to cisco router
2) No swapped out process
3) 15 interactive users logged in on the system
4) According to monitor process/topcpu
no process have more than 20% CPU. Is there other commands that show the workload on system?
5) The performance is degraded, I have not installed new software, users and hardware.
6) The system is running DECnet IV, TCP/IP and LAT
Antoniov.
Honored Contributor

Re: Bad performance Openvms

Hi Geir,
Do your application running using RMS files? May be internal structure is degraded.

Antonio Vigliotti
Antonio Maria Vigliotti
Jan van den Ende
Honored Contributor

Re: Bad performance Openvms

Geir,

have you somehow passed the threshold of starting to use paging?

SHOW MEM/FILE should indicate the amount used, and with
MONI PAGE
you can get an indication of paging activity.

Depending on the answers, totally different things need further investigating.

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
labadie_1
Honored Contributor

Re: Bad performance Openvms

what applications run on your system ? database, rms files ?

Anyway, you should run a performance collector tool (monitor is on any Vms system, Ecp...), and you should then be able to say you have a Cpu saturated problem, or an I/O problem (disk, bus, file saturated), or a memory problem, or a lock problem...
Andy Bustamante
Honored Contributor

Re: Bad performance Openvms


The first question is what is the source of "system performance is decreasing"? What application and how are you measure system performance?

You have a few system replies so far, in addtion has there been any network changes? You can check the Alpha NIC for errors with

$ mc lancp show dev /counter

If you don't have time to do it right, when will you have time to do it over? Reach me at first_name + "." + last_name at sysmanager net
Douglas Graver
New Member

Re: Bad performance Openvms


What does $ MONITOR DISK/ITEM=QUEUE show?
IDX Systems Corp.
Richard White_5
Advisor
Solution

Re: Bad performance Openvms

Good Morning Geir...

As if you have not already received enough responses to your dilemma, I thought that I would add a few comments as well. You have already verified that no processes are in the "out-swapped" state, but you should also verify that there are no processes that are constantly/continuously in any sort of MWAIT/RWAIT state. This can be checked via the $mon stat or $mon sys utility or $show sys command. Any process in RWAST/RWJIB usually indicate that process/job has insufficient process quotas in Sysuaf.Dat.

A few other commands for the "Monitor" utility may prove beneficial.

$mon clus is an excellent utility to quickly check how much cpu or memory utilization the system is using, (are you approaching 100% saturation,) as well as the top 6 disk-volumes for QIO's. The system does NOT need to be in a cluster to use this command. But, if this system is in a cluster, you may compare this node to the other nodes (up to 6) in the cluster, to verify consistency amongst the other nodes in the cluster.

$mon rlock and $mon dlock will show you any "dynamic-lock-remastering" as well as local vs. remote locking that is happening, if this system is indeed part of a cluster.

$mon proc/topdio may point to a process that is performing heavy direct-io, possibly to one of the top 6 disk-volumes previously shown in the $mon clus utility.

$mon fcp will show you some of the XQP/ACP File system statistics. Typically the FCP-Call-Rate is a combination of Disk-Reads + Disk-Writes, along with File-Opens and File-Lookups. Excessive overhead/stall times can be seen if the system is performing a high number of "window-turns", which would indicate fragmented files. Also Erase-Rates and Volume-Lock-Waits should be minimal-to-none, for best performance.

$mon proc/topbio will show you any processes that may be executing buffered-io perhaps to a mailbox or a network device. You want to make sure that the Mailbox Reader is at the same priority or higher than the Mailbox Writer; and that the mailbox buffer-quota is large enough, in case you have multiple mb-writers to the same mailbox. If not, you could notice a process going into RWMBX Wait-State.

$mon time will show you the number and types of TQE's the system is performing per second. Depending on your applications and user-workload, you will typically have more "System-Subroutine" TQE's (100-200), followed by "Timer" TQE's (50-150), and "Wakeup" TQE's (1-50). However, be advised that no two systems or applications are exactly the same - your mileage-numbers may vary...

I'm not sure that I saw what version of OVMS you are running, nor what platform, but a couple of these utilities were not implemented until v7.x Hope this helps..

Thanx,
whynot3k
Doug_81
Frequent Advisor

Re: Bad performance Openvms

Hi Geir:
You've had a lot of good suggestions to start working on. I thought I'd add one more possibility.

I had a system that was normally running great until suddenly, for no apparent reason, it became a real dog. i.e. 10-15 second response to a keyboard entry.

It turned out to be a workstation which was connected to the network with a bad cable (the connector was loose), so it kept connecting, loosing connection, reconnecting, etc.

This workstation was also setup to automatically log a user on, so each time it "connects", it would get a login prompt.

When you get a login prompt on VMS, you're are running loginout process and it runs at a very high priority so it will kill the system performance.

I found this when I issued a multiple
$) show user/node/fu
commands. This showed me that someone was connecting and trying to logon, then it would disappear, then re-appear, etc.

Since there was no actual login completed, I couldn't determine where it was coming from, so I got some assistance from the network boys using a sniffer.

Good luck,
Doug
Garry Fruth
Trusted Contributor

Re: Bad performance Openvms

This is a rather complex subject. You may want to use the OpenVMS Performance Management manual at http://h71000.www7.hp.com/doc/73final/6491/6491pro.html. Appendix A has a decision tree which will give you a rough outline of how to investigate.
comarow
Trusted Contributor

Re: Bad performance Openvms

Ok,
The cookbook is great place to start

Watch mon system,
what do you see?

Does the cpu state grow?


What version of vms?

Does it gradually get slow?

Mon disk/item=q do any of your disks
(except when backup is running have a queue depth ON THE AVERAGE of more than 1),

Show mem? Do you have free memory?
Are you low on page file space?

Is it also slow on the console?

Are you logging errors?

Do an
Anal/system
sda>show mem/free

do you have support?

Do you reorganize index files?

Bob
geir_2
Super Advisor

Re: Bad performance Openvms

Hi,
Thanks to everyone for outstanding suggestions. I have tried all commands, but it was impossible to find the reason for performance degradation.

At end I rebooted the system, and now it seems to be faster than ever. If not, I will go through the suggeastions once more.

Thanks to everyone:-)
Ian Miller.
Honored Contributor

Re: Bad performance Openvms

If you are not recording performance data then start now. Use T4 or EPC or other product but do use something.
____________________
Purely Personal Opinion
Galen Tackett
Valued Contributor

Re: Bad performance Openvms

If this crops up again, Availability Manager or AMDS might also provide some useful realtime information.
comarow
Trusted Contributor

Re: Bad performance Openvms

Of performance gradually gets slower turn off memory reclamation.

set npage_gentle=100
set npage_aggresive=100

otherwise non page pool with fragment.

Make sure npagedyn hasn't grown
(show mem/full/pool)
and compare current versus initial value.

To see how fragmented npagedyn is
Analy/system
clue mem/free

There should only be a few free elements.
If you see thousands, that's a problem there.

Performance problems are rarely a single factor, but this one is a killer.

In 8.2 they set those parameters to 100 by default.

Bob
Ian Miller.
Honored Contributor

Re: Bad performance Openvms

in extreme cases non-paged pool fragmentation can cause SPINWAIT bugchecks with some versions of VMS.
____________________
Purely Personal Opinion
Karl Rohwedder
Honored Contributor

Re: Bad performance Openvms

Pls. note the typo on parameter names:
it is NPAG_GENTLE and NPAG_AGGRESSIVE

regards Kalle
comarow
Trusted Contributor

Re: Bad performance Openvms

The typo corrections are important. Thanks.