Vax system freezing up

[vmsserbo]

My offices have been experiencing a problem with our Vax system 4100 the last couple days. It will freeze up for about 5 minutes at a time. When it freezes up users cannot navigate from screen to screen, some users get kicked off, and some users’ screens just freeze. If people get kicked off and try to get back on the username prompt does not come up and the cursor just blinks on the screen. This happened once last Friday and it has happened a couple times this morning. This is happening to all of my offices across the state at the same time, so I know it’s not a network problem in one office. After about five minutes, it goes back to normal and everyone is able to login.

Any suggestions on what may be cause would be appreciated!

[melvyn burnard]

posted in completely the wrong forum.
moved to more appropriate forum

[Steven Schweda]

The first step might be asking in a
VMS-related forum, where more people are more
likely to know what a VAX is.

"SHOW ERRORS"?

Anything in "SYS$MANAGER:OPERATOR.LOG"?

There are many possibilities, including a
high-priority process eating the whole CPU,
a bad disk drive, bad network hardware, and
so on.

"navigate from screen to screen" in what?
"kicked off" how? By whom?

Is anyone connected to a local serial port,
or is everyone connected through the
network?

[Jan van den Ende]

Miles,

_WAY_ too little info to solve this, but (wild shot in the dark) can you do a
$ SHOW DEV/FUL for every disk on that system?
Look for the HIGHWATER setting. If they are all disabled, forget about my hunch.
If you find some (if only one) then do a
SET VOLUME /NOHIGH for each.
Doing some decent-sized allocations on /highwater drives CAN cause just these symptoms.
Then again, I might be completely wrong here. Not enough info....

Proost.

Have one on me.

jpe

[vmsserbo]

All disabled!

[labadie_1]

Check the pagefile utilization and the non-paged pool.

Can you do
$ mc agen$feedback
(this just gives back the prompt, but creates a new file sys$system:agenfeedback.dat, with peak values)
and post the result of the following command
$ sea sys$system:agen$feedback.dat pagef,fail
$ sh mem/fil

Ideally, try to have a process (batch, detached, interactive...) doing something like

$ loop
$ sh sys/out=a.tmp
$ wait 00:01:00
$ goto loop

to see if you have processes in "interesting states" (rwxxx, pfg...)

Good hunt.

[Martin Hughes]

When you experience freezes, is it just with application users? or does it also affect your DCL login?.

Assuming that all remote connections freeze, I would be trying to connect to the local console to troubleshoot when the problem happens again. This will either give you a login that works to troubleshoot, or if it also freezes, at least this will eliminate some possible causes.

As Steven suggested, check your operator log, and also check your binary errorlog.

[Robert Gezelter]

Miles,

As others have pointed out, there are any number of hardware problems, resource shortages, and other behaviors that can give rise to what appears to be a "hang".

Whether the system is actually hung, or merely VERY busy is another possibility.

This problem is one of the class of problems where troubleshooting in the forum is very difficult.

For example, while a communications problem at one office is somewhat ruled out, a problem somewhere in the network backbone is still a distinct possibility.

Are there any messages coming out on the console when the system appears to be hung?

While it is self serving (my firm has done, and does research this type of problem for clients), if this is a recurring problem, retaining experienced assistance is likely the shortest path to a solution.

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

Robert G. is correct; this is going to be difficult and problematic to diagnose in a tiny little window. The process is going to be interative.

Get a connection into the system console port on this VAX 4000 model 100 series box, via a redundant path; via remote (secured) modem dial-in or otherwise. The point here is to figure out if the system is hung, or the network is hung.

Software parameters are certainly a possibility, but I'd be aiming first at the hardware and the local network backbone. At the system error log, and at communications errors that can get logged.

Oh, and just because it hits all offices in the state doesn't mean that all the connections don't go through a bad router in, say, Hawaii -- network wires don't always run where you'd expect. (Ok, so I've been hoping for an on-site call for a bad router in Hawaii.) Even multiple vendors and supposedly redundant communications vendors can run all the connections through some rather odd paths, as the occasional backhoe operator has found.

But in any case, the connection into the console will give you some clue as to the particular source of the problem.

If the system is wedged at the console, then the next step can involve recording a series of PC addresses and then initiating a forced crash at the console, and the creation of a crashdump. From a crashdump, some idea of the trigger for the problem can become visible.

Disclosure: as is Robert G., HoffmanLabs is also in this troubleshooting business.

Stephen Hoffman
HoffmanLabs

[Volker Halle]

Miles,

as others have said, system hangs - especially when they are temporary - are extremely hard to diagnose. You need to collect as much information as possible and carefully observe the symptoms.

Try to connect a 'console manager' type application to the console port (OPA0:) - a laptop with a terminal emulator and an activated session log file will do. Make sure OPA0: is SET TERM/BROADCAST/PERM.

Consider to use AMDS (RMDRIVER) data collector on the OpenVMS VAX system and run Availabilty Manager on a separate node to capture any events automatically reported by AMDS.

Check errorlog and OPERATOR.LOG for any related entries at the internval of the system hang.

Try to think of a couple of specific tests to be executed when the system hangs next time, to rule out or confirm certain possible scenarios.

- ping the node
- have 'hung' users type CTRL-T (make sure it's enabled)
- try to issue commands on OPA0: or another local terminal

Volker.