pathworks logging

[Kirk Reindl]

I'm new to VMS/pathworks. We recently had a blip in pathworks where all of a sudden no one could connect to their shares. A colleague of mine stopped and started the pathworks service and the shares are now accessable.

Could you tell an HPUX guy how to interogate the pathwork log files so I could maybe get a clue as to why we had this outage?? Even if I don't find anything, I'd like to know how to examine.

Thanks,
Kirk

[Ian Miller.]

see directories referenced by the logical names
PWRK$LMLOGS and PWRK$LOGS

They contain various .log files which may be of interest.

[Lokesh_2]

Hi,

Which version of pathworks you are running ( newer versions are known as advanced server ) . Also, over what protocol you access your pathworks share ..tcpip, decnet..etc. For eg. if over tcpip, pwrk$knbdaemon process is needed. So, for example you are not able to access your shares, check for the pwrk processes ( sho sys/proc=*pwrk* ) and check whether knbdaemon is there or not....

HTH,
Thanks & regards,
Lokesh

[Kirk Reindl]

Lokesh:

We are are making the share connection using tcpip. Here is some info that should answer the rest of your questions:

OSIJX1> pwshow
OpenVMS V7.3-1 on node OSIJX1 30-JAN-2004 12:16:00.18 Uptime 19 14:32:16
Pid Process Name State Pri I/O CPU Page flts Pages
2062A987 PWRK$ADMIN_0 LEF 6 62 0 00:00:00.03 177 113
2062997E PWRK$KNBDAEMON HIB 12 108 0 00:00:00.22 278 319
20627980 PWRK$LICENSE_R HIB 11 777 0 00:00:00.31 564 485
2062318B PWRK$LMBROWSER HIB 9 282 0 00:00:00.51 800 728
20628985 PWRK$LMMCP HIB 11 8331 0 00:00:01.22 1259 495
20626989 PWRK$LMSRV HIB 10 287082 0 00:01:32.68 2859 2252
2062A183 PWRK$MASTER HIB 6 159 0 00:00:00.17 463 168
2062697C PWRK$NBDAEMON HIB 10 62 0 00:00:00.04 129 160
OSIJX1>


The log files I checked in PWRKS$LMLOGS and PWRK$LOGS didn't reveal too much. I can see the pathworks service is coming up but that is about it.

Any other suggestions?
thanks

[Antoniov. · Accepted solution]

Hi Kirk,
for better help you we need more info.
Type:
$ ADMIN
ADMIN>SH VER
..... please report the answer
ADMIN>SHOW AUDIT POLICY
..... please report the answer
.. Then you could see event log
ADMIN>SHOW EVENT /SINCE=TODAY
.. and you can see connection/session
ADMIN>SHOW SESSION
ADMIN>SHOW CONNECTION
.. and you have help on line
ADMIN>HELP

If you have license problem you can type
$ ADMIN/LICENSE

Antoniov

[Dave Gudewicz]

We have the same problem from time to time.

Checking event viewer system on a v7.3 advanced server tells us that a PDC is not being found. Event 5719

Checking same on a v7.3a eco2 server tells us nothing.

Toggling the server puts humpty back together again.

Above Advanced Servers are configured as member servers in 2 different domains. They live on what I would call a large network. Hundreds of other servers thousands of desktops. Connections to these servers number in the dozens at most.

[Kirk Reindl]

Thanks for all replies. If anything becomes of this I learned some good pathwork admin commands. It's funny, it is one of those things if I don't troubleshoot why we had this outage, it will happen again.

Here is the output you requested.

OSIJX1> admin
CAPITAL\\OSIJX1> show ver

Advanced Server V7.3A for OpenVMS

CAPITAL\\OSIJX1>

SYS_OSIJX1>admin
CAPITAL\\OSIJX1> SHOW AUDIT policy

Audit Policy for domain "CAPITAL":

Auditing is currently Disabled.

Audit Event states:

Audit Event Success Failure
------------------ -------- --------
ACCESS Disabled Disabled
ACCOUNT_MANAGEMENT Disabled Disabled
LOGONOFF Disabled Disabled
POLICY_CHANGE Disabled Disabled
PROCESS Disabled Disabled
SYSTEM Disabled Disabled
USER_RIGHTS Disabled Disabled

CAPITAL\\OSIJX1>

CAPITAL\\OSIJX1> SHOW EVENT /SINCE=TODAY

Events in System Event Log on server "OSIJX1":

T Date Time Source Category Event User Computer
- -------- ----------- --------- --------------- ------ -------------------- ---------------
I 01/30/04 08:50:48 AM Eventlog None 6005 N/A OSIJX1
W 01/30/04 08:50:43 AM BROWSER None 8021 N/A OSIJX2
W 01/30/04 08:50:15 AM BROWSER None 8021 N/A OSIJX2
W 01/30/04 08:50:15 AM BROWSER None 8021 N/A OSIJX2
W 01/30/04 08:50:04 AM BROWSER None 8021 N/A OSIJX2
W 01/30/04 08:50:04 AM BROWSER None 8021 N/A OSIJX2
W 01/30/04 08:49:53 AM BROWSER None 8021 N/A OSIJX2

Total of 7 events

CAPITAL\\OSIJX1>

CAPITAL\\OSIJX1> SHOW SESSION

User Sessions on server "OSIJX1":

Connected Users Computer Opens Time Idle Guest
-------------------- ----------------- ----- -------- -------- -----
PC12015 1 0 07:33 0 07:33 No
PC17173 0 0 07:27 0 07:27 No
ACOLE PC14653 0 0 07:04 0 06:58 No
ACOLE PC14653 0 0 07:04 0 06:58 No
ACOLE PC14653 0 0 07:04 0 06:58 No
ADMINISTRATOR OSIJX1_2062B12F 1 0 02:49 0 00:00 No
ADMINISTRATOR OSIJX1_2062B12F 0 0 02:49 0 02:49 No
AHILL ANGEL 0 0 07:43 0 04:22 No
ATURNER PC11813 0 0 07:37 0 04:39 No
ATURNER PC11813 0 0 07:37 0 04:39 No
ATURNER PC11813 0 0 07:37 0 04:39 No
ATURNER PC11813 0 0 07:37 0 04:39 No
CHECKWRK PCCHECK 0 0 07:48 0 00:00 No
DEDMONDS PC17177 0 0 06:19 0 00:10 No
JCLARK PC14515 0 0 03:46 0 01:36 No
JCLARK PC14515 0 0 03:46 0 03:33 No
KREINDL PC12015 1 0 07:33 0 02:30 No
MDEZOMA PC14599 0 0 06:03 0 01:06 No
MGREENE PC14300 0 0 07:51 0 00:06 No
OAJAYI PC17173 0 0 07:27 0 07:15 No
OSCAP OSCAP 0 0 07:55 0 00:00 No
OSCAP PCJX04 0 0 07:55 0 00:00 No
PWYNN PC14363 0 0 07:03 0 00:40 No
RKILLORY PCJXVP 0 0 07:36 0 00:39 No
RLEE PC14601 0 0 07:42 0 00:48 No
RLEE PC14601 1 0 06:06 0 06:06 No
TSIMPSON PC15815 0 0 00:46 0 00:19 No

Total of 27 connected users

CAPITAL\\OSIJX1>

CAPITAL\\OSIJX1> SHOW CONNECTION

Connections on server "OSIJX1":

User name Computer name Share name Opens Time
-------------------- --------------- ------------ ----- --------
ACOLE PC14653 CCC_PCDATA 0 0 07:06
ACOLE PC14653 JAX_ADMIN 0 0 07:06
ACOLE PC14653 JCLARK 0 0 07:06
ADMINISTRATOR OSIJX1_2062B12F ADMIN$ 0 0 02:51
ADMINISTRATOR OSIJX1_2062B12F IPC$ 1 0 02:51
AHILL ANGEL BCPMS 0 0 07:45
AHILL ANGEL CCC_PCDATA 0 0 07:45
AHILL ANGEL JAX_ADMIN 0 0 07:45
ATURNER PC11813 BCPMS 0 0 07:39
ATURNER PC11813 BC_CLIENTS 0 0 07:39
ATURNER PC11813 CCC_PCDATA 0 0 07:39
ATURNER PC11813 JAX_ADMIN 0 0 07:39
CHECKWRK PCCHECK CHECKWRK 0 0 07:45
CHECKWRK PCCHECK MS 0 0 07:50
DEDMONDS PC17177 API_PCDATA 0 0 06:21
DEDMONDS PC17177 CCC_PCDATA 0 0 06:21
JCLARK PC14515 BCPMS 0 0 03:48
JCLARK PC14515 CCC_PCDATA 0 0 03:48
KREINDL PC12015 IPC$ 1 0 07:35
KREINDL PC12015 PWCOMMON 0 0 07:35
MDEZOMA PC14599 BCPMS 0 0 06:05
MGREENE PC14300 GREENE 0 0 07:53
MGREENE PC14300 PCREPORT 0 0 05:27
MGREENE PC14300 SRB 0 0 04:47
OAJAYI PC17173 API_PCDATA 0 0 07:29
OSCAP OSCAP EXPERIAN 0 0 07:54
OSCAP OSCAP HEILIG_CAP 0 0 07:57
OSCAP OSCAP MDIST 0 0 06:05
OSCAP PCJX04 CHECKWRK2 0 0 07:56
OSCAP PCJX04 HEILIG_PFS 0 0 07:57
OSCAP PCJX04 VERIZON 0 0 07:56
PWYNN PC14363 API_PCDATA 0 0 00:54
PWYNN PC14363 FFL_PCDATA 0 0 00:54
PWYNN PC14363 KITS 0 0 00:54
PWYNN PC14363 OHI_PCDATA 0 0 00:54
PWYNN PC14363 OSICS_DOCS 0 0 00:54
PWYNN PC14363 PCREPORT 0 0 06:09
PWYNN PC14363 PWCOMMON 0 0 00:54
PWYNN PC14363 PWV51 0 0 00:54
PWYNN PC14363 SRB 0 0 00:54
PWYNN PC14363 TASKDB 0 0 00:54
PWYNN PC14363 TEST 0 0 00:54
PWYNN PC14363 WYNN 0 0 00:54
RKILLORY PCJXVP API_PCDATA 0 0 06:59
RKILLORY PCJXVP BCPMS 0 0 06:59
RKILLORY PCJXVP CCC_PCDATA 0 0 06:59
RKILLORY PCJXVP KITS 0 0 06:59
RKILLORY PCJXVP OHI_PCDATA 0 0 06:59
RKILLORY PCJXVP PWCOMMON 0 0 06:59
RKILLORY PCJXVP SQL_BRIDGE 0 0 06:59
RLEE PC14601 IPC$ 1 0 06:08
TSIMPSON PC15815 TEST 0 0 00:48

Total of 52 connections

CAPITAL\\OSIJX1>

[Antoniov.]

Hello Kirk,
you are very precise posying all information about your system; first one now I know you have A/S (Advanced Server that's new name of Pathwork) V7.3A that's last HP released.
You need interrogate system so you can read A/S administrattion guide, here there is page about troubleshooting http://h71000.www7.hp.com/doc/73final/6543/6543pro_016.html#events_ch
Look at 6.1.4 you can find all log files and their meaning.

Antoniov

[Paul Nunez]

Hi Kirk,

First, welcome to the world of OpenVMS and Advanced Server for OpenVMS. I hope it's all you expect and more. As you can see, there's plenty of help here and there's an Internet newsgroup called comp.os.vms that might be of interest too. As for your Advanced Server issue...

The only true way to know what version of Advanced Server (and patches thereto) you're running is to use $ PWVER (must have run $ @sys$startup:pwrk$define_commands.com first). I suspect you're not running ECO2; the image version for eco2 is v7.3-120a (eco1 would be v7.3-110a and original version would be v7.3-100a).

There are known problems in v7.3a and eco1 which can lead to all sessions being consumed. Some of those problems are fixed in ECO2, so that's the place to start.

How many "clients" do you have the server configured for ($ admin/config). I noticed in your session display that a few clients had more than 2 sessions (which should rarely occur). That's a symptom of the problem I'm referring to (known as the "pipe leak" problem).

Are there any errors in pwrk$logs:pwrk$knbdaemon_.log;-n pleading for an increase in the number of clients?

ECO2 is available from the itrc ftp server:

ftp://ftp.itrc.hp.com/openvms_patches/layered_products/alpha/ADVANCSERV-V0703-A2.PCSI-DCX_AXPEXE

(be careful, because there are Japanese versions of AdvSrv kits there too).

There's one add'l "log" to be aware of and monitor/manage. It's officially known as the comment event log, and it will log license related errors, operating system (OpenVMS) errors, and other errors that can't be posted to the NT-style System event log. The common event log is viewable/managed using the $ ADMIN/ANALYZE command (there's no on-line help for this one; you must refer to the docs for a command description, syntax, etc).

So a quick recap:

Use $ pwshow to verify the pwrk$lmsrv, pwrk$lmmcp and pwrk$knbdaemon processes are running. If not, restart and then check pwrk$lmlogs: and pwrk$logs: for .dmp files. Check tail end of process log files for errors and traceback info.

If all processes are present, is pwrk$lmsrv (and to a lesser extent, pwrk$lmmcp) doing anything or is it stuck in LEF or some other bad state? On the other end of the spectrum, pwrk$lmsrv could be consuming significants amount of CPU and doing large numbers of buffered i/o and sometime equally as large number of direct i/o (use $ show proc/cont/id= to see the rates).

If it's using a lot of CPU, that's often a client flooding the server with requests. For that you can use:

$ pwmon client/top=operations

Dir Lookups and Transact columns should be watched closely - these are expensive and often the client is just looping. The server is keeping up as best it can. You can also use $ TCPTRACE (or tcpdump) to get network traces and analyze them on a Windows client using Ethereal (www.ethereal.com).

If the pwrk$lm* processes seem idle ($ PWSHOW shows them in HIB state), do all $ ADMIN commands also hang and eventually fail? If so, you may be out of sessions. Use the tools:

$ pipe nbshow knbstatus | search sys$input use

to see if the number of sessions in use is at or very near the maximum number of sessions.

You can get a good/accurate list of TCP/IP sessions, including thoses reserved by various Advanced Server processes, using:

$ nbshow knb

similarly you can use the SDA ($ ANALYZE/SYSTEM) command:

SDA> PWRK TRANS KNB SESSIONS

to see those sessions.

BTW, you also have the PWRK$NBDAEMON (note the missing "K") process running which is the NetBEUI protocol. Unless you really need it, I highly recommend disabling that ($ ADMIN/CONFIG --> Transports). You can use $ NBSHOW NB and $ NBSHOW NBSTATUS to see NetBEUI session info (to verify no one is using it; even if they are, they can likely reach the server using tcp/ip as well).

Of course, I still encourage you to read the docs and release notes :) They're actually very well written (unlike some docs). The only caveat is that the $ PWMON utility is not supported/documented (other than $ pwmon help).


Again, welcome to OpenVMS (and AdvSrv)...

HTH,

Paul Nunez
HP Services

[Brad McCusker]

Wow - Its too bad you can't give Paul about 20 points for that last answer. But, he is _the_ man when it comes to PW/AS support, so listen to what he says carefully.

The one thing I will add I think will help you understand address your server stopped working the first time. Look in the log file PWRK$LMLOGS:PWRK$LMSRV*.LOG which was active at the time server stopped. Look at the last 30 or so lines - D you see a process traceback? If so, that might tell us a lot about what the problem was, and, if its a known problem, what you can do about it. If there is a traceback, post it, and, we might be able to give you more information.

Brad McCusker