Phantom DNFS devices

[Brad McCusker]

i Folks,

I've got a strange one I've been puzzling over for the past month or so. It's by no means critical, just one of those things I'd like to understand better.

It's related to TCP/IP Services for OpenVMS:

$ tcpip show version

DIGITAL TCP/IP Services for OpenVMS Alpha Version V5.0A - ECO 2
on a AlphaServer ES40 running OpenVMS V7.1-2

$

(I know it's old. Upgrades, and related work are actually underway and should be complete by year end. But, as most of you know, upgrading a business critical VMS system is not something one does on a whim).

This is a single node system, no clustering. Runs Oracle Rdb, and a bunch of custom code built on Powerhouse.

This system is using NFS with one remote device mounted and always available. Our tools monitor the system for amongst many other things, the appearance of new devices. We can see that new DNFS devices are getting created, and within 10 minutes (the period of our device monitoring) the DNFS device is gone. It’s likely that the device is gone within seconds. We see this a couple times per day but suspect it’s happening much more frequently (The unit numbers between sitings are 10-30 higher) except our tools don’t catch it (device comes and goes between the 10 minute period of our tools doing their scan).

A typical device looks like:

Device DNFS131 is being monitored for the first time

Disk DNFS131: (xxxxxx), device type Foreign disk type 7, is online, file-oriented
device, shareable.

Error count 0 Operations completed 0
Owner process "" Owner UIC [SY,SYSTEM]
Owner process ID 00000000 Dev Prot S:RWPL,O:RWPL,G:R,W
Reference count 2 Default buffer size 512
Total blocks 8388608 Sectors per track 0
Total cylinders 0 Tracks per cylinder 0
Allocation class 1
Device access control list:
(ALARM=SECURITY,ACCESS=READ+WRITE+SUCCESS+FAILURE)
(AUDIT=SECURITY,ACCESS=READ+WRITE+SUCCESS+FAILURE)

We can’t find any code that might be creating these devices.

We’ve tried putting security ACLs on the template device and on the DNFSMOUNT.EXE but unfortunately we don’t see any security audit events related to these creations (we do see the audit events if we manually create a device). Any ideas of something else to try to audit?

This smells a little bit like a possible automount situation, but as far as I can tell, there are no automounts set up. I'll admit that I’m not sure I fully understand automount enough to know - for example, I'm not sure how to have TCPIP display any paths which are set up as AUTOMOUNT but not currently mounted? Is there a specific image (if it is not part of the TCPIP kernel) that does the AUTOMOUNT, are there any notifications related to AUTOMOUNTs?

Another thing that really surprised me was that there does not appear to be any debug or logging available from NFS the way there is from other TCPIP applications. If anyone knows of any undocumented logicals to turn on some NFS logging, maybe that would help.

We've been poking at this for a while now, we've looked at just about everything we can within the TCPIP commands. I'd dump it all here, but I don't think that would be productive. If there is something specific you'd like to see, I'll be happy to provide it.

Any other thoughts on other things we might be able to do to track this down, or other information to help us understand DNFS devices, automount, or NFS client in particular would be appreciated.

Oh - lastly, I am well aware that this is an old version of TCPIP and quite frankly this vintage of TCPIP and it's applications were about as stable as plutonium. We're "OK" with writing this off as "old version" and not worrying about it much more until after we upgrade.

Thanks in advance,

Brad McCusker
Software Concepts International

[John Gillings]

Brad,

Probably stating the obvious...

First thing I'd do is get a higher frequency monitor. Doesn't need to do everything your existing monitor does, just track the lifetimes of the devices of interest. Maybe look every 10 seconds, then track the any new devices every second? Any patterns?

If it's a network drive, do you know what node it's served from? Maybe try to correlate some finer grained time stamps with network activity?

[Brad McCusker]

Hi John,

Your suggestions are very valid, and I consider them to be Phase "next". They'll take a bit more effort and quite frankly before we went to that next phase I was hoping someone would point out something that we were overlooking to answer the question.

Brad