Openvms FTP Server Log File

[Imker]

Is there a way to controll, limit or configure
the log file size of the ftp server in Openvms?

[Ian Miller.]

probably.

What version of OpenVMS and which version of what TCPIP Product are you using?

[Hoff]

>Is there a way to controll, limit or configure
the log file size of the ftp server in Openvms?

Yes.

[Imker]

thanks.

OpenVMS V8.3-1H1

$ ! Product: HP TCP/IP Services for OpenVMS
$ ! Version: V5.6-ECO3

[Imker]

Yes is part of the answer I hoped for. Please provide some hints on how to achieve such a configuration.

Thanks.

[Steven Schweda]

What would you like to happen when the log
file gets too big? Stop logging? (Then why
have a log file?)

It should be pretty easy to rename+compress
the log file periodically (or periodically
check it to see if it's too big, and then
rename+compress it). Sufficiently old
(partial) log files can also be deleted
(where the exact meaning of "sufficiently
old" is your decision).

Do you get a new log file version if you
disable+enable the FTP service?

Is there some actual problem which you are
trying to solve?

[Imker]

there should be a roll over configuration or something like that.

The problem now is that the log file (one single big file) grows large and larger (beyond gigabytes of data).

it should not stop logging, but at least it should splitt the files... on a date or size rollover. the way it is now, the large file is to big to be of any use.

[Steven Schweda · Accepted solution]

> Do you get a new log file version if you
> disable+enable the FTP service?

Ok. If you won't try it, then I suppose that
I'll need to.

alp $ dg ftplog

Directory SYS$SYSDEVICE:[TCPIP$FTP]

TCPIP$FTP_RUN.LOG;2572
115 6-APR-2011 00:13:41.70 (RWED,RWED,RE,)
TCPIP$FTP_RUN.LOG;2571
1 6-APR-2011 00:09:43.25 (RWED,RWED,RE,)
[...]


ALP $ tcpip disable service ftp
ALP $ tcpip enable service ftp


alp $ dg ftplog

Directory SYS$SYSDEVICE:[TCPIP$FTP]

TCPIP$FTP_RUN.LOG;2573
0 20-APR-2011 10:21:50.09 (RWED,RWED,RE,)
TCPIP$FTP_RUN.LOG;2572
115 6-APR-2011 00:13:41.70 (RWED,RWED,RE,)
TCPIP$FTP_RUN.LOG;2571
1 6-APR-2011 00:09:43.25 (RWED,RWED,RE,)
[...]


So, yes, apparently you get a new log file
version if you disable+enable the FTP
service. This does disable the FTP service
(very) briefly. (But it is faster than a
reboot.)

> [...] (beyond gigabytes of data).

Your FTP server must be much busier than
mine. (Or else you're logging is much more
detailed than mine.)

[Hoff]

You'd think that there would be an automatic roll-over or automatic housekeeping or related, yes. But there is not. This is VMS, and the system manager gets to implement this sort of on-going maintenance stuff specifically for each local site. For better or worse.

Here are some general introductions to the topic of housekeeping on a VMS server...

http://h71000.www7.hp.com/wizard/wiz_5964.html

http://h30499.www3.hp.com/t5/System-Management/House-keeping-for-file-version-in-Openvms/m-p/4467712#M24960

http://h30499.www3.hp.com/t5/System-Management/Disable-FTP-Log-and-anonymous-log/m-p/4199045#M21095

What generally happens here is a periodic purge as part of the usual housekeeping job most everybody runs (usually a batch job) that might zip files or purge older versions of the logs or whatever is locally preferred.

Here are some of the typical considerations and targets; FTP is among these, but accounting and auditing and operator logs are, too.

http://www3.sympatico.ca/n.rieck/docs/openvms_notes_system_manager.html

It's also possible to set a directory version limit on the logs and related, which then gets rid of the older files automatically.

Here's the general template of the batch job, and some of the usual targets for purges and related:

http://labs.hoffmanlabs.com/node/97
http://labs.hoffmanlabs.com/node/939

Some folks will just brute-force the ftp-related problem and shut off all FTP logging. There are a couple of ways to do that, not the least of which is creating a runt ;32767 version as the current log version. That'll block new logs.

Assuming that was you, please post up the answer over on Superuser, too. (There's not a big VMS population over on SU, FWIW.)

[Jess Goodman]

If you are willing to change FTP servers, I am running an unreleased version of the HGftp software that has this capability.

Our site's VMS FTP servers handle over 125,000 sessions per day. That's more than one ftp login every second, and on average about 7 files get transferred per second.

The logs for these servers record all of this activity, and that takes up about 160MB per day. We are a 24x7 shop, but I can create new versions of the log files every day without dropping one user session.

That's very useful for running our daily reports. If you are interested, one of us can check with Hunter Goatley, but I doubt that he would have any problem releasing it to you.

Jess

[Steven Schweda]

> [...] without dropping one user session.
> [...]

I'd need to check, but I believe that a
disable+enable cycle on the TCPIP FTP server
doesn't kill an open session. (As I recall,
that procedure was ineffective at stopping
one of those "administrato" attacks, and the
individual FTP session process had to be
whacked independently.)

[Imker]

Thanks all.

The informations provided were highly helpfull.


I'll post a link to this over on SU.

[Jess Goodman]

I did some tests, and apparently DISABLE SERVICE FTP does nothing at all, other than allow you to then enter an ENABLE SERVICE FTP command. You are correct in that this sequence of commands will not disconnect users.

The ENABLE SERVICE FTP command does open up a new TCPIP$FTP_RUN.LOG file, because that command attempts to run a new ftp listener process, and this log file is that process's SYS$OUTPUT.

The problem is that if you did not run TCPIP$FTP_SHUTDOWN.COM, or use STOP/ID-, then the old ftp listener process is still running, and the "new" log file is quickly closed, ending with this:

%SYSTEM-F-DUPLNAM, duplicate name
%TCPIP-E-FTP_BIND, cannot bind the address to the socket
$ !
$ ! completed ftp server execution
$ !
$ exit:
$ exit $status .or. %x10000000
SYSTEM job terminated at 21-APR-2011 06:45:03.99

The old process will still be logging new connections, user names, and disconnections to the old version of TCPIP$FTP_RUN.LOG, assuming that you specified P1 with TCPIP$FTP_STARTUP.COM.

If you first stop the current ftp listener process before you start up a new one, then the new version of the log file will be active - but all current ftp users get disconnected.

The HG FTP software writes to an activity log file that is separate from the SYS$OUTPUT of the listener process. By default the listener logs connections and disconnections in it, but you can configure the software so that all directory listings, file transfers, CWD commands, etc. are also logged to this file.

To be clear, this thread is a discussion about a system-wide ftp log file, not the user-specific log files found is the ftp user's home directory.

[Steven Schweda]

> The problem is [...]

Hmmm. Now that you stir the old memories,
this sounds familiar. Everything's
complicated. (Especially with lame
software.)

[Hoff]

>I did some tests, and apparently DISABLE SERVICE FTP does nothing at all, other than allow you to then enter an ENABLE SERVICE FTP command. You are correct in that this sequence of commands will not disconnect users.

True.

The ftp server passes off incoming ftp connections to connection-specific server processes, and which is one of the reasons why the connection server processes can be configured with timeouts via logical name.

AFAIK, there's also no central controlled shutdown mechanism for those ftp server processes. They just get nuked when some lower-level piece shuts down, or they time out and exit.

The timeout gets those server processes to vaporize after some site-determined time interval, and to then allow those logs to be accessed for deletion or otherwise managed. But that's not a synchronous shutdown, it's a case where you shut off the ftp server, and then wait for those individual servers to timeout.

I'd certainly like to see centralized and particularly distributed logging for VMS, but that's not likely to be implemented given the corresponding need to modify all the logging clients around to use that logging. The existing logging mess is just too baked into the VMS environment.