topic Re: COPY/FTP Command and Date/Time Stamp in Operating System - OpenVMS

COPY/FTP Command and Date/Time Stamp

John T. Farmer — Wed, 11 Mar 2009 12:47:01 GMT

Hello all,

On OpenVMS 8.3, two Alpha servers on IP network (no DECNet), I have implemented the COPY/FTP command in several DCL procedures to move files between developement and production servers. One thing I noticed was the file date/time stamp of the file takes the current date/time on the destination system. Is there anyway to retain the original file attributes using FTP? DECNet is not allowed in our corporate network (IP only).

Also, just reading about COPY/RCP, but have odd results using that form. Odd results meaning the destination file takes on a date/time other than the original file date or the destination system date. Hmmmmm...

Thanks,

John

John dot Farmer at Genworth dot com

Re: COPY/FTP Command and Date/Time Stamp

Craig A — Wed, 11 Mar 2009 12:58:08 GMT

John

I think that is expected behaviour.

Why is it a problem for you? Are you trying to synchronise procedures across nodes?

If so, the CHECKSUM command might be useful.

Craig

Re: COPY/FTP Command and Date/Time Stamp

Hoff — Wed, 11 Mar 2009 13:02:44 GMT

I've found that looking at the dates on containers is surprisingly fragile and often problematic. There are cases within DECnet were the low bits of the timestamp are lost, for instance.

Common alternative choices here are MD5/SHA1 and verification of the file against a known checksum, or the use of git or SVN or Hg or such (and a known-good repository), or to zip the bits and protect the date that way.

BTW: DECnet runs just fine over IP, if you want to use that.

ftp is a massive security hole and a security hole that's also basically incompatible with firewalls, too. FWIW.

Re: COPY/FTP Command and Date/Time Stamp

John T. Farmer — Wed, 11 Mar 2009 13:03:11 GMT

We are copying source, obj and exe from development to production servers. Would like to keep file date/time in sync for easier visual verification when troubleshooting, verifying copy results, etc.

John

Re: COPY/FTP Command and Date/Time Stamp

Steven Schweda — Wed, 11 Mar 2009 13:20:39 GMT

FTP is an old protocol, and unless things
have changed in the last decade, it doesn't
know or care about file date-time info.

If you're _fetching_ files, you might try
using wget to do it. Wget tries to parse an
FTP server's LIST output to determine a
file's original date-time, and then set the
attributes of the fetched file accordingly.
(Different time zones may still cause some
trouble, however, and "ls -l"-format listings
from UNIX servers can be imprecise.)

http://antinode.info/dec/sw/wget.html

(Pushing a file is more demanding, as you'd
need to add capability to the FTP server,
which seems unlikely to happen.)

Or, use BACKUP or Zip (or even "tar") to
preserve those attributes before
transmission. Many things are possible.
(Even if most of them are sub-convenient.)

Re: COPY/FTP Command and Date/Time Stamp

Hein van den Heuvel — Wed, 11 Mar 2009 13:24:51 GMT

Zip'm up before sending, unzip at destination. Stick'm in a backup file first if you have any problems.
It is an extra step (or two) but it may well be faster in the end.

fwiw,
Hein.

Re: COPY/FTP Command and Date/Time Stamp

Wim Van den Wyngaert — Wed, 11 Mar 2009 13:26:00 GMT

Use NFS and a copy from the NFS disk to the destination directory (so a remote command on the destination node).
This preserves the mod dates (we sync files using this mechanism).

Or write some DCL and use touch (freeware) to set the date correctly.

Wim

Re: COPY/FTP Command and Date/Time Stamp

John T. Farmer — Wed, 11 Mar 2009 13:30:52 GMT

Ok, sounds like the general consensus is to package the files (backup, zip, etc), transmit and then package on the destination. Given that approach, I'd need something on the receiving end to automatically process the file (backup, unzip, etc). I've looked at JOB_DAEMON from the Process Software FILESERV site, as a way to monitor and act on a file. My thoughts there would be to package up one or more files, along with a DCL script to process the files once unpackaged... Any thoughts there, before we close out?

Thanks,

John

Re: COPY/FTP Command and Date/Time Stamp

Hoff — Wed, 11 Mar 2009 13:33:57 GMT

I'd look to content management. This is often git or svn or Hg or such. If this were Unix, I'd suggest rsync. Treat your software releases and your files as products.

Or you need a PCSI or other installation kit, and productize your files. It's entirely feasible to build these kits entirely from DCL, so you package and install your bits. For one or two servers, this might be overkill. But for larger pools of servers, this can be invaluable. This approach is highly reproducible.

Or you use a pool of files and save off the MD5 or SHA1 checksums, and a tool that verifies the filename against the checksum.

Worst case, you need some local and enhanced variation of the directory DIFF tool I posted on the Freeware a while back. That tool looked for file content skews between the contents of two directories. (Search HOFFMAN_EXAMPLES on Freeware V8 for *DIFF*, IIRC. I don't have a FW8 distro handy to check ATM, and don't remember the exact name of the file. And MVB's archives at SAIC are gone.)

File dates, BTW, can be forged with a SET FILE command. Which has two implications. One is a lack of trust or a lack of reliability. The other of which is to simply set the low bits of the file date - or the whole file date - to an encoded form of the release version.

Re: COPY/FTP Command and Date/Time Stamp

Craig A — Wed, 11 Mar 2009 13:39:27 GMT

John

Personally, I would use BACKUP (if only 'cos I am a VMS bigot through and through)

You will need to investigate whether it is appropriate to compile and link on your DEV box, just moving the .EXE's to your PRD box.

I'd rather compile/link on your PRD box but accept that might not always be possible.

Also bear in mind that it would be helpful if your "deployment" DCL was robust enough to be able to support a back-out where necessary.

Finally, bear in mind that you might have to wait for a window to apply such changes on PRD. So use of some sort of logical name or flag file might be useful (especially if you have lots of PRD servers, possibly spanning timezones, etc..)

HTH

Craig

Re: COPY/FTP Command and Date/Time Stamp

Robert Gezelter — Wed, 11 Mar 2009 13:49:17 GMT

John,

I will concur with the others. The FTP protocol has no provision for preserving time stamps (a platform independent feature).

Personally, if I want to ensure that meta data is not damaged, I use BACKUP to build a saveset, then the OpenVMS ZIP with the "-V" switch to preserve attributes. I then ftp the ZIP archive.

It is slightly cumbersome, but it avoids the accident of dealing with an accidental corruption in transit (e.g., someone forgetting to issue the BIN command within FTP).

- Bob Gezelter, http://www.rlgsc.com

Re: COPY/FTP Command and Date/Time Stamp

Richard Whalen — Wed, 11 Mar 2009 14:05:17 GMT

There are FTP extensions aimed at preserving time stamps, but I don't think that you'll find them widely implemented.

MDTM reports the modification time of a file
MFMT sets the modification time of a file
MFCT sets the creation time of a file.

http://tools.ietf.org/html/draft-somers-ftp-mfxx-04
http://www.ietf.org/rfc/rfc3659.txt?number=3659

Re: COPY/FTP Command and Date/Time Stamp

Steven Schweda — Wed, 11 Mar 2009 14:09:47 GMT

> [...] I use BACKUP to build a saveset, then
> the OpenVMS ZIP with the "-V" switch to
> attributes. [...]

As I try always to say, if anyone finds a
case where Zip (-V) alone fails to save any
useful file attributes, be sure to file a
complaint.

Re: COPY/FTP Command and Date/Time Stamp

Hoff — Wed, 11 Mar 2009 14:15:48 GMT

While zip "-V" and BACKUP do work just fine together and the former does successfully protect the latter against transfer errors, I've found use of zip "-V" (alone) preferable for various reasons.

Run some local tests of these two and see what you get for results.

And consider the complexity of zipping a saveset.

(This outside the discussion of portability. While there are tools to get access into BACKUP savesets on other platforms, zip and unzip are nearly ubiquitous. I have updated the classic vmsbackup tool for ODS-5 and some other relatively recent changes; that source pool available for download for folks that need access to savesets on other platforms.)

But here? This looks to be source code control. Mercurial. Subversion. Git. rsync. Whatever. It's quite unfortunate that there is no remote CMS client for use on OpenVMS available here, though you have to roll your own.