We tried to compile the Subversion client on OpenVMS but we failed because the Apache runtime library is not available on OpenVSM. Is'nt it? Or does anyone compiled it successfully? My customer wants to use Subversion because it's sources are so different types and the OpenVMS sources are about 25% of the overall projects that the major part is out of OpenVMS but an essential part of it.
The Apache portable runtime is an OS abstraction layer developed by the Apache web server project (covering things like memory allocation, threading etc.). It is only loosely related to Apache itself; and an alternative web server, not at all. Subversion is a CVS (or CMS) like source code management system.
I seem to recall a kit around for Apache 1.0 but that would be too low an APR version. Digging the current one out of the Apache sources and getting it to build sounds like a largeish project. Having a properly kitted APR library would be useful; in the meantime, I think the practical thing to do would be to do something like put the working copy on some flavour of Unix and NFS mount it on VMS.
The APR must be in there somewhere. Whether Subversion would exercise all the same bits as Apache I don't know.
I've often thought a Subversion client would be a nice thing to have on VMS. A first cut would probably have to assume stream files. Even nicer would be a way to store RMS attributes as metadata so you could deposit any type of VMS file. All of this probably requires real work.
Here is another user very interested in a Subversion client. It could be so much more reliable and efficient than what we now do with FTP.
On the other hand, an advantage of going via a PC is having Tortoise to check the Commit.
I recently ported a pair of CMS libraries (including history and classes) to Subversion, and considered trying the apache port, but eventually decided to write a client on VMS to push the stuff via a server on a PC into Subversion.
The RMS attributes should be easy enough to encode with Subversion attributes. These should be symbolic (no numeric codes) as in Set File /Attributes, and in a namespace such as "svn-vms:".
Who knows which version of the APR Subversion requires, and what has currently been ported?
======= The RMS attributes should be easy enough to encode with Subversion attributes. These should be symbolic (no numeric codes) as in Set File /Attributes, and in a namespace such as "svn-vms:".
Who knows which version of the APR Subversion requires, and what has currently been ported? ========
It would not just be a matter of encoding the attributes from the file header. For example, with a VFC file, you need another 2-byte "header" for each record in the file. Even if you successfully read in and store the control bytes when checking something in, you would need a way to recognize record boundaries when checking it back out. No doubt this type of problem has been solved many times before, such as in the STRUVMS support in the Process Software FTP utilities. But as I said before, it would be real work to implement this in Subversion, though well worth doing if someone has the time.
What's been ported would be whatever version of the APR is in Apache 2.0.
> It would not just be a matter of encoding > the attributes from the file header.
Yes, it would.
> For example, with a VFC file, you need > another 2-byte "header" for each record in > the file. [...]
No, those bytes are in the data in the file. They only need to be interpreted properly, which will be done if the file attributes are restored along with the data.
To see the "record boundaries" yourself, compare the output from "DUMP /RECORDS" with that from plain "DUMP" on a suitable file. The only requirement is that the raw file data be stored and retrieved, rather than letting anyone try to interpret the records.
> No doubt this type of problem has been > solved many times before, [...]
Well, maybe, Steven. But if it's sending you a set of differences rather than an entire file, I think it would be pretty tough to know when the VFC bytes are control bytes and when they are just data. Which is why I'm hoping you'll do the port so I won't have to :-).
It may be _storing_ a set of differences, but I'd expect it to be sending you a reconstructed file, with all the bytes reconstructed properly. Assuming that the critical file attributes do not change, or that the changes are also maintained, what could go wrong?
> I think it would be pretty tough to know > when the VFC bytes are control bytes and > when they are just data.
No, it's easy. The first two bytes are the first byte count. After that, just count the bytes to find the next byte count. And so on. (It might be hard if you didn't start at the beginning, so don't do that.)
> Which is why I'm hoping you'll do the port > so I won't have to :-).
It might be more educational for you. And I'm currently engaged in a tedious effort to push some VMS changes into the next cdrtools (cdrecord and friends) release.
>It may be _storing_ a set of differences, but >I'd expect it to be sending you a >reconstructed file, with all the bytes >reconstructed properly.
Subversion does not conform to that expectation. It sends differences in both directions over the network.
>> I think it would be pretty tough to know >> when the VFC bytes are control bytes and >> when they are just data.
>No, it's easy. The first two bytes are the >first byte count. After that, just count the >bytes to find the next byte count. And so >on. (It might be hard if you didn't start at >the beginning, so don't do that.)
I have not yet looked into the details of what Subversion sends across the network, but more than likely you get a byte offset and a chunk of data. If the byte offset puts you in the middle of a record in a record-oriented file with variable-length records, things could get interesting.
> Subversion does not conform to that > expectation. [...]
None of that matters. Any file on VMS comprises a stream of bytes, and meta-data which explain how to interpert those bytes.
Any system which can correctly reproduce the byte stream and the meta-data can do the job.
I don't know if Subversion can be persuaded to do the job without adding features, but the way that the VMS file system interprets the byte stream is of no consequence to the actual byte stream. Bytes are bytes. Look again at that DUMP output. One byte looks about the same as the next one. Records (and their byte counts) exist only in your interpretation on those bytes, not in the bytes themselves.
I admit I had not been thinking about VFC or for that matter indexed-sequential files. I was principally concerned with storing platform-independent source-code.
In general, how VMS files should be represented in depends on what you want to do with them, and especially on whether you want to use them on non-VMS platforms.
One needs to remember that a VMS file (as presented by RMS) is conceptually a stream of records, with optional redundant(?) indices. There are at least three views of a VMS file: programmatic, presentation and storage.
1) The view an application programme has of a file (as presented by RMS or some programming language) is a stream of records. These may be of fixed or variable length and contain formatting (VFC) information, which is not regarded as part of the application data (anything else?). There may also be the ability to retrieve records based on content, using indices maintained by RMS, which may be reconstructed from the record stream and index specifications. The record stream view is sufficient(?) to reconstruct the entire file, provided we include the file header information and index specifications.
2) The view a user has of a readable file presented by commands such as TYPE is of a stream of bytes. This is not in general sufficient to reconstruct the record stream, since any byte used as a delimiter could equally well occur in a record.
3) The view that the RMS routines have of a file is of the raw data as stored in the On-Disk Structure via the QIO interface to the ACP. This may be regarded as a random access array of bytes or bits or indeed as a stream of bytes.
To store a file in Subversion, it must be represented as a byte stream, which is trivial for the presentation and storage view. In both these cases, the only Subversion attributes required would be for the file-header information. For the record-stream view, however, one needs to add record lengths and formatting information to the byte stream. The most flexible format would be a variable-length record header. One would also need to add the index specifications to the Subversion attributes.
For many text files, especially programme source, the presentation view will be adequate, and the ambiguity about the record stream acceptable. Such files are also easily processed on other platforms.
To process a record stream on a non-VMS platform would need special purpose software. Although not terribly complicated, it would have to be supplied specially.
To process the raw view of a sequential or direct-access file is also manageable, but it is forbiddingly complicated for indexed-sequential files, even leaving aside possible versioning issues.
On the other hand, storing the record stream of an indexed file makes reconstructing it much costlier, as the indices need reconstructing.
I have tried the Java variant of subversion called SVNKit on OpenVMS which seems to be an easier way to go rather than trying to compile a native subversion client. I have JDK 1.4.2-4 installed on the VMS and downloaded the org.tmatesoft.svn_1.1.0.standalone.zip file found on http://svnkit.com/download/index.php The biggest challenge with the SVNKit is to set all logicals for Java so that UNIX-style filenames and similar things work correctly when running SVNKit on OpenVMS. I have succeeded to add files to a repository on Windows and also updated my working folder on VMS from Windows. The command line syntax for SVNKit is the same as when running the standard command line client (svn up, svn co http://xxxx, svn cleanup, etc).
If you manage to write it up as requested, could you perhaps also mention what view of the VMS file system it gives you.
I presume it will be suitable for source code, but clarity on this point would be useful. (See my reply of Oct 9, 2006 09:50:46 GMT for come of the issues).
I have during the week worked with SNVKit, a Java implemetation of the SVN client, on OpenVMS with some success and also sent patches to TMate software that hopefully will be included in the next version of this package. (http://svnkit.com)
I haven't tested everything but I have it working with remote servers, svn- and http-protocol, and also with a local repository. When I tried https it failed and I think it is related to Java/VMS stuff. I am running 8.2-1 on an Integrity with Java 5.0-1 and will soon try it on my Alpha.
I think this can be something to use and as soon as we have a stable environment we will move our code from CVS to Subversion. The server will be on Linux...
Using SVNKit: What logicals did you set? I'm having problems with filenames containing more than one dot (.). All dots except the last one are converted to underscores (_).
My current settings (worked for CVS): $ set proc/parse=extended $ define/job decc$efs_charset enable $ define/job decc$efs_case_preserve enable
just to throw this out there, a very small, bare-minimum, working example from Alpha VMS 8.3 (all patches/eco's, ods5 working device), using Java 1.5.0 against a linux SVN server.
Caveat: i haven't tried much more than 'svn list' so far.
I started getting SVNkit going under a GNV (2.1-2) bash shell, then worked my way back to getting SVNkit working from a simple VMS (dcl) login.
I haven't worked with Java very much on VMS, or elsewhere.
I was able to get this minimal example to fly, with almost no special JAVA$ logicals. in the real world, I'd expect that wouldn't be the case. especially wrt. adds/commits and/or various merges/diffs.
I can confirm that DECC$PIPE_BUFFER_SIZE = 32767 is needed. Before I defined this, an SVN diff would give me a stack dump in java$java_shr UNIXPROCESS_MD remapPath (I will not clutter the thread with the whole traceback, but can post it if anyone wants to investigate further).
But I have a more fundamental problem. Without privs turned on, jsvn creates read-only files when I do a checkout which then cannot be renamed, e.g.
$ jsvn co svn://ntts/dyserver svn: Error writing entries file for '/DKC200/DYMAX/BG/./dyserver' svn: Cannot rename file '/DKC200/DYMAX/BG/./dyserver/_svn/tmp/entries' to '/DKC200/DYMAX/BG/./dyserver/_svn/entries'
This would appear to be a difference in assumed Unix filesystem semantics from VMS. In Unix you ought to be able to set a file read-only and then rename it. It is the permissions on the *directory* that govern whether or not you can rename files. Not so on VMS. If you set a file read-only, it cannot be renamed. I don't know how to fix this. Obviously endowing all developers with system privileges so that they can rename their own read-only files is not a solution. Can someone please help?
I have further confirmed basic Subversion client functionality working with OpenVMS V8.3 Alpha, Java 1.5.0-3 and SvnKit 1.1.7 (released 11 Apr 2008, 19:07! lucky timing on my part, I guess).
I have tried: jsvn import jsvn checkout jsvn add jsvn commit jsvn update jsvn diff
Also: checkout twice and commit a conflicting change
jsvn update ! reports C & adds merge markers resolve the conflict jsvn resolved jsvn commit
I was (seemingly) able to work past the check-out read-only problem using the following DECC$ feature logicals. (below)
it was/is of interest to me, to find out what decc$, and java$ feature logicals folks have found to useful/helpful. (John Malmberg has a conference (PORTING_TO_VMS) on the encompasserve Notes server, that may be of interest)
Are you using SVNkit primarily from DCL or from the GNV bash shell? (just curious)
DECC$UMASK specifies the default value for the permission maskumask. By default, a parent C program sets theumask from the RMS default permissions for the process. A child process inherits the parent's value forumask.
To enter the value as an octal value, add the leading zero; otherwise,it is translated as a decimal value. For example:
$ DEFINE DECC$UMASK 026
DECC$FILE_PERMISSION_UNIX
With DECC$FILE_PERMISSION_UNIX enabled, the file permissions for newfiles and directories are set according to the file creation mode andumask. This includes mode 0777. When an earlier version of the file exists,the file permissions for the new file are inherited from the earlierversion. This mode sets DELETE permission for a new directory whenWRITE permission is enabled.
With DECC$FILE_PERMISSION_UNIX disabled, modes 0 and 0777 indicateusing RMS default protection or protection from the previous version ofthe file. Permissions for new directories also follow OpenVMS rules,including disabling DELETE permissions.
Even with a very permissive DECC$UMASK (000) and DECC$FILE_PERMISSION_UNIX enabled, I still get the same error. I guess regardless of my umask, jsvn wants to set the file permissions read-only for the entries file. Since write permission isn't specified, implied delete isn't relevant. Nor is inheriting permissions from an earlier version of the file relevant since in a checkout, all files are new.
So while I appreciate the tips, I still don't have a solution yet from the information you provided. I will have a look at the doc you referred to, though, and see if I can figure it out.
