Looking for some help-VMS Severs running Openvms V8.3-1H1

[The Gunners]

Hi There, Im looking for a bit of VMS help. I have two non clustered standalone VMS Severs running Openvms V8.3-1H1. I want to create a simple job that looks at the disks on one and backs up the differences mabey every hour (like an incremental) to a saveset - it then copies this saveset to the second VMS box and restores it. (Sounds easy and I know the basic commands to do all this except my DCL skills are non existent) Was wondering if anyone has a similar type script mabey or could advise ?

[abrsvc]

If you can "cluster" the systems using the current standalone system disks, you can "serve" the drives and use host based shadowing to keep the drives equivalent. The main problem with your sepcific request is that many files may be "in use" which can cause some synchronization problems or access problems. While the backup utility has the ability to "ignore interlocks" and copy the files anyway, there are potential issues that can arise using this method.

Perhaps if you could describe the reasoning behind the request, we can provide options to accomplish your goals.

Dan

[The Gunners]

Hi Dan,
Thanks indeed for the reply. Well basically these are test systems. The users are developers and just want to try sync any executeables , commands , basic user files but not huge amounts of data or databases. Its just a failsafe really in case the first system goes down

[Ian Miller.]

How much data are your users prepared to lose?

[The Gunners]

Hi Ian , as I said its a test area so they basically asked would it be possible to run some kind of job that checeks every hour or so for exe's and specific files that they update quiet regularily and just copy them over. Its a best endeavours approach is the way Im working with them on it , in my mind it doesnt sound like a major task - perhaps Im not appreciating the complexities

[RBrown_1 · Accepted solution]

More detail is required. Consider this example. Start with this on the local system:

$ BACKUP/SINCE=BACKUP/RECORD/VERIFY REMOTE::XFR.BCK/SAVE

On the other system:

$ BACKUP XFR.BCK/SAVE

But this may not do what you want if you already have a regular backup job that includes /RECORD. In that case you would probably want to save the backup time in a file or logical name, and use that saved time in your /SINCE.

Other considerations:

Files that are open at the time that the BACKUP/SINCE runs will *not* be backed up. We count on them being available to back up next cycle. Are there files that you will want to back up that will happen to be open *every time* you attempt to back up?

Will the backup always go in the same direction? If not, then there needs to be additional cleverness.

Doing it automatically? I would put these commands in a couple of self-resubmitting batch jobs.

As an alternative to BACKUP, consider the attached command file, which runs on the receiving system. It maintains a "last backup date" file in each directory, but it looks like you have to manually create version 1 for that file.

[RBrown_1]

"Consider that attached command file".

[John McL]

You say ...

"Thanks indeed for the reply. Well basically these are test systems. The users are developers and [we ?] just want to try sync any executeables, commands, basic user files but not huge amounts of data or databases. Its just a failsafe really in case the first system goes down."

Just a failsafe ?

Is there any special reason why this test system going down should present a major problem? Your .EXE's, .COM's etc should still be okay on the disk.

Is it that a reboot takes so long that you want to move the users to the other system while the first system is rebooting?

If that's then case then clustering the two systems might give you what you need without any requirement to run these Backups and restores, moreover there's no issue with disk and directory compatibility (e.g. size). Sure a crash and reboot will cause a minor disruption to the cluster, but it should be only a couple of minutes at most.

Controlling which node users can access should be child's play.

[Shriniketan Bhagwat]

Hi,

BACKUP can be used to create the saveset over network. It works over DECnet. Please refer section 11.5.3 from HP OpenVMS System Manager's Manual, Volume 1: Essentials. Below is the link.
ftp://ftp.hp.com/pub/openvms/doc/AA-PV5MJ-TK.PDF

You can use the below steps to create the files on other node as per your requirement.

1. Create the image backup of disk on node1 across network on to node2 with /RECORD qualifier.
2. Restore the image backup on node2.
3. On regular bases create incremental backup of disk on node1 across network on to node2 with /SINCE=BACKUP and /RECORD qualifier.
4. Restore the incremental backup on node2.

Use /IGNORE=INTERLOCK qualifier.

Regards,
Ketan

[The Gunners]

Guys thank you very much for the replies so far. I noticed that the cluster option has been mentioned several times. This might be worth exploring. Can I say that these two servers are in different locations physically (about 10 miles apart) so we would be 'clustering' over a network. Would this be possible and what would the steps be ? I hadnt considederd clustering at all due to this. The users are thinking of the second node as a sort of DR box that they would have their latest exe's if the first building went splat or flooded etc

[Shriniketan Bhagwat]

Hi,

You can use the cluster over IP feature for your DRP setup. This is a new feature in V8.4. Please refer section 5.1 from the v8.4 new features documentation.

http://h71000.www7.hp.com/doc/84final/6679/6679pro_002.html

Also refer section 3.3 from VMS clusters.

http://h71000.www7.hp.com/doc/84final/4477/4477pro_004.html

Regards,
Ketan

[Shriniketan Bhagwat]

Hi,

Refer below links for more information on Cluster over IP.

http://www.connect-community.org/resource/resmgr/library_whitepapers/cluster_over_ip_whitepaper.pdf

http://www.connect-community.de/Events/OpenVMS2009/folien/07-OpenVMS_clusteroverIP.pdf

Regards,
Ketan

[The Gunners]

Hi Shriniketan,
Thanks for the info there , we are still on V8.3 at the mo unfortunately , Im not sure they will agree to an upgrade just yet. I'll have to put that on the 'things to do' list that we can look at down the line , I guess its back to the backup or copy option again so :(

[tsgdavid]

Clustering with Host based shadowing would be nice. However if you can't do that, I am wondering if a simple BACKUP will do what you want. It will be, as long as you restore the files on node 2 after every BACKUP.

If you don't do the restore, you would end up with a whole bunch of savesets on node 2 and then in the event someone wanted to use this system, the files would have to be restored at that time.

I would suggest that you might want to consider using a COPY/MODIFIED/SINCE= command directly to the directories where the files should reside on the other node. It might even be better if you incorporate a COPY command in your software build procedures rather than checking every hour (if that is practical). This should at least be sufficient for your .EXE, .OBJ, .OLB (binary) files that have to be generated using a build procedure.

What ever you do, you will want to make sure that nothing gets missed (e.g. if node 2 is down or if any changes get made on node 2 that do not exist on node 1). You may want to have some way to verify that the two systems are identical. The DIFFERENCES command could be used to verify this.

Dave Williams

[Hoff]

In addition to what Ketan has indicated, it has been possible to cluster over IP for a decade or two (and supported), using what is known as LAN bridging. Bridging a LAN requires no software upgrades and no related processing, but does require bridging hardware in your network. Cisco managed switches have been commonly used for this for many years; you'll see this technique used both for SCS and for LAT, too. (The V8.4 stuff saves you on some LAN hardware, but requires you to mess around with some of the low-level setup and with IP. Bridging allows connections via classic SCS, so it works as it always has.)

One of the other options (which is easier than clustering, requires no VMS V8.4 upgrade and requires no clustering) is DECnet over IP via DECnet-Plus (also referred to as DECnet Phase V, DECnet/OSI), and that allows you to use all the usual old VMS tools directly. This is software-only and you likely have licenses for it, and this allows you to use DECnet DCL task-to-task and related among your hosts. Even via the open internet, if you have enough bandwidth for that.

Here's a simple DECnet tool for BACKUP:

http://labs.hoffmanlabs.com/node/598

However... Clustering via the V8.4 support or clustering via bridging is probably overkill here. Probably also DECnet-Plus here, too.

While you won't find modern IP networking tools on VMS, any experienced VMS DCL programmer can write some baseline stuff easily. For this case, that's usually based on a zip "-V" (to compress the files and the "-V" to protect the RMS metadata attributes) and then COPY /FTP /BINARY over to the remote host. Zip works as well as BACKUP for this stuff, and you can access the zip contents on arbitrary platforms. And since you're just doing backups, a simple approach is to ftp (or sftp) the zip archives over to HP ProLiant or HP-UX or VMS or other analogous servers; just dump the copies on some big disk storage somewhere else.

http://labs.hoffmanlabs.com/node/684

Alternatively, use BACKUP and a DCL procedure such as the following:

http://www.decuslib.com/decus/vmslt99a/net/bck_com.txt

and that procedure probably modified to zip "-V" (to compress and with "-V" to protect the saveset's metadata) the resulting BACKUP saveset and toss each saveset over to the file server via ftp or sftp.

And yes, I keep pointing to "-V" on the zip command. Read up on it. Hug it. Kiss it. Use "-V" on your command. Yes. And yes, with the quotes, unless you're running extended RMS parsing. "-V" is your friend.

http://labs.hoffmanlabs.com/node/575

As for the original question, I'd probably be looking at the predicament of the development staff. In all bluntness, a VMS programming staff that cannot effectively write baseline DCL backup procedures and similar utilities and tools probably needs some additional training and platform time, or the organization needs an escalation or custom support path for getting these utility procedures written, or needs to acquire or access to some staff that can provide these tools. Not having basic DCL experience available means your organization is inherently not using the VMS platforms to their full advantage. Your organization is potentially leaving performance optimizations, process optimizations and simplifications on the proverbial table. This is entirely your call and your management's call here, of course, but something to ponder.

[The Gunners]

Ok , thanks to everyone for all the replies so far (points to come :) , I have a bit more info now having chatted to one of the users. It seems out of all the disks , there are only really four that will be updated. I dont have the exact directories yet from them but basically all they want is a script that will run every hour or two on these directories , check what has been created or modified , and copy it over to the second node into the same named directories. A house keep job could then be setup on the send node to purge down versions etc. - Re one question that someone asked re the users skillset ( I think it is more political to be honest) they asked/want us to do it as it falls under backups as far as they are concerned

[Robert Gezelter]

Gunners,

Clustering would be a nice option, but it may well be overkill in this context (and I say this as a OpenVMS cluster advocate).

This is more a problem of source/image control. Personally, I would be inclined to treat this using a build type tool together possibly with proxy file access.

One cautionary note, depending upon precisely how this is implemented, it may be very important to keep both sets of system clocks fairly well synchronized (e.g., an external source of UTC).

BACKUP may even be overkill, this may well be implementable using vanilla COPY (although the archived BACKUP save sets would provide a useful audit trail).

- Bob Gezelter, http://www.rlgsc.com

[Hoff]

Are these users seeking the tools to manage and maintain source code and build products (images, libraries, etc) across multiple systems?

Are these users seeking to acquire or to implement some sort of a distributed version control system (DVCS)?

If so, then I'd probably skip all the DCL and ftp and the periodic file copies and related, and start looking at setting up and migrating to a Mercurial (Hg) distributed version control repository, and using that with VMS.

http://www.vmspython.org/DownloadAndInstallationMercurial

(Git is another very popular DVCS. And while there is a jgit tool available for accessing remote git repositories from VMS and other Java-capable platforms, what few discussions are posted around imply that an approach based on jgit on VMS might not be particularly robust, nor necessarily as well-documented as other options.)

There are other and older DVCS systems around, too. Subversion (SVN) can work, for instance. There are others.

Choosing to create and deploy your own source code control and versioning system can be somewhere between wasteful and impolitic.

[The Gunners]

Ok , I finally have the definitive requirement.
Basically we have two standalone systems in separate locations that wont be clustered (about 10 miles apart just for info and the network is quiet good).

lets call them SYSTEM A and SYSTEM B both running Openvms version 8.3-1H1.

System A is the primary Developement server and if theres an issue with this , then they will use SYSTEM B (at a best endeavours basis).

So, what they have asked for is mabey twice or 3 times daily a job that either does an incrementaltal backup or a copy/since the previous one (ONLY IN CERTAIN DIRECTORIES ON ONE OR TWO DISKS)
and then copies the contents over to SYSTEM B (which will have the same directory structure either by means of a restore from a save_set or either a direcct copy over decnet (if this is possible Im not sure)

Ive been thinking of a few ways of doing this including taking into a/c some of the advice you have given me, and Im thinking stuff like 'backup/record' and then a since qualifier in tandum with that command , however Im stuck as regards how to actually write this stuff up (My main role is Oracle DBA) so I dont have the dcl skills - could someone even advise on a simple script here , and I can then modify it to my own requiremnts. This wold be a great help and I may even learn something about dcl in the process.
Thanks as always guys n gals

[Joseph Huber_1]

The easiest for copying a single directory update using DECnet is using COPY on the target system (SYSTEMB::):

$ SET DEFAULT dev:[targetdir]
$ COPY SYSTEMA::remdev:[sourcedir]*.*/since="''f$time()'-4" [][/log

This assumes to have a DECnet PROXY defined for the user issuing the command, otherwise the remote system specification has to include the user credentials like
SYSTEMA"user password"::

To do incremental backup of several subdirectories, use backup on the source system to a remote saveset like:

$ backup/record [...]/since="''f$time'-4" SYSTEMB::last.sav/save

[Joseph Huber_1]

Ah sorry , for backup it is of course best to use /since=backup:

$ backup/record [...]/since=backup SYSTEMB::tmp.sav/save

In the backup case the result are backup save sets; You may want to have the savesets extracted to the target directory.
This can be done by another commnd-file, wich is submitted by the backup job:

$ submit/remote SYSTEMB::expand_backup.com

[Joseph Huber_1]

and finally, instead of SUBMIT/REMOTE, You can synchroneously execute the backup expansion job from inside the backup job by executing a DECnet task like:

$type/nopage SYSTEMB"user"::"0=expand_backup"

This methode requires to have the DECnet TASK object installed and enabled, or have expand_backup.com defined as a network object: ask Your system manager if this is possible/allowed.

[Joseph Huber_1]

and finally 2: since You said having no experience with DCL, and probably no system system manger to do it for YOU:
setting up DECnet access requires proper setup of FAL and TASK objects, and establishing proxies;
it might be easier to us TCPIP ssh,scp,sftp tools with public key authorization.

[The Gunners]

Joseph , all of your suggestions are a tremendous help. Thank you indeed. I will have a good think about this over the weekend and put something together for testing next week. I will update.

[Robert Gezelter]

The Gunners,

Caution is indeed recommended, it is easy to make a small error and incur spectacular consequences.

Some information about the size and makeup of files would be helpful.

In most cases, I would personally avoid doing a direct COPY over the network. Instead, I would use BACKUP to create a save set, then use ZIP to compress the same set, COPY/file transfer the resulting ZIP archive to the remote system, and then reverse the process.

A side effect of this process is the log (and contents) of what was propagated when, which is useful collateral when something goes awry, as it inevitably does.

This might not be the best beginners DCL/system management project, there are many ways things can go "off the rails". You may want experienced assistance/counsel [Disclosure: We provide services in this area, as do several other regular contributors to this forum.]

- Bob Gezelter, http://www.rlgsc.com