File Backup to Cloud Storage

[Dan Herron]

OpenVMs 7.3-1

I am looking into storing file backups at one of the many "cloud storage" facilities that are available instead of to magnetic tape.
There seem to be a million such offerings: ADRIVE.COM, LIVEDRIVE.COM, etc, etc.

Question 1: What are the OpenVMS options for transferring the backup files to the cloud storage in an automated fashion. The only thing I can come up with is FTP.

Question 2: Anyone have experience dealing with these cloud storage vendors in the same context as I am considering? Recommendations?

Thanks for your thoughts,
Dan Herron

[Steven Schweda]

I know nothing, but ...

> [...] backups [...] instead of to magnetic
> tape. [...]

BACKUP doesn't know about many storage
options, so, if that's what you're using,
then I'd guess that you'd need to put your
save set onto an intermediate local disk.

> [...] The only thing I can come up with is
> FTP.

Sounds likely. Support for which doesn't
seem to be included in the typical free plan.
As you wouldn't be using the vendor's backup
application, I'd expect something like FTP to
be the only possible access method. (Unless
you can figure out how to do something with
CIFS.)

> [...] Recommendations?

Hope that you never need support, unless you
can identify the correct answer to the
unavoidable question: "VMS? Is that a PC or
a Mac?"

[Art Wiens]

It's not necessarily up to you to figure out how to get your files there, it should be up to the vendor to descripe what file access methods are available to the potential customer. For example, your first suggestion - adrive.com:



Compatibility with ADrive
Q: Which operating systems are compatible with ADrive?
We currently support Windows, Mac, and UNIX operating systems.

(that doesn't bode well)

Q: Which Internet browsers are compatible with ADrive and what settings do I need to use to make them work properly?

(Uh oh ...)

ADrive supports the use of Internet Explorer and Mozilla Firefox. To experience the full features of ADrive, please configure your browser settings to the following:
1. Cookies enabled
2. Internet Explorer security set to medium-high or lower
3. JavaScript turned on
4. It is recommended that Internet Explorer's information/download bar be disabled and ActiveX be enabled. We are working to support Google Chrome, but do not support full compatibility with Google Chrome at this time.

So theoretically you could run a Decwindows interface and using a somewhat compatible browser, drag and drop your files on your "adrive". Would it actually "work"? Is it practical? Most likely not if you have anything over a (several?) hundred MB.

I would suggest calling/contacting these companies to see what else they might offer ... maybe "the cloud" has a legacy FTP solution for those of us stuck in the 80's/90's. Self submitting DCL procedure on a batch queue and Bob's yer' uncle ;-)

Cheers,
Art

[Bob Blunt]

Dan, there are as many options for transferring your files (backup savesets or ZIP files of them) as there are networking protocols that your environment can share with the cloud vendor in question.

There are, however, other questions that you should really answer before you sign a contract or use their services. Primarily why is this appealing, what's driving you to research this option? Secondly most important is what security and retention promises can they offer. It would be a shame and a mess to find out that their service "did something" to make your files unreadable or unusable just when you have to recover a mess of disks.

Two decades ago I was helping setup a production facility and one of the shortcomings was the cheaper solution tapedrives that we sold the customer. Even with two TA79 master unit drives churning out 2400' 9-track tapes it took *all* night to save the site's data to a big stack of tapes (and this was 20yr ago with "big" 1gb and 2gb disks...pre-HSJ days even). It was easier to build a set of disks large enough to hold entire savesets and then use file compression utilities once the savesets were xferred to "permanent storage. This usually saved about 50% or slightly more.

Then one of the program managers asked "have you actually tried to recover from that process?" Well, no, and we quickly ran into problems, first with finding disks big enough to restore and then decompress the savesets and then we discovered some interesting ways to clobber the CI when trying to restore. Trade-off? Ease and speed versus flexibililty and recovery.

Consider the amount of data you're transferring, the amount of time that it would take to get from your system to their "cloud," the return trip, the restoration using UNZIP (or pick your compression tool and it's restoration partner), the potential for file scrozzling during the transfer (either way) and then you've just gotten the file "home" and ready for restore and you haven't even started recovering the data yet.

Also consider a massive failure. If your OpenVMS systems are hard down then you can't get the data onto those systems to recover the data at all. You've still got to restore a bootable system (and IF you saved your system disk to "the cloud" then you're really in a hurt) and reload mucho software before you can pull the saved data onto a VMS system to restore it. While you MIGHT have something local that could hold your compressed savesets you still have to have a usable OpenVMS system for the restoration so did you save much?

Seems like a tough gamble unless you have a big safety net and time isn't a big factor while Humpty-Dumpty gets recovered and readied for data...so to speak.

bob

[Hoff]

1: poor to bad, as compared with other platforms.
2: yes; but with no recommendations.

Cloud Storage works pretty well for some of the stuff I deal with. The ubiquity of access that can be available here is as big a change as was clustering an eon ago.

Amazon S3 is one of the larger providers, and various other common services are built upon that. There are other providers around. tarsnap (where there are clients) is interesting, and there are open-source clouds that deal with the replication and encryption needs of many sites.

VMS lacks a FUSE API, which makes this quest a rather larger project. AFAIK, there are no clients. You end up shoving data out the pipe via ftp or sftp or via a DB-level mirroring or replication interface. (AFAIK, HP has no heterogeneous-vendor data mirroring products. Some databases do have this.)

While I'm not aware of a cloud disk driver or tape driver for VMS for block-level or tape-like access, I do know how to build one. (But before that quest starts, look at the volume of data, the rate of data change, and at how well your data compresses, and compare that with, say, half of the rated bandwidth of your network pipe. How long will that take? The bandwidth of the courier truck filled with the tapes or disks being pulled from your offsite archives is monstrous, though with poor latency.)

Yes, bare-iron recovery adds some complexity, but not really all that much more than any other bare-iron restart from off-site archives would encounter. And an offsite archive or a courier can lose tapes, just as a cloud storage provider can have blocks go missing.

The next usual consideration is whether you want to move your VMS boxes out of your data center; to boot your applications under emulation, and outsource your whole hardware investment. (I don't know off-hand of any providers with Itanium hardware available, though various do offer platforms which would support VAX or Alpha via emulation.)

There's also not much difference here with a remote server that you control; a server hotel. ProLiant (via emulation) or a few Itanium boxes or an old Alpha, if you really get get off of V7.3-1) would get you your own cloud.

You might also want to consider a "mutual aid" agreement with another vendor you work with; swap archives and quite possibly even swap servers. I've been installing six-terabyte arrays; storage itself is cheap. Form your own (private) cloud.

Even with the comparatively dinky disks found on most VMS boxes, if you're doing full disk uploads or a cold restart download, you need a *big* network pipe, or (given you need the biggest wad of bandwidth during recovery) a way to sneakernet your storage over from the closest *big* pipe. Most sites using off-site storage buy their bandwidth by the "courier truck", given the cost of network bandwidth. But if you have an empty OC48 SONET link handy, have at.

[John Gillings]

Dan,

I have no experience at all dealing with cloud vendors. However, if you decide to go with this option, the issue of getting the files out there is not that hard.

Unless there's an obvious direct solution, I'd suggest going with the easiest option that the cloud vendor supports. For example, a PC with a few $100 2TB drives and a GB ethernet connection. Use whatever mechanism works best to go from OpenVMS to the PC (FTP, SMB...) Transfer from there to the cloud. That also gives you a spinning hot copy of your backups. No reason this couldn't be automated.

You should probably compress and encrypt whatever you send.

As always, design your backup strategy from the RESTORE backwards to the SAVE, and make sure you test it!

[Terry.giblin]

"VMS? Is that a PC or a Mac?"

That is easy.

Has anyone seen the "HAL 9000 'Apple' ad"

Remind me, which one had the 'first calendar' application?

Cloud storage over ftp, that's a first.

[Dan Herron]

Thanks for all your input!