- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Change Data capture with RMS on OpenVMS
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 11:02 AM
тАО07-10-2008 11:02 AM
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 11:09 AM
тАО07-10-2008 11:09 AM
Re: Change Data capture with RMS on OpenVMS
There's no single central API.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 11:12 AM
тАО07-10-2008 11:12 AM
Re: Change Data capture with RMS on OpenVMS
My understanding is that there is not a way, but CONNX's website gave me hope. So data will be stored in the RMS system and we will want to take customer data as it changes and propogate it out to an Oracle System.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 11:35 AM
тАО07-10-2008 11:35 AM
SolutionBiggest issue here is usually with the volume of data generated; with the size of what can effectively be the journal file.
If you don't have the source code of the application (or there are prohibitions around changing same), you'll probably have to tap into the system service intercept mechanism, tap into the lock manager processing, or scrape the changes via RMS journaling or such. These approaches are technically feasible, probably (entirely?) unsupported, and rather more work and likely also far more expensive than tweaking the source code.
What I know of CONNX is its ODBC and JDBC pieces, and you can tap into that; I can certainly understand how to tap into that and implement change management as CONNX has control over the API involved.
I don't know of a way to use CONNX locally to access RMS that doesn't also involve re-writing your COBOL code to access the CONNX APIs. And once you do that...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 12:28 PM
тАО07-10-2008 12:28 PM
Re: Change Data capture with RMS on OpenVMS
There are various ways that this can be accomplished, but as Hoff noted the most general case together with the requirement to "not change code" can be a challenge.
Is it possible to be more specific as to the precise requirements? Also, sometimes it is possible to implement logging facilities by modifying relatively discrete sections of code.
As always, the devil is in the details.
- Bob Gezelter, http://www.rlgsc.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 01:37 PM
тАО07-10-2008 01:37 PM
Re: Change Data capture with RMS on OpenVMS
- On initial load, create lookaside table with entry for each record in rms file.
- On (scheduled) update the full rms file is scanned and for each record it is determined whether it is new, update, or deleted using that lookaside table.
- For the full table scan connx does NOT use RMS $GETs private code going much faster.
This is based on how I would solve the problem, as well as:
http://www.connx.com/products/change-data-capture/DataSync.asp
http://www.connx.com/ftpdir/white/Change_Data_Capture_White_Paper.pdf
"By storing the hash keys along with the primary key of each record in
a separate CONNX private data store, the CONNX DataSync tool performs
incremental updates from source to target at a fraction of the usual
full update time."
I have asked a contact at connx for comment on the above, pointing to this topic.
Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 01:44 PM
тАО07-10-2008 01:44 PM
Re: Change Data capture with RMS on OpenVMS
I'm the VP of Engineering at CONNX, so I can explain how our CDC for RMS works.
You are correct that RMS does not have a log file to read from. CONNX CDC works by completely scanning the RMS file(s) in question using blockIO during scheduled intervals. A hash of every record along with the primary key is stored in an intermediate datasource, and the hashs from the last scan are compared to the current hashs. This enables CONNX Datasync to quickly determine which records have been changed, inserted, or deleted since the last sync. Then the changed data is propogated to the target data source. This technique does not require any modification to the existing COBOL applications, nor any modification to the structure of the RMS files. It also does not require a timestamp.
Thanks
lm
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 01:55 PM
тАО07-10-2008 01:55 PM
Re: Change Data capture with RMS on OpenVMS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 02:00 PM
тАО07-10-2008 02:00 PM
Re: Change Data capture with RMS on OpenVMS
For sake of completeness...
Other vendors also offers solutions in this space. For example.
http://www.attunity.com/data/uploads/Data%20Sheets/Attunity%20Stream%20DS.pdf
"I'm not aware of log files they can scrape,"
Yes and No. RMS has an optional product (license included in highest OpenVMS licenses), called Journaling which offers AI (After Image), BI (Before Image) and RU (Run Unit = Transactional) journalling.
Its usage can be transparent to the application functionality, but it certainly has performance and disk usage implications.
The BI journalling would be the easiest path to CDC. However, the before images are of the entire bucket, and they are there for all buckets>. They are not per record, and not just for primary data buckets.
So there is 5 to 100 times more data in the BI stream than strictly needed for CDC. And the data itself is the raw possibly compressed RMS format. While it is no rocket science to decode that and cull data records from there, it woudl certainly be 'tedious' work. Notably unravelling a bucket split cause could be 'fun'.
An other transparent alternative could be trhough SSI (System Service Intercept)... a major undertaking.
Now given that it is a COBOL application, and assuming that any and all data modificaton to be captured is processed by Cobol images there may be a much easier options. Just create an alternative COBRTL
(google: gillings fake rtl) listen careful to what comes by for the dcob$open, dcob$writexxx, dcob$rewrite, dcob$deletexxx and dcob$close calls and package it all up neatly. SMOP!
I have built a Cobol RMS TRACE tool, instrumenting performance of each step, using a much similar technique. Quit useful and a big step into the direction of a CDC tool.
Hope this helps some more,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО07-10-2008 02:07 PM
тАО07-10-2008 02:07 PM
Re: Change Data capture with RMS on OpenVMS
Thanks
lm