topic Re: Add header to backup save set in Operating System - OpenVMS

Add header to backup save set

Dan Mellem — Thu, 26 Feb 2015 18:49:12 GMT

Hi,

We have a backup tape that was eaten by the drive, and someone "helped" by prying the tape out and winding it back by hand, damaging the lead-in and beginning of the backup set. We sent the tape in for recovery and they were able to pull the individual files off, but they're still in backup format (it has record markers and block padding, for example). I don't really care about the attributes and paths in the header but need to get this back into the disk (it's an RMS database with about 800 files).

Do you have any suggestions for repairing the backup file when the beginning is missing?

The recovery company can provide the individual file segments from the .BCK file, or whatever they can recover as a single file.

Thanks,

-Dan

Re: Add header to backup save set

abrsvc — Thu, 26 Feb 2015 22:18:09 GMT

A lot will depend upon how much data was lost. You indicate that the recovery company recovered a number of "files". If these files are savesets and complete, backup should be able to restore them.

A little more info is required here. Can you provide details about what specifically waas recovered?

Also: Backup has CRC blocks contained within the saveset so that block errors can be recovered.

What type of drive created the savesets and did you have compaction turned on? This may have an effect as well.

Dan

Re: Add header to backup save set

Dan Mellem — Fri, 27 Feb 2015 01:59:10 GMT

Thanks for your response.

The tape had a single backup file on it for an old year-end backup, created on a 8mm Cybernetics with compaction off. Essentially the recovery folks were able to split the remaining backup file into the respective files in the backup set, but the data is still in the same format as it is from a BACKUP. For example, if I back up an existing version of one of the files and compare the BCK file with what was sent by the recovery company, they are similar (NULL padding, BS [CHR 8] record terminators, etc.) when I compare them with a hex editor. If I compare the file from the recovered file, they are different (LF record terminators, less padding).

When I look at a backup of the newer file, it has about 737 bytes more at the beginning with information such as the original filename, the command used to execute the backup, original node, volume, and path, and more metadata. I thought about trying to stick the beginning of one backup onto the recovered file but I don't know enough about the format, plus this could take a long time.

Thanks,

-Dan

Re: Add header to backup save set

abrsvc — Fri, 27 Feb 2015 11:43:50 GMT

A backup saveset is a single file as was found on the tape. What I fail to understand is what the recovery company has delivered. Unless they utilized the backup utility to recover the files from within the saveset, there are no "individual files" that can be obtained from the tape. Either the saveset is available or it is not. The recovery company should have delivered a single file. That file may not have been received in the correct format however.

I will assume that the file now exists on a VMS system. If so, then please post the output from a DIR/FUL on that file. I suspect that the record format is not correct for a saveset and that a "convert" can resolve this.

Dan

Re: Add header to backup save set

Volker Halle — Fri, 27 Feb 2015 13:24:29 GMT

Dan,

you'll need support from someone, who is sufficiently familiar with the BACKUP internal data structures.

A Backup Saveset on tape is a series of backup saveset blocks of the same size. The size of the blocks is stored in the backup saveset block header. You can learn about some of this by using BACKUP/LIS/ANALYZE of some backup saveset.

I had added a function SEARCH/BCK to the DISKBLOCK utility (V6.2 or higher), which might be available on the OpenVMS Freeware CDs. It will search through a disk to find what looks like valid BACKUP saveset blocks. As those saveset blocks contain a checksum and a specific header, they can be easily identified.

Example:

AXPVMS $ mc diskblock

DISKBLOCK V6.2
Copyright (c) Hewlett Packard 2002. All rights Reserved.

This software can and WILL corrupt disks if it is mis-used,
use it with care and back everything up before you start.

Type HELP for instructions

DISKBLOCK> sel p.bck/file
DISKBLOCK> sea/bck
%DSKB-I-SEARCHING, Searching blocks 1 to 126 for BCK "*" at 14:16:19
LBN: 1 BCK-block: 1 Saveset: P.BCK
LBN: 64 BCK-block: 2 Saveset: P.BCK
%DSKB-I-SEARCHDONE, Search completed at 14:16:19

DISKBLOCK> read 1
%DSKB-I-BLKREAD, Block 1 (%X00000001) of USERDISK1:[HALLE]P.BCK;1 successfully read
DISKBLOCK> dump/bck

BACKUP Block header ( LBN: 1 )

Size in bytes of block header:                  256
Operating system ID:                                2048
Block Data Type (DATA=1/XOR=2)               1
Block sequence number:                       1
Block CRC:                                                   1477378562 (%X580F0202)
Block size in bytes:                     32256 (%X7E00 = 63 blocks)
Saveset name:                         P.BCK
File ID:                                         (    0,    0,    0)
Directory File ID:                     (    0,    0,    0)
Filename:
Allocation of current file:                  0
Checksum: (Valid)                       64400 (%XFB90)

The tape recovery people would NOT know about OpenVMS Backup Saveset structures, so the only thing they could have delivered as 'files' would be the individual backup saveset blocks from the tape.

If you can repeat a backup with the same BACKUP command as used in those old backups, you'll get a backup header to look at and eventually use to reconstruct the old backup saveset.

Volker.

Re: Add header to backup save set

Dan Mellem — Sat, 28 Feb 2015 02:27:52 GMT

Hi, Dan,

The recovery company's recovery utility is familiar enough with the BACKUP file structure to pull out individual files and name them appropriately, but it didn't convert it to a usable file. They then emailed a few samples for testing (they will provide the files on an NTFS disk). I was expecting a single file as well but the beginning would be cut off. Perhaps I should ask for a full file anyhow since the headers wouldn't be stripped out.

Hi, Volker,

Your patch may be exactly what I need, especially if I can get a single file from the tape. I've been looking for either the source of BACKUP or a description of the file format to see if I could Frankenstein two backups together. I haven't worked with Diskblock before - would there be a way to extract the file or do the equivalent of a loop mount to retrieve the files?

Thank you for your help.

-Dan

Re: Add header to backup save set

abrsvc — Sat, 28 Feb 2015 02:37:30 GMT

Your best bet is to have the singular file created on the tape as best as the recovery company can provide. Since the beginning of the file is the part that is missing, the majority of the saveset should be recoverable.

As Volker indicated, you may require the asstance of someone that knows the backup saveset format. Since the block contents have been modified over the years, it would also be helpful to know what version of VMS was in play when the backup was made. This is usually in the header, but that part is missing right?

Keep us posted on your progress

Dan

Re: Add header to backup save set

Dan Mellem — Sat, 28 Feb 2015 02:58:36 GMT

Yes, that part is lost, but I'm pretty sure we were running 5.5.2H4 at the time.

Re: Add header to backup save set

Volker Halle — Sat, 28 Feb 2015 09:16:50 GMT

Dan,

you can download the DISKBLOCK utility from:

http://www.digiater.nl/openvms/freeware/v80/diskblock/

That version should contain the modifications I've made regarding SEARCH/BCK and DUMP/BCK

The most important question to answer is: what does the 'recovery company' really deliver. If they would deliver the BACKUP saveset blocks from the tape, you could at least check/verify with DISKBLOCK, whether the data is correctly readable as BACKUP saveset blocks. Also you might be able to check, how many saveset blocks were lost from the beginning of the tape (saveset blocks are numbered and DISKBLOCK will show the numbers). Going this route may enable you to re-construct the full backup saveset. This would make sure, that you can extract the files with the correct attributes and contents.

If the 'recovery company' thinks it knows the backup internal structures, they should also know, that some type of OpenVMS files (e.g. index-sequential RMS files) can NOT be correctly delivered on a 'NTFS' disk.

Anyway you have a lot of work ahead of you...

Good luck.

Volker.

Re: Add header to backup save set

Hoff — Sat, 28 Feb 2015 15:34:00 GMT

@Dan Mellem wrote:

...I've been looking for either the source of BACKUP or a description of the file format to see if I could Frankenstein two backups together. I haven't worked with Diskblock before - would there be a way to extract the file or do the equivalent of a loop mount to retrieve the files?

The vmsbackup tool might be interesting to you; that's a portable open-source tool that's mostly compatible with OpenVMS BACKUP.

Most (all?) of the backup-related headers are included in a VMS distro. In SYS$EXAMPLES:, oddly enough.

The source listings to BACKUP itself are available on the VMS source listings CD or DVD media.

Re: Add header to backup save set

Brad McCusker — Sat, 28 Feb 2015 17:32:28 GMT

Dan - I am going to be direct here. You will need help to recover these files. Our firm can do it for you. I believe my e-mail address is in my profile, otherwise there is a contact us form on the web site.

Best Regards

Brad McCusker

Software Concepts International

www.sciinc.com

Re: Add header to backup save set

Dennis Handly — Sat, 28 Feb 2015 23:46:46 GMT

>I believe my e-mail address is in my profile

Nobody else can see that private info.

Re: Add header to backup save set

Dan Mellem — Mon, 02 Mar 2015 22:34:35 GMT

I'm going to ask them if they can provide as much of the backup file as possible to see if I can do anything with that. In the interim, they've provided a raw file from backup that looks closer to what's needed. However, the header is of course missing. I've tried to change some of the attributes to see if I can read it, but haven't had luck. I've tried:

$ set file/attributes=(org:rel,LRL=144,RAT:CR) ace97121.dbm

$ DUMP/HEADER/BLOCK=COUNT=0 ace97121.dbm

%DUMP-E-OPENOUT, error opening DBM_DISK:[DS.DATA]ACE97121.DBM;1 as output
-RMS-F-IFA, illegal file attributes detected (file header corrupted)
-RMS-F-BKS, bucket size too large (FAB)

A newer year file:

VAX1> DUMP/HEADER/BLOCK=COUNT=0 ace04121.dbm

Dump of file DBM_DISK:[DS.DATA]ACE04121.DBM;1 on 2-MAR-2015 14:29:46.79
File ID (18,1,2)   End of file block 100 / Allocated 100

                             File Header

Header area
    Identification area offset:           40
    Map area offset:                      100
    Access control area offset:           243
    Reserved area offset:                 255
    Extension segment number:             0
    Structure level and version:          2, 1
    File identification:                  (18,1,0)
    Extension file identification:        (0,0,0)
    VAX-11 RMS attributes
        Record type:                      Fixed
        File organization:                Relative
        Record attributes:                Implied carriage control
        Record size:                      144
        Highest block:                    100
        End of file block:                101
        End of file byte:                 0
        Bucket size:                      5
        Fixed control area size:          0
        Maximum record size:              144
        Default extension size:           5
        Global buffer count:              0
        Directory version limit:          0
    File characteristics:                 Contiguous best try
    Map area words in use:                2
    Access mode:                          0
    File owner UIC:                       [SYSTEM]
    File protection:                      S:RWED, O:RWED, G:RE, W:
    Back link file identification:        (11,1,3)
    Journal control flags:                <none specified>
    Active recovery units:                None
    Highest block written:                100

Identification area
    File name:                            ACE04121.DBM;1
    Revision number:                      73
    Creation date:                        29-SEP-2003 08:29:01.52
    Revision date:                        16-JUN-2004 15:41:19.54
    Expiration date:                      <none specified>
    Backup date:                          <none specified>

Map area
    Retrieval pointers
        Count:        100        LBN:        424

Access Control List
    (IDENTIFIER=DS$OPERATOR,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL)
    (IDENTIFIER=DS$USER,ACCESS=READ+WRITE)

Checksum:                                 13253

Is there an easy way to copy the header from the newer file over the old one to see if that helps?

Thanks,

-Dan

Re: Add header to backup save set

Volker Halle — Tue, 03 Mar 2015 09:35:37 GMT

Dan,

please closely look at the error message:

-RMS-F-BKS, bucket size too large (FAB)

and the dump of the file header of a current file:

Bucket size: 5

You could change the bucket size with $ SET FILE/ATTR=(BKS=5) ace97121.dbm

and try again...

Remember that I wrote, that certain OpenVMS files cannot be correctly delivered on a 'NTFS disk' ! But you may be able to repair them.

Volker.

Re: Add header to backup save set

Dan Mellem — Tue, 03 Mar 2015 19:07:47 GMT

Thanks. Setting the bucket size made it readable. This process seemed to work:

$! Copy file onto the VAX as binary, then:

$! Get data from an existing file

$ DUMP/HEADER/BLOCK=COUNT=0 ace04121.dbm

$! Set attributes to recovered file

$ set file/attributes=(org:rel,LRL=144,RAT:CR,BKS=5) ace97121.dbm

We then reindexed and reordered the data.

We're still going to ask for the BCK file since doing this file-by-file will take a long time, but this is a viable approach.

Thank you all for your help. Once I get the rest of the files we'll try this with the rest of the files and see if we have a complete database. Unfortanately they can only deliver the files on a Windows disk so we're stuck with this process unless we can get the partial BCK file working.

-Dan

Re: Add header to backup save set

abrsvc — Wed, 04 Mar 2015 11:17:50 GMT

If the saveset was actually created with version 5.5-2, I have a machine available wit that version. Backup on newer versions should be able to read and expand the older savesets without problems, but if needed, let me know.

Dan

Re: Add header to backup save set

Dan Mellem — Thu, 05 Mar 2015 23:23:58 GMT

Hi, Dan,

Thanks for the offer but we've been successful in restoring other year-end backups on our 6.2 VAX.

-Dan