topic Re: Missing '/foreign' labels on image backups in Operating System - OpenVMS

Missing '/foreign' labels on image backups

Ken Randell — Mon, 22 Nov 2010 18:02:31 GMT

VMS on IA64, 8.2-1, 8.3, 8.3-1H1

Upon rare occasion I will have a condition where a disk seems to lose its 'mounted/foreign' label. By this I mean, I can mount the disk (physical or logical unit as part of a raid set) with the usual mount/over=id commands, but a mount/foreign will return a 'null' label as if the disk were empty. Otherwise, the drive looks perfectly normal with no missing information. ANA/DISK doesn't show any problems.

If I take an image backup of disk in this condition and restore it to another drive, the 'null' label moves with it.

All drives were originally created with the usual INIT command and a non-image restore (i.e., I restore a directory tree to a disk).

I have never seen this on ALPHA; I am using ODS-5 drives in all cases. As mentioned, I have seen this on 18gb and 72gb drives as well as logical units carved up from a p400 raid controller.

The only fix I know of is to INIT the disk again, and do a non-image restore of the drive using various backup qualifiers such as /owner=original, etc. SET VOLUME/LABEL=xxxx doesn't fix this.

The only reason this matters is that there's a command procedure in play at boot time that wants to check the volume label to make sure the right drive is in the right place. I realize I could remove this check, but it still seems kind of bizarre that this is happening...

ENGZZD $ mount/over=id dkb0:
%MOUNT-I-MOUNTED, DSPS mounted on _$1$DKB0: (ENGZZD)
ENGZZD $ dism dkb0:
ENGZZD $ mount/for dkb0:
%MOUNT-I-MOUNTED, mounted on _$1$DKB0: (ENGZZD)
ENGZZD $ dism dkb0
ENGZZD $ mount/over=id dkb0:
%MOUNT-I-MOUNTED, DSPS mounted on _$1$DKB0: (ENGZZD)
ENGZZD $ set volume/label=dsps dkb0:
ENGZZD $ dism dkb0
ENGZZD $ mount/for dkb0:
%MOUNT-I-MOUNTED, mounted on _$1$DKB0: (ENGZZD)
ENGZZD $ mount/over=id dkb0:
%MOUNT-I-MOUNTED, DSPS mounted on _$1$DKB0: (ENGZZD)
ENGZZD $ ana/disk dkb0:
Analyze/Disk_Structure for _$1$DKB0: started on 18-NOV-2010 11:10:31.89

%ANALDISK-I-OPENQUOTA, error opening QUOTA.SYS
-SYSTEM-W-NOSUCHFILE, no such file
ENGZZD $ dump /block=count=1 dkb0:

Dump of device DKB0: on 18-NOV-2010 11:17:26.64

Logical block number 0 (00000000), 512 (0200) bytes

03039401 001E65C0 11C00200 15C600A0 ..F...@.@e...... 000000
8BDFFF76 905F0000 000501FB 000609F7 w...{....._.v._. 000010
20202020 20205350 53440087 80FDFF74 t.}...DSPS 000020
73206120 746F6E20 73692020 20202020 is not a s 000030
0000000A 0A0D6B73 6964206D 65747379 ystem disk...... 000040
00000000 00000000 00000000 00000000 ................ 000050
00000000 00000000 00000000 00000000 ................ 000060

etc.

Any ideas/hints/suggestions gratefully accepted. Thanks.

Re: Missing '/foreign' labels on image backups

John Gillings — Mon, 22 Nov 2010 21:00:36 GMT

Ken,

What does DUMP show when a disk is in the odd state? In particular, the field that normally contains the label.

Perhaps a reasonable workaround in your check would be to accept a null label, or look elsewhere on the disk.

For example, there are lots of alternate home blocks in the first few clusters of INDEXF.SYS - you should be able to find numerous copies of the volume label.

See DUMP/BLOCK=COUNT:10 dev:[000000]INDEXF.SYS

Surely they can't all have gone bad? Even if the device is mounted /FOREIGN, finding INDEXF.SYS isn't too difficult.

Re: Missing '/foreign' labels on image backups

John Gillings — Mon, 22 Nov 2010 21:02:53 GMT

Oh, almost forgot...

When/if you can get some documented information about the (mis)state of the disk, and the sequence of events leading up to it, please log a case with HP. They can't fix what they don't know about (there are only a few HP engineers who read ITRC - and responding to issues raised here is not in anyone's metrics).

Re: Missing '/foreign' labels on image backups

Robert Gezelter — Mon, 22 Nov 2010 21:14:14 GMT

Ken,

A DUMP/ASCII/HEX of the primary and spare copies of the home block would be helpful.

- Bob Gezelter, http://www.rlgsc.com

Re: Missing '/foreign' labels on image backups

John McL — Mon, 22 Nov 2010 21:52:42 GMT

I wouldn't expect a volume label when mounting a disk /FOREIGN.

From the DCL manual, under the MOUNT command...

"The volume-label parameter [2nd parameter] is not required when you mount a volume with the /FOREIGN or /NOLABEL qualifier or when you specify /OVERRIDE=IDENTIFICATION."

(Note that it doesn't say why it's not required.)

and later ...

"/FOREIGN
Indicates that the volume is not in the standard format used by the OpenVMS
operating system. Use the /FOREIGN qualifier when a magnetic tape volume is not in the standard ANSI format, or when a disk volume is not in Files-11 format. If you mount a volume with the /FOREIGN qualifier, the program you use to read the volume must be able to process the labels on the volume, if any."

(i.e. volume labels are not processed)

MOUNT/FOREIGN is used with BACKUP/IMAGE because you want the volume label to copied from the disk that you are backing up.

The fact that when you do a MOUNT/OVER=IDENT and the disk appears with the correct volume label looks to me as if it was saved on the disk but not accessed with the MOUNT/FOREIGN command.

Now I expect that someone will tell me that I'm wrong ...

Re: Missing '/foreign' labels on image backups

John Gillings — Mon, 22 Nov 2010 22:25:15 GMT

re: John McL

>Now I expect that someone will tell me
>that I'm wrong ...

What you say is correct - you are not required to specify a volume label when mounting a volume /FOREIGN, or with /OVERRIDE=ID.

However, when you mount such a volume, I would expect the system to display the volume label it finds - see the example %MOUNT-I-MOUNTED messages in Ken's post. Indeed, mounting /FOREIGN is one way to determine an unknown volume label (which is what I believe Ken is depending on)

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 00:12:54 GMT

Ken,

One thing I would try is to use

$ set volume/label=DSPSNEW $1$DKB0:
$ set volume/label=DSPS $1$DKB0:

It is possible that there is some "optimization" being done and the volume label isn't rewritten if it is the same as the present label. This seems to be the case on Alpha OpenVMS 8.3 with an ODS2 disk, it reads the home block but does not update it if the previous label was the same.

This still doesn't address the underlying problem, but it may be a work around that is less work than restoring a disk (if it works).

If you have LDDRIVER loaded, it would be interesting to see what LBNs are being accessed when the disk is mounted.. If you are going to try this, do it before attempting to change the volume label. This can be done by using the

$ set proc/priv=all
$ LD connect /replace $1$DKB0: LDA100:
$ LD trace LDA100:
$ mou/ov=id LDA100:
$ dism LDA100:
$ mou/for LDA100:
$ dism LDA100:
$ LD show/trace LDA100:

and see what blocks are being read.

Then to revert to normal access:

$ ! assumes already dismounted
$ LD notrace LDA100:
$ LD disconnect LDA100:
$ mou/ov=id $1$DKB0:

Jon

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 03:13:17 GMT

Ken,

I can reproduce on an Alpha with a disk initialized /GPT (and use of DISKBLOCK). Sorry, the following is a running commentary in chronological order, and I didn't discover the reproducer until near the end...

I don't have an IA64, but I am wondering if this behavior could be related to /GPT (which is the default on IA64, but not on other architectures). If you have a [000000]GPT.SYS file, your disks are initialized with the GPT overlays, and these are in the blocks normally occupied by the boot and primary home blocks.

With /GPT, there is no longer a primary homeblock. Perhaps the mount/foreign code doesn't use the same checks for the home block as the normal mount code does. I don't know what checks are made by the mount/foreign code for valid home blocks, but it may be finding what it considers to be a home block that really isn't one, and then the volume label wouldn't be correct. For example, if the mount/foreign code only check for valid checksums, a block of 0x00 bytes would pass the test. Perhaps the mount/foreign code is not using the same search sequence for home blocks as the normal mount code, or perhaps its checks are not as strict.

Here is a good way to get a formatted dump of the HOMEBLOCK of the disk.

ftp://ftp.hp.com/pub/openvms/freeware/diskblock/diskblock.zip

This is DISKBLOCK 6.2, and the kit will install on IA64.

Whoever created the zip file must not have preserved the vms file attributes, because the contents need to be
repaired.

You will need to do the following to "reconstitue" the vmsintal kit.

$ unzip diskblock
$ backup/list/repair diskblock062.a
$ backup/list/repair diskblock062.b
$ backup/list/repair diskblock062.c
$ backup/list/repair diskblock062.d
$ backup/list/repair diskblock062.e

Install diskblock, then do the following:

$ mount/for $1$dkb0:
$ mcr diskblock
DISKBLOCK> select $1$dkb0:
DISKBLOCK> read 1 ! primary homeblock (but will be bogus with /GPT)
DISKBLOCK> dump/home

See if the checksum if valid

DISKBLOCK> read 2 ! or what the Alternate Home LBN:
DISKBLOCK> dump/home

Well, it isn't too hard to reproduce the problem on an Alpha (as long as we are able to use DISKBLOCK to write to the disk).

My guess is that the EFI console is either writing to some block in the GPT and then mou/for is seeing this as a home block, or there may be left overs as described in Hoff's writup on OpenVMS Tips: EFI GPT Structures on ODS-2 and ODS-5 Disks

http://labs.hoffmanlabs.com/node/112

Here's what I did to reproduce the "effect" of having mou/ov find a label that mou/for does not. Note that this can be a serious problem if you are using shadowing, because the normal bladeguards in the SCB are bypassed (because the SCB can't be found if a valid homeblock is not found). For example if the disk is a member of a shadow set, and then the shadow set is dismounted or a member is dismounted, normally VMS will writelock this device when it is mounted without /shadow or /over=shadow. But that is not the case if a valid homeblock is not found.

$ init lda100: itrc01 /sys /own=[1,1] /cluster=1 /limit=10000 /gpt

At this point mount/for will find the label ITRC01.

However if we overwrite LBN 2 with all zeros (using DISKBLOCK), the volume will still mount correctly with mount/over=id, but will mount with no label displayed when mounted with mount/for. Also, if this volume is converted to a shadowset, it will not be protected against accidental write by mou/for.

See attachment for more details.

Jon

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 03:32:50 GMT

Ken,

By the way, changing the volume label won't help the problem, since the set volume/label updates the valid homeblocks, not the block that is being interpreted as a homeblock by mount/foreign.

Two possible work arounds that I can think of. If /GPT isn't needed the next time you got through the init, restore, do an init/nogpt.

The other is to try to determine what block is being interpreted incorrectly, and try to modify the block so the checksums are incorrect. But this may break something if the EFI is really using the block. The tool to modify the block is DISKBLOCK.

The real fix is for HP to make mount /foreign use the same search/checks for homeblocks as the normal mount code does, and issue a patch. Unfortunately they will probably only supply that patch to people with valid software contracts, even though it is not an enhancement, only a correct implementation of functionality.

Jon

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 03:51:42 GMT

Ken,

I just tried the following, but it does not work. So a non-image restore appears to be the only supported way to "cleanup"

$! following does not work!
$ init ... /nogpt
$ mou/for
$ backup/image/noinit

Unfortuneately, the new disk will still have GPT.SYS mapping lbn 0-33.

And you can't delete GPT.SYS while the disk is mounted files-11. It may be possible to use DISKBLOCK to delete the GPT.SYS file, but it would be much easier to just use a non-image restore like you discovered.

Jon

Re: Missing '/foreign' labels on image backups

Shriniketan Bhagwat — Tue, 23 Nov 2010 03:54:03 GMT

Hi,

Even I expect MOUNT/FOR to display the label of the disk.

$ mount/for UTIL$LDA1:
%MOUNT-I-MOUNTED, LD_DISK mounted on _UTIL$LDA1:
$

If the MOUNT/FOR command is not displaying the label, then what does the SHOW DEV dkb0: command display after the mounting the disk with /FOR qualifier?

Below are the sample commands with which you can read the label of the foreign mounted disk.
$
$ init UTIL$LDA1: LD_DISK
$ mount/for UTIL$LDA1:
%MOUNT-I-MOUNTED, LD_DISK mounted on _UTIL$LDA1:
$
$ dump/blo=coun=1 UTIL$LDA1:
Dump of device UTIL$LDA1: on 23-NOV-2010 08:02:18.52

Logical block number 0 (00000000), 512 (0200) bytes

8BDFFF76 905F0000 000501FB 000609F7 03039401 001E65C0 11C00200 15C600A0 ..F...@.@e......w...{....._.v._. 000000
73206120 746F6E20 73692020 20202020 2020204B 5349445F 444C0087 80FDFF74 t.}...LD_DISK is not a s 000020
00000000 00000000 00000000 00000000 0000000A 0A0D6B73 6964206D 65747379 ystem disk...................... 000040
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 ................................ 000060
â ¦
â ¦
â ¦
$ open/read x UTIL$LDA1:
$ read x record
$ label = f$extr(38,12,record)
$ show symbol label
LABEL = "LD_DISK "
$ close x
$ dismount UTIL$LDA1:
$

Regards,
Ketan

Re: Missing '/foreign' labels on image backups

Shriniketan Bhagwat — Tue, 23 Nov 2010 04:15:03 GMT

Hi Jon,

>> Unfortuneately, the new disk will still have GPT.SYS mapping lbn 0-33.

BACKUP has been enhanced with /GPT qualifier recently to control the propagation of GPT.SYS from source to destination disk. This has been released as part of VMS84A_UPDATE-V0400 and VMS84I_UPDATE-V0400 kits.

Regards,
Ketan

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 04:25:38 GMT

In my Nov 23, 2010 03:13:17 comment I said "However if we overwrite LBN 2 with all zeros (using DISKBLOCK)". That should have been LBN 1 (the first candidate location for a homeblock). The attachment had the correct info.

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 04:34:42 GMT

Ketan,

Thanks for the info about updates to 8.4, but that won't help Ken (see his initial post).

Just for your information, what is in LBN 0 may look like the label, but it is just text put there by INITIALIZE.

Try the following and you will see that what is in LBN 0 is put there by the init command, but it isn't modified by the set volume /label command (which updates the homeblocks)

$ init util$lda1: itrc01
$ mou/for util$lda1:
$ dump/blo=(c:2) util$lda1: ! note that lbn 1 has the label
$ dism util$lda1:
$ mou/ov=id util$lda1:
$ set vol/lab=ketan util$lda1:
$ dism util$lda1:
$ mou/for util$lda1:
$ dump/blo=(c:2) util$lda1: ! note that lbn 0 still has ITRC01, but lbn 2 has KETAN
$ dism util$lda1:
$ init util$lda1: itrc01/gpt
$ mou/for util$lda1:
$ dump/blo=(c:2) util$lda1: ! note that lbn 2 is no longer a homeblock, it is EFI related stuff
$ dism util$lda1:
$ mou/ov=id util$lda1:

Jon

Re: Missing '/foreign' labels on image backups

Shriniketan Bhagwat — Tue, 23 Nov 2010 06:05:53 GMT

Hi,

As I remember, the GPT structures conflict with the traditional block locations of ODS structures including the bootblock, the ODS primary homeblock, and with the index file. Accordingly, the homeblock has been relocated using the existing homeblock search algorithm, and all ODS homeblocks are now effectively secondary homeblocks, and the index file has been relocated. The GPT structure requires data structures which occupy the first 34 blocks of the disk rounded up to the cluster factor, and the last 33 blocks of the disk (also) rounded up. GPT basically maps to Protective Master Boot Record, Partition Table Header, and several Partition Descriptors. The Partition Descriptors are an array which provides the starting and ending disk block values for the partition, its name, and a GUID. Hence with GPT, the LBN2 is not the homeblock.

Regards,
Ketan

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 06:26:48 GMT

I keep getting off by one errors because LBNs are zero based, not one based.

LBN 1 is what used to be the primary homeblock, but with GPT is not. I keep saying LBN 2 when I mean "second block" (or LBN 1).

Regardless, the first block on the disk (LBN 0) is what was formerly called the Boot Block, and it does not contain the label used for the mount checks. It may contain text that has the label, but that is just a comment.

The label is stored at offset HM2$T_VOLNAME in the homeblock (and should be updated in all copies of the homeblock when the set volume/label command is given). There can be many more than 2 copies (it depends on the cluster factor on the disk).

What appears to be cause of the behavior reported by Ken, is that mount/foreign is not as strict with its checks as the normal mount code. There are many fields that "must be nonzero for a valid home block", and mount/foreign evidently is not making these checks, or it would not be fooled by filling LBN 1 with 512 bytes of 0x00. That was the way I reproduced the problem condition. That probably is not the LBN involved in Ken's case, and perhaps it isn't even a block full of zeros, but something is making mount/foreign behave differently than normal mount does.

Jon

Re: Missing '/foreign' labels on image backups

Ken Randell — Tue, 23 Nov 2010 14:12:28 GMT

Thanks so far for all of the clues/suggestions/answers.

I have not been able to reconstruct the exact scenario that got this disk in this shape.

Attached is the output of the LD TRACE suggested by Mr. Pinkley above. If I am reading it correctly, block 1 is being read for both mount/over=id and mount/foreign (?)

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 20:04:56 GMT

Ken,

I am in the middle of work right now, but a quick glance makes me suspect that mount/foreign is misinterpreting LBN 1 as a valid homeblock (since that's all it looks at).

08:55:41.81 00.00 000B684E 0 0 NORMAL PACKACK|INHERLOG
08:55:41.81 00.00 000B684E 1 512 NORMAL READPBLK
08:55:41.81 00.00 000B684E 1 512 NORMAL READPBLK
08:55:41.81 00.00 000B684E 1 512 NORMAL READPBLK
08:55:41.81 00.00 000B684E 1 512 NORMAL READPBLK
08:55:47.57 00.00 000B684E 0 0 NORMAL UNLOAD|CLSEREXCP

Can you do the following?

$ set proc/priv=log ! or /priv=all
$ dump /block=(s:1,c:1) $1$dkb0: ! dump LBN 1

I expect to see all zeros.

If you look at the blocks being read by mount/over=id, you will see that mount keeps searching (in a specific sequence) for the home block. I would guess that it found it at LBN 15427243 (and the SCB at 15422416)

You can also do the following:

$ dump/header/block=c:0 dkb0:[000000]indexf.sys

and look at the first few retrieval pointers.

Also to see where the SCB is:

$ dump/header/block=c:0 dkb0:[000000]bitmap.sys

and look at first retrieval pointer.

Jon

Re: Missing '/foreign' labels on image backups

Ken Randell — Tue, 23 Nov 2010 21:01:55 GMT

Information as requested...

logical block dump as:

dump /block=(s:1,c:1) dkb0:

shows all zeroes.

retrieval pointers for indexf.sys

Map area
Retrieval pointers
Count: 16 LBN: 48
Count: 16 LBN: 1024
Count: 16 LBN: 2064
Count: 16 LBN: 15558336
Count: 131536 LBN: 15426800

Re: Missing '/foreign' labels on image backups

Jon Pinkley — Tue, 23 Nov 2010 21:34:02 GMT

Ken,

From the retrieval pointers, it appears I was incorrect in my prediction about where the homeblock used by mount/over=id was found. I now think it was LBN 1034. I also think the cluster factor on the disk is 16.

Before yesterday, I wouldn't have expected that mount/for would have been fooled into thinking a block full of zeros was a valid homeblock, but that appears to be what is happening. That is especially bad because a disk initialized with /erase will have many zero filled blocks. It just so happens that the checksums for a zero filled block are also zero, so the homeblock checksums "pass". There are many other checks that should be in play, but evidently are not.

You can probably work around the problem by writing something other than all zeros to LBN 1 (this could even be done with DCL). However, mount /for may then be confused by another zero filled block in its search for a homeblock, so it may be an iterative process. The most robust way to avoid the problem would be to init/nogpt (in my opinion, but I don't have an IA64, so perhaps that isn't a good idea. Hoff would know a lot more about it, since he was involved with the design for ODS/GPT coexistence.

More later...

Jon