Operating System - OpenVMS
1748185 Members
4326 Online
108759 Solutions
New Discussion юеВ

Re: INITIALIZE /INDEX=xxxxx

 
SOLVED
Go to solution
Hein van den Heuvel
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

>> My experience is that the output disk of a BACKUP/IMAGE restore uses the space after the index before using the space before the index.

Odd, but not inconsistent with my observations in:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1136467
There I show /IND=BEG and back/image filling from low LBNs onwards.

Jon>> The point is that when doing an image restore, the lowest available blocks are not what is chosen.

They are, with INIT/IND=BEGI an BACKUP/NOINIT

Jon>> File positioning information is not preserved by an image backup,

WHich is just as well! Many image backups are specifically done to de-frag, thus move extents.


Jon>> Having the index in the default middle position shouldn't affect performance one way of the other.

So that argues for using BEGIN (or END) if it has no performance value, then it has no value.

Cheers,
Hein.
Wim Van den Wyngaert
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

You can inspect disk layout in the defragmenter with /int=decw. And click on the map to get file details.

In my case, it shows a mess for most disks (all kind of raids on ma8000) and this while we defragment every weekend. But without stopping anything.

May be it would be a good idea to stop everything during a weekend a do a defrag with hotfile placement.

I also noticed that on a disk with db files that were created with /contig were not placed in a serial manner. The first one in the beginning, the 2nd one at the end of the disk. The 3rd was created /nocontig because the index in the middle prevented /contig. And resulted in a piece following the 1st file (before the middle) and a 2nd directly after the middle. Thus we have holes before the middle and before the last file.

fwiw

Wim
Wim
Wim Van den Wyngaert
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

I started with a fresh defragmented empty disk of 71 MBl. I created in a loop contig files of 10 Mbl. After 5 this was impossible because contig space was gone.

I discovered in defrag map that the files were created with an interspace of about 7MBl.

No idea why.

Wim


Wim
Art Wiens
Respected Contributor

Re: INITIALIZE /INDEX=xxxxx

Dean ... 10 points for mentioning RM05's!! I used to work for Control Data - the OEM for those drives - the CDC 9766. So many hours of my life I'll never get back - isopropyl on the punch card, swiping the new head in an "S" motion to clean them ... aligning heads with the 'scope and the servo pack ... only being able to do two or three before you had to put the shroud back on to let it come back up to temp ... all for 300MB!! Good times!!!

Anyways, when it comes time to init my EVA volumes, I will consider all the suggestions provided here ... index at the beginning and lots of headers and directories.

Cheers all,
Art
Dean McGorrill
Valued Contributor

Re: INITIALIZE /INDEX=xxxxx

Art,
so you worked on the rm05's, wow! that
goes back there. I coined the phrase disk
herpes with those. how many times I'd see
the operators put a bad disk in a good drive, and make it a bad drive. then put
a good pack into the new bad drive. on and on. when they were working, they were pretty
good drives. I've included my initdisk, with a fix to select index=begin. tweak as needed it preallocates headers and directorys.

Dean
Jon Pinkley
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

Several comments in response to Hein's responses to my note.

When I stated that file placement isn't preserved, I meant explicit file placement retreival pointers are not preserved by a backup/image.

Specifically, a file placed with something like

$ cre/fdl=sys$input $5$lda1:[000000]RESERVED.SPACE
file;allocation 10000;contiguous yes;area 0;EXACT_POSITIONING yes;position logical 190000
Exit
$ dump/head/bl=c:0 $5$lda1:[000000]RESERVED.SPACE

most removed for brevity

Map area
Retrieval pointers
Placement control: Exact
Count: 10000 LBN: 190000

Checksum: 8800
$

After the backup/image this file is not treated specially, other than to remain contiguous (if possible).

In my experiment, the new file had the following as retrieval pointers:

Map area
Retrieval pointers
Count: 10000 LBN: 101086

Checksum: 40955

Also, my comment "Having the index in the default middle position shouldn't affect performance one way or the other." was in the context of an EVA controller. On single spindle device, it probably does make a difference, but whether you would notice it depends on how frequently the file headers are referenced while accessing other files on the disk.

I'll respond to the file placement done by backup/image in another note.

Jon
it depends
Jon Pinkley
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

Wim Van den Wyngaert>>>>I started with a fresh defragmented empty disk of 71 MBl. I created in a loop contig files of 10 Mbl. After 5 this was impossible because contig space was gone.

>I discovered in defrag map that the files were created with an interspace of about 7MBl.

>No idea why.

-------------

My guess is that this was in a cluster. Other nodes "reserve" a range of available free space so they don't have to bother coordinating with the other members when allocating extents. If a node requests contiguous allocation, and can't satisfy it, it will force the others to flush and try again. However, once the gaps are there, they in effect reduce the "largest free extent".

You can use explicit placement to force the other nodes to relinquish the space they have reserved but not used. See my previous note for one possible say to do this. Using the output from defrag show /free will help you find where to place the file.
it depends
Jon Pinkley
Honored Contributor

Re: INITIALIZE /INDEX=xxxxx

In my response from Jun 18, 2007 19:29:20 GMT, I stated:

--------------------------------------------------------------------------------
The point is that when doing an image restore, the lowest available blocks are not what is chosen. I can't say whether the initial free space after the indexf.sys is initially larger or smaller.
--------------------------------------------------------------------------------

I've done some experiments with LD V8.3 on VMS 7.3-2 with patches current as of Mar 24, 2007.

It appears I was not correct in my assumption of how space was allocated by backup/image, where I guessed that it kept a pointer to the highest used space, and then allocated sequentially from there. The clue was that some files were copied to very low LBNs.

I now think Backup /image uses the smallest areas available first, so it can leave the largest free extent when it is complete. The following report is from DFG.

By the way, well worth installing Defrag just for the reporting, and no license is needed for that. An example installation is included in the logfile attached.

The following is from a 200,000 block LD device:

$ init lda3: /cluster=1/header=500 itrc
$ mou/ov=id lda3
%MOUNT-I-MOUNTED, ITRC mounted on _$4$LDA3: (SIGMA)
$ defrag show /location/file=4 /free lda3:
Disk File Optimizer for OpenVMS DFG V2.7
├В┬й 2002 Compaq Information Technologies Group, L.P

F r a g m e n t a t i o n R e p o r t

DISK$ITRC 19-JUN-2007 17:39:15.99

The fragmentation index is 1.0
1 - 20.9 is excellent
21 - 40.9 is good
41 - 60.9 is fair
61 - 80.9 is poor
81 - 100 indicates a badly fragmented disk
Approximately 0.8 (out of 80.0 possible) is due to file fragmentation
Approximately 0.2 (out of 20.0 possible) is due to freespace fragmentation


Freespace Summary:
Total free space: 199425 blocks
Percentage free: 99 (rounded)
Total free extents: 4
Maximum free extent: 98965 blocks, LBN: 1035
Minimum free extent: 514 blocks, LBN: 100571
Average free extent: 49856 blocks
Median free extent: 98914 blocks

Freespace Extents:
LBNs 2 to 1033 (1032 blocks free)
LBNs 1035 to 99999 (98965 blocks free)
LBNs 100571 to 101084 (514 blocks free)
LBNs 101086 to 199999 (98914 blocks free)


File Fragmentation Summary:
Number of files (with some allocation): 4
Total file extents on the disk: 7
Average number of file extents per file: 1.750000
Median number of file extents per file: 1

Most Fragmented File:
[000000]INDEXF.SYS;1 (4 extents)

1 file with 4 or more extents:

[000000]INDEXF.SYS;1 (4 extents)
VBN 1, LBN 0 to LBN 1 (2 blocks)
VBN 3, LBN 1034 to LBN 1034 (1 blocks)
VBN 4, LBN 101085 to LBN 101085 (1 blocks)
VBN 5, LBN 100052 to LBN 100564 (513 blocks)

$

In this case, the 98,965 block free extent starting at LBN 1035 is the largest, and backup/image apparently preserves that until the smaller pieces have been used.

When I initialized with /index=80000 the free space at the end of the volume was preserved.

Note that backup/image is writing to a disk that is mounted foreign, thus the XQP is not involved in the allocation of blocks. Backup itself is in full control of the allocation. In the attached log, you can see that a non-image backup of files to a newly initialized disk results in a different placement, which appears to be based on first available, but this was a special case, as the disk was mounted privately and was just initialized. In volumes mounted cluster wide, other nodes can reserves a cache of free extents to optimize allocations from a time perspective.

I am attaching the log file of my experiments, if you are interested in seeing output of DFU report, etc.

I also ran into an issue with OpenVMS 8.3 factory installed software (without patches) and LDDRIVER V9.0. Doing a backup/image of a 200,000 block LD to another 200,000 block LD resulted in a disk that could not be mounted.

$ mou/for $5$lda3:
%MOUNT-I-MOUNTED, ITRCMID mounted on _$5$LDA3: (SIGMA)
$ backup/image $5$lda1: $5$lda3:
$ dism $5$lda3:
$ mou/ov=id $5$lda3:
%MOUNT-F-FILESTRUCT, unsupported file structure level
$ help/mess FILESTRUCT/fac=mount


FILESTRUCT, unsupported file structure level

Facility: MOUNT, Mount Utility

Explanation: This message occurs on pre-Version 7.2 systems when you
attempt to access a disk that was initialized on a Version 7.2
or later system. Version 7.2 extends the size of the BITMAP
field and the increased size is incompatible with earlier
versions.

User Action: Check the size of BITMAP.SYS on the disk you want to access.
If the size is 256 or greater and you want to access the
disk from a pre-Version 7.2 system, you must copy the data
to a disk with a BITMAP.SYS that is less than 256 blocks.
If you use the DCL command BACKUP/IMAGE, be sure to use the
/NOINITIALIZE qualifier.

$

Doing a backup/noinit didn't help.

More details in the attachment if you are interested.

The same sequence of commands works on a VMS 7.3-2 system with patches current as of Mar 24, 2007.

I haven't determined the cause of the problem, I have dumps of the homeblocks in the logfile.

I see there is a patch to backup, and I think that is more likely to be at fault than LDDRIVER.
it depends
Jur van der Burg
Respected Contributor

Re: INITIALIZE /INDEX=xxxxx

Jon,

I don't think this is an LD issue. I could reproduce it with LD V8.2 on VMS V7.3-2 too. Also reprodcued it on Alpha V8.3 with LD V9.0, and on Itanium V8.3 with LD V9.0.

I remember that I've seen something like this in the past, when creating small containerfiles and initing them. I'll see if I can find something about that in my mail, as I remember sending a certain person in VMS engineering mail about that.

If you leave out /CLUSTER=1 on the INIT command it works, so my guess is that it's something in INIT code which is used by backup.

Small reproducer:

$ ld create d1/size=100000
$ ld create d2/size=100000
$ ld connect d1 lda1
$ ld connect d2 lda2
$ init lda1: /cluster=1 /system /head=500 /user=itrc itrcdemo
$ mount lda1: itrcdemo
$ backup sys$loadable_images:*.* lda1:[SYS$LDR]/own=system/trun/ver
$ mount/for lda2
$ backup/imag lda1: lda2:
$ dism lda2
$ mount/over=id lda2

Jur.
Jur van der Burg
Respected Contributor

Re: INITIALIZE /INDEX=xxxxx

Seems like this is the issue:

VMS82A_BACKUP-V0200:

5.2.2 %MOUNT-F-FILESTRUCT Error When Mounting Image Backup

5.2.2.1 Problem Description:

After restoring an image BACKUP, the disk may fail to mount with a %MOUNT-F-FILESTRUCT error

5.2.2.3 Problem Analysis:

BACKUP now preserves the volume expansion size, an attempt to restore a save set with expansion size smaller than 25MB will result in failure to mount the disk.

Jur.