Initializing disks

[roose]

Hi all,

I am currently analyzing some of our disks on our Alpha systems (OVMS v7.3-1, ES80, EMC SAN disks) since recently, we are seeing slow response on some disks, specially our system disk, whenever we do a DEFRAG SHOW/VOL. So far, fragmentation does not seem to be an issue on this, but HP support asked us to do a backup/restore of our system disk on our next downtime.

We are already planning to do this, but I would like to get some input from our experts here concerning the current setup of our system disk. It seems that when my predecessors upgraded our systems from the old GS60 to our current ES80, they migh have forgotten something. Specifically, I am wondering if the current cluster size of the disk is appropriate (see attached DFU REPORT result). Also, can you please clarify to me the following INIT parameter information:

1. /HEADER - is there a way to check what is the current header settings of my disks? If not, what tool can I use to estimate the value of this parameter?
2. /MAXIMUM_FILE - if I just use this parameter, without the /HEADER, will the HEADER value be computed from the value of MAXIMUM_FILE?

Lastly, what we are planning to do for the backup/restore is to do an image backup of our disk to another disk (saveset on a disk). Other threads suggested using a file-by-file restore of the disk, rather than using /IMAGE. Will this work for a system disk, and at the same time, address any fragmentation, if there is?

Thanks in advance for your help.

[Karl Rohwedder]

You have a max # of files of 8838720 and a total of around 15000 files of which 14500 are contigous. At a 1st glance, fragmentation seems not to be a problem. The /CLUSTER_SIZE is quite small, I would consider increasing it, if you do a BACKUP/RESTORE. May be there a special recommendations for EMC storage.

The setting of /MAX has no effect on the /HEADER value. From the help:

o /HEADERS: 0.5 percent of the size of the current device
MAXBLOCK (an F$GETDVI item code)

regards Kalle

[Robert Gezelter]

Roose,

If I am read your attachment correctly, virtually no files on your system disk are fragmented (14915 files have an alloation, of which 14504 are contiguous). The number of extents involved in ERRLOG.SYS (793, to be precise) is caused by the ongoing extension of the file over the years, one small amount at a time.

If I am understanding you correctly, disk performance is reduced everytime that you check on the fragmentation of the volume. This is unsurprising, as the analysis of fragmentation is a fairly intensive operation in terms of disk operations.

It would be speculation, but there is a good chance that all of the files that are fragmented are the various log files. As these are being appended to on a regular basis, they will not be contiguous for but a brief moment.

But back to the basic problem. Other than when you are running the DEFRAG SHOW/VOLUME (which is, but definition, very intensive in disk operations), what is your overall disk performance. What do the various MONITOR displays show?

A system performance problem when doing one, exceptional operation (e.g., an intensive disk analysis) should not generally be considered a "performance problem". With all due respect to HP support, it would be far more productive, and far less disruptive, to identify those 411 files on the system disk that are not contiguous, amd relevant (not log files of some sort that are generally only appended to). Those files that fragmented, and relevant to performance can then be copied (using the COPY command with the /CONTIGUOUS option). Generally, this can be done without a re-boot or disruption of any kind.

With all due respect to HP support, a "full restore" of the system disk, to fix only 411 files, most of which are "write-always, read-almost-never" files is not likely to change system performance (it MIGHT have an impact on the operation of the DEFRAG SHOW/VOL operation, but that would be from a reduction of the analysis involved).

If there are overall performance issues, I would do a full performance review. If there are (and that is a big IF), fragmentation of the system disk (as detailed in your posting) is not the most likely cause.

- Bob Gezelter, http://www.rlgsc.com

[Heinz W Genhart]

Hi Roose

after a short look at your attachement, the only thing I see is, that higwatermarking on the Systemdisk is enabled. If there are no special security regulations at your site, I would recommend to disable highwater marking.
This task could increase disk performance.

$ SET VOLUME /NOHIGHWATER 'disk-name'

Regards

Heinz

[Volker Halle]

Roose,

for disk fragmentation to become a real 'performance problem', the fragmented files need to be read and those reads would then be causing many window-turns (visible with $ MONITOR FCP as a high Window Turn Rate).

I would strongly doubt, that DEFRAG SHOW/VOL processes more than INDEXF.SYS (and BITMAP.SYS), so it won't probably even touch those fragmented files and won't be generating window turns.

DEFRAG SHOW/VOL may be causing a high IO-load on your system disk and storage controller. A cluster-size of 3 may not be an ideal value for the storage controller. For HP EVAs, the general advice is to use a cluster-size with a multiple of 4 blocks.

Volker.

[Robert Gezelter]

Volker,

I will defer to Hein, who I believe did the tests, but for most cases, the cluster size of 3 does not affect the actual IO operations to the system disk, merely the allocation of space.

Since most of the files on the system disk are contiguous, that would not be a problem in this case.

- Bob Gezelter, http://www.rlgsc.com

[Robert Brooks_1]

Volker wrote . . .

A cluster-size of 3 may not be an ideal value for the storage controller. For HP EVAs, the general advice is to use a cluster-size with a multiple of 4 blocks

Bob wrote . . .

I will defer to Hein, who I believe did the tests, but for most cases, the cluster size of 3 does not affect the actual IO operations to the system disk, merely the allocation of space.

--------------

The EVA has some annoying performance characteristics such that Volker's statement regarding the cluster size is quite correct. If the starting LBN of an I/O is not aligned
correctly, substantial performance degradation can happen. Using a cluster factor that is a multiple of 4 reduces the likelihood of a misaligned I/O. For that reason, we've modified a few things (COPY, BACKUP, SHADOW_SERVER process for shadow copies and merges) to make sure that they do their I/O in sizes that are a multiple of 4.

Bob's statement is also correct in that when an I/O occurs, the number of bytes transferred is the number of bytes requested; the I/O request is *not* rounded up to the cluster size.

-- Rob (VMS Engineering)

[Robert Gezelter]

Rob,

What I meant in my posting (to which you graciously followed-up) was that the Cluster Factor address space allocation, not IO operations.

I agree with Volker's comments about the performance of the EVA, to the extent that EVA performance is improved if requests are aligned on a four block boundary.

I have not seen any calibrated benchmarks on system performance when PRECISELY the same workload is done by two systems, that are otherwise identical except for the system disk cluster factor.

- Bob Gezelter, http://www.rlgsc.com

[GuentherF]

Is this all about long runs of DEFRAG? Is this the only concern that leads to the statement "we are seeing slow response on some disks"? Is there a REAL disk file(s) I/O performane issue? Detected/measured how?

/Guenther

[roose]

Gunther: It started as such, having a longer than previous runs of Defrag on our system disk. Thus, when we ask HP about it and recommended that we do a backup/restore of our system disk, I started investigating on my own as I doubt that there is really fragmentation on the system disk itself. With due respect to HP support, I opened this thread to ask other's opinion on this. And as always, got additional interesting information from our VMS experts here in the forum :)

[roose]

All, thanks to all who replied. The information that I got is quite an education for me, as always.

Just before I close this thread, can anyone just help me on my last question here:

"Lastly, what we are planning to do for the backup/restore is to do an image backup of our disk to another disk (saveset on a disk). Other threads suggested using a file-by-file restore of the disk, rather than using /IMAGE. Will this work for a system disk, and at the same time, address any fragmentation, if there is?"

[Thomas Ritter]

Roose, about the backups read this thread...

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1028893

[roose]

Thanks Thomas. I might have not made myself clear. What I am asking is this: we are shutting down our nodes and do what HP recommends, backup/restore. What we plan is actually boot from the CD OS disk so that the system disk can be mounted privately, and no open files on it. We will do an image backup of the system disk to a saveset on another disk, init the disk and will later on restore from that saveset.

My question is: after doing an init on the disk, should we do a BACKUP/IMAGE/NOINIT (disk mounted as FOREIGN) or a file-by-file restore (disk mounted regularly) should be done, primarily to address any fragmentation issue (if there is)? Also, will a file-by-file restore be advisable on a system disk?

[Karl Rohwedder]

A file-by-file backup on a systemdisk will destroy the systems structure (file backlinks to VMS$COMMON). Only use BACKUP/IMAGE [/NOINIT], use /NOPINIT if you have altered the disk characteristics!

regards Kalle

[Volker Halle]

Roose,


We will do an image backup of the system disk to a saveset on another disk, init the disk and will later on restore from that saveset.


Why don't you just INIT another disk (to be later used as the system disk) and then do a BACKUP/IMAGE from the system disk to that disk. Them use the other disk as your new system disk. Either change the console boot path or change the LUNs on the storage subsystem.

You must backup/restore the system disk using /IMAGE

Volker.

[roose]

Thanks to those who replied to my query. I will be closing this thread now as I already have the necessary information that I need.

[Robert Gezelter]

Roose,

You need to do an image restore, if you are doing the full image.

On the other hand, you could ELIMINATE the fragmented files by identifying the most fragmented file using DEFRAG, and then using COPY/CONTIG to make it contiguous, without any interruption to system operation.

- Bob Gezelter, http://www.rlgsc.com

[Gregg Parmentier]

Closed, or not, I thought I'd throw this one in.

Does VMS still have a limit to how fragmented the index.sys can be? Is the defrag program you are using able to defragment the index.sys? Might the slow response be due to index.sys fragmentation? I do not recall how to check how fragmented the index.sys file is, but suspect that this is why HP is recommending the save/restore.

In general, when I do an init, I set /header and /maximum_files with the same large value. The index.sys will then be created with that number of blocks and is less likely to be extended and fragmented over time.

[Robert Gezelter]

Greg,

Your question of fragmentation within INDEXF.SYS (actually xxx:[000000]INDEXF.SYS) is a good question.

It could be a problem, and indeed a full restore would correct it. On the other hand, it may not be the issue.

On a general basis, my objection to the "Do a FULL BACKUP/RESTORE" approach is that it is a shot in the dark. It is not reasoned science, and there is no reason to take dramatic steps (e.g., shutdown an entire cluster for a significant period) without the science.

If the problem is INDEXF.SYS fragmentation, a simple DUMP/HEADER/BLOCKS=(START:0,END:0) SYS$SYSDEVICE:[000000]INDEXF.SYS will clearly show the problem.

Now that we have Dynamic Volume Expansion, I have evolved my recommendations to clients, in that I recommend configuring volumes with parameters that will (hopefully) preclude having to do a backup/restore operation to enlarge the volume. This means:
- maximizing the storage bitmap (max files)
- pre-extending the INDEX file to a resonable level (smaller than max files, but reasonably large).

IMHO, the name of the game in OpenVMS cluster operations is to minimize the need for disruption to the business and users, a cluster shutdown should be a VERY rare event.

Actually, making a clone of the existing system pack, and switching system roots back and forth, combined with rolling reboots can, if managed properly, accomplish the same results of the image restore, with NO CLUSTER DOWNTIME (upper case added for emphasis).

- Bob Gezelter, http://www.rlgsc.com

[Jan van den Ende]

Roose,

Robert wrote

IMHO, the name of the game in OpenVMS cluster operations is to minimize the need for disruption to the business and users, a cluster shutdown should be a VERY rare event.


well, to support his stance, _OUR_ cluster has an uptime of over 9.5 years (and it has NOT exactly been unchanged!)

fwiw

Proost.

Have one on me.

jpe