SAN Storage and DFG

[MJ26]

I have a GS160 connected to a SAN environment.  We are running OpenVMS 8.4      My question is, how effective is OpenVMS DFG, becuase on the SAN side, isn't the data already fragmented around on multiple disks as the SAN manages its own capactiy?     Just a genearal explanation would help :)   Thanks!

[Bob Blunt]

At best it would be difficult to say for sure.  Yes, in many VRAID storage configurations, your storage "volume" is spread across a number of disks defined by the setup that you've established for that particular "volume."  Because that "volume," to OpenVMS, is supposedly a MOSTLY contiguous area your data can still *appear* to be fragmented and subsequently it might not be read efficiently.  If DFG (or another tool) says you've got fragmentation then you'd benefit from some form of compression to restore contiguity because that's how VMS would also see the data.  Also DFG, in my experience, works on only CLOSED files and makes them contiguous and leaves open or active files alone so the benefit of DFG or any other solution intended to inprove contiguity would be minimal unless you can quiesce activity to the files in question.

In most cases OpenVMS doesn't magically burrow into VRAID volumes presented to it and the controller(s) know best what lies underneath any particular "device" and the actual bit-for-bit I/O to and from the physical devices that are defined to make up a "device" or "volume" is managed by the controller that receives the I/O commands from OpenVMS.  VMS might know from the device ID information presented that a volume is presented from an EMC Symmetrix, HSG80, XP, Hitachi or whatever but native OpenVMS tools probably can't tell you exactly how that VRAID is configured "behind" the controller (how many disks, what chunk size, etc).

[John Gillings]

My personal bias has been that defragging on OpenVMS is, for all but pathological cases, unnecessary, and more risk than benefit.

Simple example, on a multi user, multi processing operating system, suppose I'm reading sequentially through a contiguous sequential file. I've just read some blocks, leaving the head in exactly the right place for my next read. What are the chances the head will still be there when I want to read the next block? ZERO. Since there's a small chance that with a fragmented file, the access pattern will be better, it's arguable that having contiguous files is suboptimal.

That said, it IS important that all the fragments in the file fit inside a single header. 

Add in caches, RAID, SAN and other storage options and the very concept of contiguity becomes very fuzzy.

The only way you will know if DFG helps is by measurement.

My attitude is I'll make sure my file options are set to have sane values for extensions and collect reasonable performance metrics. Leave RMS to look after the files and worry about only contiguity as ONE facet of file performance IF the metrics (or user complaints) reveal a problem

[Brad McCusker]

With all due respect to John (sincerely) - I think his statement "defragging on OpenVMS is, for all but pathological cases, unnecessary, and more risk than benefit" is somewhat mis leading.

If we are talking about locations of data on physical devices (whatever they may be) - then I agree with John's statement: "very concept of contiguity becomes very fuzzy".  No problem there. - quite frankly, who cares.

But what about the VMS file system?   VMS doesn't care where bits are on the disk, but it certainly does care how many logical fragments the file consists of, if there are multiple file headers, if it has to do excessive window turns, etc.   John's last paragraph really understates the importance of file system tuning and monitoring.  Poorly tuned file system (e.g. logically fragmented) will lead to excessive and unnecessary I/O. 

Our advice is to run a defrag tool - or at the very least run something like DFU to monitor logical fragmentation.  Pay attention to performance metrics - Window Turn Rate can be an indicator of wasted I/Os due to logical fragmentation.  And if you notice fragmentation, then run the defrag tool and revisit your file system configuration.

I don't see any risk in running a defrag tool on a fragmented disk.  I absolutely see risk in NOT doing it.

Brad McCusker

Software Concepts International

www.sciinc.com

[abrsvc]

I will agree with both John and Brad with the following clarifications:

In most cases the only performance hit will be with extremely fragmented files that incur winndow turns etc as described by Brad.  Following along with John's summation, in practice (especially with SAN environments), the true performance cost of these window turns is minimal and most likely unmeasurable unless extremely excessive in amount.  GIven the potential overhead incurred with the actual defrag operation on 24x7 operations, there will be little measurable gain for the possible performance hit during the defrag operation itself.

The botom line is, as usual, YMMV.

If you feel that the file system performance is being compromizd by fragmentation, then by all means defrag.  It won't hurt.  I would not expect to see significant performance gains with most systems.

Internal "defragmentation" or CONVERTing files may yield more of a performance gain if there are indexed files with many insertions and deletions of records.  This will be more independent from the actual hardware system than a typical defrag operation.

Dan

[Duncan Morris]

excessive fragmentation like this from DFU?

$1$DGA5:[INTERNET.MAS]YRISK.MAS;1                                           443275264/443275264 276/17805
$1$DGA5:[INTERNET.MAS]SCLIEN.MAS;1                                            2230144/2230144     8/479
$1$DGA5:[INTERNET.MAS]SPOLIC.MAS;1                                            2397056/2397056     8/496
$1$DGA5:[INTERNET.MAS]SHIST.MAS;1                                             4938496/4938496    14/910
$1$DGA5:[INTERNET.MAS]SRISK.MAS;1                                            31761536/31776704   27/1613

Even BACKUP struggles with these files, frequently failing with READATTR errors etc.

[GuentherF]

I fully agree with John. There are only two cases where you need to defragment: INDEXF.SYS file header full and no contiguous space to extend a directory file.

For anything else I have seen more harm to disk performance by a defragger than improvements.

In a real life case a system disk with application data was defragged. The application files ended up - nicely contiguous - on either end of the disk drive. This was the day of local disks with no controller caches. I/O performance dropped after the defrag.

With state-of-the art disk subsystems I don't see any need for a defragger except for the two emergency situations mentioned above.

But as has been said...YMMV.

/Guenther