Would defragging an oracle filesystem cause a 15 minute lock on a control file?

[Geoff Wild]

I have a script that defrags vxfs file systems - the other day on one of our servers, oracle froze. DBA says one of the control files was lock by something for 15 minutes!

I checked my logs, and sure enough, the fsadm was running on that file system at that time:

defraging /oracle/BWP/sapdata1 at Wed Jul 18 23:15:17 MDT 2007

Directory Fragmentation Report
Dirs Total Immed Immeds Dirs to Blocks to
Searched Blocks Dirs to Add Reduce Reduce
total 3258 289 2970 0 0 0


Directory Fragmentation Report
Dirs Total Immed Immeds Dirs to Blocks to
Searched Blocks Dirs to Add Reduce Reduce
total 3258 289 2970 0 0 0

Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
5125 9955 5 12974680
blocks used for indirects: 77
% Free blocks in extents smaller than 64 blks: 0.00
% Free blocks in extents smaller than 8 blks: 0.00
% blks allocated to extents 64 blks or larger: 99.99
Free Extents By Size
1: 0 2: 16 4: 22
8: 10 16: 11 32: 1
64: 1 128: 1 256: 0
512: 0 1024: 0 2048: 1
4096: 1 8192: 1 16384: 1
32768: 1 65536: 1 131072: 2
262144: 0 524288: 2 1048576: 1
2097152: 1 4194304: 0 8388608: 1
16777216: 0 33554432: 0 67108864: 0
134217728: 0 268435456: 0 536870912: 0
1073741824: 0 2147483648: 0

Extent Fragmentation Report
Total Average Average Total
Files File Blks # Extents Free Blks
5125 9955 1 12974705
blocks used for indirects: 52
% Free blocks in extents smaller than 64 blks: 0.04
% Free blocks in extents smaller than 8 blks: 0.00
% blks allocated to extents 64 blks or larger: 99.99
Free Extents By Size
1: 79 2: 83 4: 65
8: 91 16: 84 32: 93
64: 117 128: 105 256: 81
512: 71 1024: 55 2048: 117
4096: 77 8192: 37 16384: 19
32768: 88 65536: 2 131072: 8
262144: 7 524288: 3 1048576: 2
2097152: 1 4194304: 0 8388608: 0
16777216: 0 33554432: 0 67108864: 0
134217728: 0 268435456: 0 536870912: 0
1073741824: 0 2147483648: 0

So, could this have caused the issue?

Note: the other change we had this week - on Sat, we migrated to a new EMC frame and went from 10 lvols to 1 x 500GB filesystem for the oracle db (a standard we have on most of our servers and have never had any issues).

Rgds...Geoff

[Sandman!]

IMHO the only thing that comes to mind is contention between fsadm and Oracle processes as may happen when the backend LUN is not striped correctly. Any stats that you can glean from Glance or Statspack?

[A. Clay Stephenson]

The honest answer is that I don't know but I have never seen this behavior. I can't imagine a control file so large that it would require 15 minutes to reorganize its extents -- especially given that the .ctl files are pre-allocated and I have to believe that they contained only a few extents anyway. You would have to ask a vxfs expert but I very much doubt that the files are actually locked on an individual basis bur rather the VX_FREEZE and VX_THAW ioctl calls are used to briefly freeze the entire file system for at most a few seconds at a time -- and should be invisible to all applications.

I generally don't go to as few LVOL's as you but with modern arrays I do tend to use only two or three LVOL's/filesystems and the split is only to allow different mount options. I do assume that the scsi queue depth has been bumped up.