Best way to set NO_CACHING for CACHE.DAT files

[Clark Powell]

On our Cache database we are not allowed to use XFC file caching becuase Cache is already using its own reserved memory for caching these databases. This is Intersystems rule for using XFC caching with Cache.

We are doing this by mounting the disks /NOCACHE. But Intersystems says we could also set the permanent file attribute on each CACHE.DAT database file, e.g. SET FILE/CACHING_ATTRIBUTE=NO_CACHING CACHE.DAT
A typical database disk has about 1,000 files, two of which are CACHE.DAT files or files NOT to be cached and the other 998 can be cached but are not used frequently. My question is, would there be any additional advantage to mounting the disk cached (ie without /NOCACHE) and just restrict caching on the two CACHE.DAT database files? Or would the simple solution of mounting the database disk, /NOCACHE, be just as effective?

[Jim_McKinney]

Since those other 998 files are infrequently accessed, it seems you might as well mount the volume /NOCACHE and make it a bit more efficient as the XFC will then be able to ignore the volume altogether rather than have to make a decision each time a file is opened.

[Steven Schweda]

> [...] would there be any additional
> advantage [...]

Yeah, it would let the cache stuff do its job
on the files where it'd be safe to let it.

> [...] not used frequently [...]

Then the benefit might be small. Hard to
estimate without a good definition of
"frequently".

> [...] just as effective?

Perhaps "practically as effective". Running
the experiment might be the best way to find
out.

If there's some risk that someone would
create a new no-cache file without setting
its no-cache attribute, then that might weigh
against the explicit per-file method, but I
suspect that I'd use the per-file method.

Leave a Post-it on the system console
terminal. Or add another test to your daily
system-check batch job.

[John Gillings]

Clark,

Those of us unfamiliar with your product can only make general (wild) guesses as to the effectiveness or otherwise of different optimisation strategies. Such guesses may be correct, but there are many more ways they can be incorrect. The less information about the your system, the more inaccurate (and it would be difficult to see how we could know less at the moment!).

Of course, you can perform your own experiments to gather objective data, but unless you're striving to extract the absolute maximum performance out of your system, it's likely to be a rapidly diminishing ROI.

My recommendation is to follow the advice of the software suppliers who (hopefully) know and understand the performance characteristics of their product.

If you have a performance issue, please post some more detail, including versions, hardware models, configurations, gathered statistics and objectives.

[Jon Pinkley · Accepted solution]

Clark,

Mount/NOCACHE turns off all XQP (FCP) caching

From $ HELP MOUNT /CACHE

If you specify /NOCACHE for a disk device, all caching is
disabled for this volume. Note that the /NOCACHE qualifier is
equivalent to /CACHE=(NOEXTENT,NOFILE_ID,NOQUOTA,WRITETHROUGH).

It also disables XFC file caching, but there is currently (VMS 8.3) no way to disable the XFC on a volume basis without disabling all XQP (FCP) caching as well (at least I am not aware of a method).


See the following thread from May, 2006 "InterSystems Cache and XFC..."

http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1028304

Jon

[Hoff]

Best way? Replace that hunk of rotating rust with an SSD.

(That's only partially a joke.)

[P Muralidhar Kini]

Hi Clark,

XFC caching can be disabled at various levels on a given Node

- Caching is disabled on Node
(VCC_FLAGS=0)

- Caching is disabled on Volume
(MOUNT/NOCACHE)

- Caching is disabled on file
(SET FILE/CACHING_ATTRIBUTE=NO_CACHING)

- Caching is disabled on IO
(using function modifier IO$M_NOVCACHE in the QIO call)

- IO size is greater than VCC_MAX_IO_SIZE


MOUNT/NOCACHE will disable both XFC (Data Cache) and XQP (Metadata
cache). If you Mount the disk with /NOCACHE, then you would not get
caching performance benefits from XQP also.

>> A typical database disk has about 1,000 files, two of which are CACHE.DAT
>> files or files NOT to be cached and the other 998 can be cached but are
>> not used frequently.
As you have said, there are only 2 files in the disk that are used on a frequent
basis and you dont want these files to be cached. Other 998 files are used less
frequently and you dont mind XFC caching those files. As per this requirement
you can enable XFC caching on the other 998 files to get some performance
improvement.

>> My question is, would there be any additional advantage to mounting the
>> disk cached (ie without /NOCACHE) and just restrict caching on the two
>> CACHE.DAT database files?
My suggestion would be to MOUNT the volume with /CACHE so that both XFC
and XQP caching are enabled on the volume. Set the 2 .DAT files to no
caching (i.e. SET FILE /CACHING_ATTRIBUTE=NO_CACHING)
The advantages would be
1) MOUNT/CACHE will enable both XQP and XFC caching on the entire disk.
You would get XQP related performance benefits.
If will also enable XFC caching on the disk (i.e. files other than .DAT)

2) XFC would start caching the other 998 files and hence you would get
performance benefits whenever those 998 files are accessed.
As you have mentioned they would not get accessed frequently and hence
you may get only small performance improvement.

>> Cache is already using its own reserved memory for caching these
>> databases. This is Intersystems rule for using XFC caching with Cache.
This is intresting, but is the reserved memory of the Cache database large
enough to cache the entire database ?

Regrds,
Murali

[P Muralidhar Kini]

Hi Clark,

>> Mount/NOCACHE turns off all XQP (FCP) caching
Yes.
So far the MOUNT command the "/CACHE" and "/NOCACHE" were always
applicable to both XFC and XQP. i.e. "/CACHE" will enable both XFC/XQP
and "/NOCACHE" will disable both XFC/XQP.
There was no way to selectively enable/disalbe XFC or XQP cache.

From OpenVMS V8.4(Yet to be released), a new feature would be provided,
which would allow the user to selectively enable/disalbe XFC cache
irrespective of the XQP cache.

The /CACHE qualifier of the MOUNT command would now accept two new
keywords namely "DATA" and "NODATA".
DATA -> would cause the XFC cache to get enabled.
NODATA -> would cause the XFC cache to get disabled.

Example:
To Disable XFC cache while still having the XQP cache enabled
$ MOUNT/CACHE=(NODATA)

Note that this feature is not there on current VMS version and would be
available only from OpenVMS V8.4 (yet to be released).

If you are planning to do a MOUNT/CACHE, then once the OpenVMS V8.4
version gets released it will help you mount the volume with XFC disabled
and XQP enabled.

Regrds,
Murali

[P Muralidhar Kini]

Hi Clark,

Lets talk about the advantage of other option i.e. doing a MOUNT/CACHE on the disk

1) More XFC cache for other volumes in the system
XFC cache is node specific. i.e. all the volumes in the system are going to use
the same cache memory. This would mean that all the volumes would in turn
have to compete for the cache memory. As far as XFC is concerned, when a file
gets accessed in the system, it would allocate cache memory for it.
This is when caching is enabled on the volume/file ofcourse.

Lets say your node N is having volumes V1, V2, V3 and V1 is the volume which
has your database files. Now if you disable caching on the volume V1, more
XFC cache memory would be available for volumes V2 and V3. This way, more
files in Volume V2 and V3 would get cached and hence would benefit from
caching. The overall performance of files accessed from volumes V2 and V3
will increase.

2) No manual work of setting .DAT to NOCACHING
If your database creates some more new files in the future then you have
remember to manually disable caching on the file using
SET FILE/CACHING_ATTRIBUTE=NO_CACHING command.

If volume is mounted with /NOCACHE, then you dont have to worry about
manually disabling caching for files in that volume.

3) XFC would ignore the file completely
Since the caching is disabled on the volume, XFC does not have to worry about
whether each file on that volume has to be cached or not. i.e. no decision
making for individual files when the volume on which the file resides, itself
is marked NOCACHE.

Hence,
First you need to check as to what is overall requirement of the system.
Based on this you can choose either "MOUNT/NOCACHE" of volume or
"SET FILE/CACHING_ATTRIBUTE=NO_CACHING" of individual files on the volume.

Regards,
Murali