Operating System - OpenVMS
1748129 Members
3619 Online
108758 Solutions
New Discussion юеВ

Monitoring internal growth of an RMS index file

 
SOLVED
Go to solution
roose
Regular Advisor

Monitoring internal growth of an RMS index file

Hi,

We have RMS indexed application files that we would like to monitor the size growth. During an activity last year, the previous application owner did optimization on the files and re-created these files with larger allocation size to cater for future growth. On the system side, we just use to monitor the allocated size growth and we just normally react when we see an expansion on the file's allocated size. Now, we would like to monitor the internal growth of the file as well.

With this, what I mean to say is that when we do a dir/size=all, we see the used and allocated file sizes:

S1A02$ dir/size=all tsbt.ism
Directory DISK$P1_SS1_DB:[SPROMS1.FIL.DATA]
TSBT.ISM;1 88653/218304
Total of 1 file, 88653/218304 blocks.
S1A02$

However, we have some files that the used and allocated values are almost the same, but we are sure that the internal size of the file is still around 50% of the allocated size. We try to do an anal/rms on this files but we are at a loss how to find the internal total size of the data.

As an example, the TSBST.ISM file I posted above have an anal/rms result in the attached file. We are not able to find the value of 88653 (used) in the anal/rms result.

How should we go about this?
7 REPLIES 7
Jan van den Ende
Honored Contributor

Re: Monitoring internal growth of an RMS index file

roose,

NO exact bookkeeping value, but it IS in there

>>>
AREA DESCRIPTOR #0 (VBN 3, offset %X'0000')

Bucket Size: 5
Reclaimed Bucket VBN: 0
Current Extent Start: 1, Blocks: 217745, Used: 179663, Next: 179664
Default Extend Quantity: 11445
Total Allocation: 217745
<<<
Look at the
<<<
Blocks: 217745, Used: 179663
>>>

That is the available vs used DATA space.

hth

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
roose
Regular Advisor

Re: Monitoring internal growth of an RMS index file

Jan,

Which one is more reliable then, the dir/size=all or the anal/rms values?
labadie_1
Honored Contributor

Re: Monitoring internal growth of an RMS index file

The number of records is, IMHO, important.

If your file is on a ODS-5 device, you may get the number of records from f$file(filename,"file_length_hint")

You can always get the number of records from

$ search file ""/noout/stat
and look at
Records matched:

or

$ convert/share file nl:/stat
and look at
Total Records Processed:
Jan van den Ende
Honored Contributor

Re: Monitoring internal growth of an RMS index file

roose,

>>>
Which one is more reliable then, the dir/size=all or the anal/rms values?
<<<

Well, the DIR/FULL output gives the ALLOCATED space, and the value of the END-OF-FILE pointer.

But --indexed-- files (potentially) have LOTS of chunks of unused (not-yet-used) space spread out throughout the file. NO way for DIR to take that into account.
ANAL/RMS contains A LOT more info, but reads less easily.

You may have noted that dir ALLOCATED value in DIR is significantly more than the block count for the DATA area from ANAL/RMS.
That is because in indexed file also holds the indexes (and some housekeeping metadata). That also uses up DISK space, and therefore is also counted by DIR/SIZE=ALLO, but NOT in the DATA AREA stats.

But if you look at
>>>
Blocks: 217745, Used: 179663
<<<
you have a fair estimate of actual usage percentage.
(some space inevitably is unusable due to actual fitting, that is why "fair" as as far as I dare give you.

-- and that is about as far as my knowledge goes, if you want more (specific) details, then just ask, and I am sure Hein will be glad to go any length and depth anyone cares to follow.

hth

Proost.

Have one on me.

jpe
Don't rust yours pelled jacker to fine doll missed aches.
Hein van den Heuvel
Honored Contributor
Solution

Re: Monitoring internal growth of an RMS index file

Hi there,

The 'Used' allocation, or EOF for an indexed file is meaningless. You should ignore it.

As Jan correctly writes, the used vs allocated sizes are available for each AREA in the file where the main data typically, but not necessarily is places in AREA 0.

Most AREA information is available through programs in the XABALL = AREA CONTROL BLOCK.
See RMS Reference manual. However, the USED field is not, and you need ANALYZE/RMS to read that from the file. (I've been trying to get that changed. No luck).

ANALLYZE/RMS unfortunately does not allow shared access (I've been trying to get that changed for years, no luck). For a tiny file as per example, just do a BACK/IGNORE=INTERLOCK and analyze the copy. For larger files you'd need a hack to read the control blocks from the first few blocks (definitions are in SYS$LIBRARY:LIB.MLB).

But what problem are you trying to solve? I suspect you want to know when to re-convert the file for maintenance correct? The used vs allocated size is only a minor indicator for that. You could just stash away the original allocation (it is in the FDL) and watch for growth.

In order of importance I would use the following indicators to trigger a convert

1) Resources consumed / response time. When a file is starting to use significantly more resources (CPU, LOCKS, IO) then expected. That's the time to convert it. You would need usage TREND information for this.

2) Index depth .. for all keys. Way back when I made a program 'show_roots' for this which is in the RMS documentation. I attached a more extended, more recent version for this little tool here.

3) Bucket splits / RRV generation.

4) Deleted record chains

5) Extents / fragmentation.

6) Age... each indexed file should be converted once a year whether it needs it or not!

For my customers I have tools and methods help with this. You can find early versions of my main tool "RMS_TUNE_CHECK" on the freeware CD & web.

Hope this helps some,
Regards,

Hein van den Heuvel ( at gmail dot com )
HvdH Performance Consulting
Hein van den Heuvel
Honored Contributor

Re: Monitoring internal growth of an RMS index file

btw... in an ideal world the remaining space in an area would be a pretty darn good indicator as to when to convert and/or tune a file. However, in years (decades!) of providing tuning RMS tuning advice to customers I find very few who manage to manage this closely enough.

As replied earlier, the number of records CAN be an other good indicator, but there are prenly of files out there for which records are purged on a regular (immedait after usage, daily, monthly, yearly) basis.
Specifically 'work or queue' may typically hold no, or very few, record yet need frequent converts and sophisticated tuning.

Gettting it roughly right solve 90% of the problem. Getting a procedure in place to do regular converts is 90% of getting it right.

Trending, IMHO is the best way to know when to convert.
I like to use SHOW MEM/CACHE=TOPQIO derived data to get a 'hotfile' list and see how many reads & write per business day a given file is requiring. If that creaps up more than the business would suggest, then it is time to convert and/or tune.

Cheers,
Hein.
roose
Regular Advisor

Re: Monitoring internal growth of an RMS index file

Folks,

As always, thanks for your great advice! We'll try to verify here what we should be doing based on these advices.