OpenVMS i64 8.2-1, ACLs, and performance issue

[EWL]

We have a user that regularly creates 25 - 30 individual files and then appends then into 1 file. The files are created on a spool disk which has many many files of all different sizes and is very active. Disk is about 6gb and has plenty of space. We've defragged just in case. We have ACLs on the directory where the files are being created and the user has one of the 10 rights identifiers able to use the directory. The rights ID he's using is the 3rd in the search order. The ACLS are sequenced highest priv to lowest. If I add the rights id that is 1st in the list his time for this process is cut in half. I didn't think reading thru ACLs should take that long. Anyone have any thoughts?

[John Gillings]

EWL,

Please note that the ORDER of ACEs in an ACL can be critical to its correctness! Before changing anything, make sure you're not breaking the security intention.

I'd be amazed if you could measure any difference in performance by moving ACEs within an ACL. Although your logic is self evidently true (matching on the first entry in a sequential search, rather than the 3rd will obviously be quicker), the matching operation (in memory) will be a miniscule fraction ot the total processing cost of checking an ACL. The disk I/O required to read the ACL in the first place (which you will incur regardless of the position of the matching ACE) will be, by far, the dominant time expense.

There may be some performance issue if the ACL is long enough to require multiple I/Os to read. Use DUMP/HEADER/BLOCK=COUNT:0 to verify that your ACL fits in a single file header. If it overflows into a second header, that will affect your performance orders of magnitude more than ACE order (but even that increase may not be measurable in the overall performance of your process).

If it's a concern, look to reducing the size of the ACL by coalescing identifiers, or replacing individual user idetifiers with a general identifier (granted to each of the users).

If your process is so performance sensitive that ACE order really matters, I'd be looking for a way to get rid of the ACL altogether!

[Hein van den Heuvel]

>> If I add the rights id that is 1st in the list his time for this process is cut in half.

Wow, I would never expect that and suspect a 'side effect'. Frankly I am surprised you can even measure the time it takes to append a handfull of files.


That's not to say appending a file is invisible: Open output, open input, copy records, update headers and such, once per file.

But supposedly those files are recently created and still in cached, and how could the reading of an extra acl in the header make that much of a difference.

What are we talking about here 1/4 second --> 1/2 ? KB files or GB files?

Things to check
- Multiheader files? (DUMP/HEAD/BLOCK=COUNT=0)
- How is the device mounted? 'normal'?
- ACP Cache misconfigured? (Headers, Directory)
- SET WATCH FILE/CLA=ALL

If there non-target device activity as a result of the appends (rightslist? auditting?)

And uh... why fart around with 8.2? Go 8.3 already! Ya never know...

More details welcome!
Hein.

[DECxchange]

Hello,
In VMS system management classes I've taken in the past, they do say that ACLs will reduce performance. But I didn't think it would make that big of a difference. What kind of access time differences are you noticing? Seconds? Minutes? Fractions of a second? Have you done any average time measurements to see if this a REAL or a PERCEIVED difference?

I agree that you should go with VMS 8.3. It seems to correct work pertty darn good.

[Wim Van den Wyngaert]

Did a test on AS500/400 7.3.

Create 50 files with 1 line of text. Then append all files into a new file.

Did the test without acl, with 1 line acl, with 19 lines of acl in which the first gave access, with 19 lines in which the last line gave access. No difference in cpu time nor elapsed time.

Wim

[Willem Grooters]

Though it might seem to be a matter of ACL processing given the difference you mention, there might be other issues involved - given the size of it.

* Where does the delay occur: ion file creation, or on the appending stage?
* What is the size of the individual files?
* How are these files created : pre-allocated, or on-the-fly?
* How are the files appended to one file: DCL, program control? One-by-one of one command?
* what is the default extent size?
* How many files are contained in one directory - what size is the directory file?
* What is the disk's clustersize?

WG

[EWL]

Thanks for all the replies/questions/comments. Iâ ve tried to respond to most of them:
-8.3 is not an option right now. We are awaiting the app vendor to verify their code.
-Dump/Header/Block-Count:0 shows a single file header
-Canâ t reduce number of ACLs because of separation of duties and other security needs
-Disk is mounted shared, cluster, multiple I/O paths, with write-back caching enabled Cluster size is 10
-ACP_DIRCACHE=Current= 14988,default= 22; ACP_HDRCACHE=current=54464, default= 36
-One job stream creates the individual files and then appends. The DB is on a disk by itself, but all of these files are created on the same disk, same directory.
-Directory size is 179/2370
-Donâ t know how these files are created as this process is dependent on vendor code
-Using 3 different accounts for testing: 1 account using the 3rd rightsid in the list ran 14 files in 3 minutes; 2nd account that was the 4th in list ran 11 files in 2 ½ minutes; the account using the 1st rightsid in the list ran 12 files in 27 seconds. As you can see, the timing is definitely different

Just for fun, hereâ s the ACLs
Access Control List
(CREATOR,ACCESS=NONE)
(DEFAULT_PROTECTION,SYSTEM:RWED,OWNER:RWED,GROUP:,WORLD:)
(IDENTIFIER=A_RIGHTS,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL)
(IDENTIFIER=B_RIGHTS,ACCESS=READ+WRITE+EXECUTE)
(IDENTIFIER=C_RIGHTS,ACCESS=READ+WRITE+EXECUTE)
(IDENTIFIER=D_RIGHTS,ACCESS=READ+WRITE+EXECUTE)
(IDENTIFIER=E_RIGHTS,ACCESS=READ+WRITE+EXECUTE)
(IDENTIFIER=F_RIGHTS1,ACCESS=READ+WRITE+EXECUTE)
(IDENTIFIER=*,ACCESS=NONE)
(IDENTIFIER=A_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL)
(IDENTIFIER=B_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE)
(IDENTIFIER=C_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE)
(IDENTIFIER=D_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE)
(IDENTIFIER=E_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE)
(IDENTIFIER=F_RIGHTS,OPTIONS=DEFAULT,ACCESS=READ+WRITE+DELETE)
(IDENTIFIER=*,OPTIONS=DEFAULT,ACCESS=NONE)

[Wim Van den Wyngaert]

Do all accounts have the same quotas (in authorize)?

Wim

[EWL]

All accounts have the exact same quoatas. That's how we narrowed down the ACLs.

[Jan van den Ende]

EWL,

where the tests also tried in reverse order, ie. has it been ruled out that caching causes the difference?

Just trying to seek SOME cause here...

Proost.

Have one on me.

jpe