- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2011 01:45 AM
10-17-2011 01:45 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
Ana,
how big is the Directory file (xxx.DIR) ? Deleting files in a big directory file is a very costly operation, as the .DIR file contents needs to be shuffled around. And all other users of that directory will see 'bad performance'.
Volker.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2011 02:29 AM
10-17-2011 02:29 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2011 03:04 AM
10-17-2011 03:04 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
The directory was 163000 files. I know that these are a lot of files and that the deletion operations are costly but, in our system, this was a special case (a cache directory of a web with many files). The problem was that, after realizing that the deletion operation was the cause of the performance degradation, we stopped the process that was running the deletion and the situation continued after some time, affecting the overall system .
Anyway, as we have, in general terms, good performance in our systems (it's a cluster behaving, basically, as a web server) and the problem described is difficult to evaluate and sporadic (except the disc with all the webs logs that has high I/O rate compared to the others) , I want to focus in increasing these parameters to better manage these special cases where the I/O rate of a disk increases sporadically.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2011 04:34 AM
10-17-2011 04:34 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
There are no particular ACP parameters around altering the performance of what OpenVMS now considers large directories, there are environment and application changes, and OpenVMS upgrades. Your OpenVMS version has the large-directory fixes that were implemented (in V7.2) and various other RMS and XQP performance tweaks), so that's not an option.
The usual approach for rapidly deleting what OpenVMS considers to be large numbers of files in these directories is the reverse-delete hack. Code your own delete, and delete the files in reverse alphanumeric order, or scrounge up one of the freeware tools that does this. The default forward-delete behavior of the DELETE command is pessimal, given the OpenVMS directory structures.
Put another way, you're not getting hammered so much on the ACP parameter settings here, you're getting hammered on fiundamental design decisions within the application, and how these decisions interact with fundamental design decisions within OpenVMS and its XQP processing, and particularly around the directory I/O processing, and around how VMS doesn't cache this directory data in memory; VMS really, really wants to write the data to disk. (When one of these directories gets rebuild from a mass-delete with cluster locking, other hosts can have delays writing to the directories, and I can see SCS locking Getting Busy with the directory locks; where you can, split your write processing and avoid using the shared storage, and start work to split up this directory.
Which in aggregate means a move to faster storage hardware (SSD will completely obliterate HDD performance) or a move to a RAM-based pseudo disk, or related I/O changes, or application design changes, or a move to a platform that better fits the needs of these sorts of application designs.
I'd also run a check for hardware and network errors, given the transient nature of this report. That probably isn't the case, but transient errors involving (for instance) heavy I/O activity can become unstable in the presence of lower-level and hardware errors.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2011 09:45 AM - edited 10-17-2011 09:48 AM
10-17-2011 09:45 AM - edited 10-17-2011 09:48 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
As noted, delete operations on a large directory can be expensive. I supported an application with similiar issues. Some of the options to mitigate this behavior, depending on your budget.
- RAMdisk. configure your report files on a virtual disk. A DEC-Ram license and memory can resolve all sorts of I/O issues.
- Use a search logical and spread the I/O to multiple disks and directories. On the fly updating of the disk sequence allows you to schedule delete operations on aged directories with minimal user impact
- Same option but spread directories on a single disk. Less cost, and less benefit, but keeps directory size small. I used a batch job to shuffle directories every 10 minutes. One site schedules daily deletes after midnight and kept writing report files to new directories.
- Review file allocations and bump the cluster size to something that allows most of these files to fit within one allocation. Faster create and delete time.
- If you have anything else generating I/O on the cache disk, move it.
The 2.x version of our web app skipped the temporary report file and and and just sent data to users. More database overhead if they wanted to go back to an old report but better performance since the disk bottleneck was relieved.
Don't expect tuning to have a significant impact. Complete this step and get caught up but be ready to move on to the next phase.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-18-2011 02:47 AM
10-18-2011 02:47 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
Thanks all.
I'll try to follow your recommendations.
I close this post.
Ana
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-25-2011 07:25 AM
10-25-2011 07:25 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
Hoff>> There are no particular ACP parameters around altering the performance of what OpenVMS now considers large directories
I beg to differ. yes there is: ACP_MAXREAD
That paramter defines the IO size in blocks which will be used during the directory shuffle.
The current default is 32
You could set to 64 for temporary releave, while working towards a real solution (cleaning out that directory).
(Set to 1 for the pre 7.2 behaviour :-)
fwiw,
Hein
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-25-2011 07:43 AM
10-25-2011 07:43 AM
Re: INFORMATION ABOUT ACP SYSGEN PARAMETERS
I'll bet the reverse-delete hack still wins.
- « Previous
-
- 1
- 2
- Next »