- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: High water marking
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 03:23 AM
05-27-2004 03:23 AM
Then I create a file of 400.000 blocks on this disk. The command takes about 30 seconds. During this time, f$search hangs and f$file too.
It seems that the disk is locked (but not for dir).
Why ?
Is it possible that a database server (sybase) crashes because of it (e.g. too much outstanding IO for the given quotas) ?
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 03:27 AM
05-27-2004 03:27 AM
Re: High water marking
re sybase - depends on how it handles quota failures - it should cope but...
Purely Personal Opinion
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 03:42 AM
05-27-2004 03:42 AM
Re: High water marking
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 03:55 AM
05-27-2004 03:55 AM
Re: High water marking
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 03:58 AM
05-27-2004 03:58 AM
Re: High water marking
The basic premiss is easy. Never allow a process to read a block from a fiel that was not writen too that file earlier. The protetcs against 'scavenging'. That is, it protects against soem job just askgn for a bunch of block and then reads those to see if there is something 'interesting' left behind from a prior usage of that block (while it was part of a different, now deleted or truncated file).
Sounds easy, but ia hard to do right in a full sharing environment. The system has optimizations to avoid writes as much as possibly, notably when it detects 'sequential only access'. However, if you just allocate a bunch of blocks for a non-sequential file, or write a block for out into the file, the XQP will erase' all intermediate blocks... while holding a volume lock! (This can probably be fixed using a lower level lock, but that is not how it is done today). So writes to a file that is already open can go on, but any volume operation will stall.
You have to understand the application/file-usage to decide whether high-watermarkign is really needed, and what the expected costs are. You are not the first one to be bitten by this (myself, I get a painfull reminder once every 3 years or so: Hmmm.. is the system down? no? then why is my 'DIR' hanging? Oh duh... HWM!).
Groetjes,
Hein
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 04:04 AM
05-27-2004 04:04 AM
Re: High water marking
he said he could create a new file, but not delete it - isn't the former an operation on the volume, too?
Wim,
did you just create an empty file or did you write data, too?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 05:49 AM
05-27-2004 05:49 AM
Re: High water marking
as I have come to understand things, it is working fine.
Maybe a monitor IO/item=que for the disk will illustrate.
Creating a file is OK. (should that be initially so as well, while the SET VOL/HIGH is still active? Or is this assuming the data present before issuing the command was and still is not that important? Or IS this what CREATE is doing: writing VERY much meaningless data? ) But then, the file gets deleted, and the full area of the disk that WAS the file, now first has to be (multiply?) written, so as to release only 'truely erased' disk blocks.
So, for a big file to delete, your 'hang' is simply waiting for an IO to complete, that is somewhere at the back of a BIG que of disk IO entries...
correct me if I'm wrong, but this is what I remember of the info way back when, when highwatermarking was introduced (somewhere V5-ish ? )
Jan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 06:15 AM
05-27-2004 06:15 AM
Re: High water marking
well the first question obviously is, do you have an applicaiton requirement to pre-zero
the diskblocks with Highwatermarking? If not,
having it enabled will generally incurr a penalty on random access files.
Greetings, Martin
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 07:10 AM
05-27-2004 07:10 AM
Re: High water marking
It seems that the disk is locked (but not for dir).
Why ?"
The short answer is the highwater erase, f$search, f$file, etc. are, by design and many good reasons, single threaded through the FCP (File Control Primitive). For the FCP to erase 400,000 blocks takes roughly the time you mentioned. The detailed answer has already been given.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 06:01 PM
05-27-2004 06:01 PM
Re: High water marking
What are exactly the read/write locks that are taken ? A database system is much simplier to understand than RMS.
Second, I find it a bad implementation if applications are stalled because of it. Suppose you have a real time application that need to close a valve but can't do the job because some DBA is creating a big database file (would block the disk for several minutes or even hours).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 06:59 PM
05-27-2004 06:59 PM
Re: High water marking
Highwatermarking IS a very-high-security feature, to be used IF YOU NEED IT, explicitly AT THE COST of performance.
I guess yhis implies that highwatermarking and real-time functionality are mutually exclusive.
The way out of this dilemma is quite simple:
have the Sytem Manager make sure that your Highwatermark-requiring data are on another physical drive then your real-time files.
Jan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-27-2004 07:12 PM
05-27-2004 07:12 PM
Re: High water marking
I disagree. Performance impact yes, blocking activities no. Simply a bad implementation. Even if there are workarounds.
I don't use it (I put it off where I see it). But used it for testing f$esrach.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-01-2004 07:19 AM
06-01-2004 07:19 AM
Re: High water marking
These type of delays are typical of High Water Marking (HWM). If your applications are too sensitive to these delays then you have two options. One, turn off HWM, or, two, control your allocations, either by delaying large allocations to the off hours or breaking up the allocations into small chunks.
HTH
Marty
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-01-2004 06:31 PM
06-01-2004 06:31 PM
Re: High water marking
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2004 05:43 PM
06-03-2004 05:43 PM
Re: High water marking
This may not be OpenVMS at all. For some combinations of controller and disk drive, the HWM ERASE function is done in the drive itself. Just an I/O ERASE operation, rather than a series of writes to individual blocks, then wait for it to return (or, to be more precise, one I/O per allocated extent). So, all the stuff you CAN do may be the result of caching, or operations that the drive and/or controller can handle in parallel. Anything else just has to wait.
Bottom line, this IS the "performance impact" that the documentation talks about. You can argue about other ways of doing things, but they're all tradeoffs. That's what engineering is all about - picking the least worst choice.
With security, there are always nasty things to think about. For example, suppose we didn't lock the disk and zeroed the allocated blocks in small chunks, obviously that would take MUCH longer to complete. Apart from the cumulative cost of multiple I/O's, we'd have to write actual blocks, so data would need to be buffered and sent over the wires to the disk.
Worse, maybe it would be possible to request the CREATE asynchronously and access the blocks from a parallel thread, ahead of the ERASE fence, thus defeating HWM?
The engineers who designed this weren't complete morons, so if they've taken the fairly drastic step of holding a lock for a long time, it definitely wasn't without good reason. If you really want to know the answer, log a case with your local customer support centre and have it escalated into engineering. They will either say "of course! how stupid" and fix it, or they may explain why it has to be that way to guarantee security, or they may say "it has to be that way, but we're not going to tell you why becuase it involves security".
This is the burden of being OpenVMS. Other operating systems can take short cuts, possibly leaving security or reliability holes around the place, but we have to keep everything secure and failsafe, sometimes at the cost of performance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2004 06:29 PM
06-03-2004 06:29 PM
Re: High water marking
Whatever the reason, no disk should be locked during a longer interval.
High water marking may not be used in financial market systems, real time applications and high availability systems, unless EVERYBODY is aware of the problem.
The performance impact that I found in the doc only talks about the extra IO for erasing those extents, not about the disk lock.
Wim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2004 08:05 PM
06-03-2004 08:05 PM
Re: High water marking
hmmm... beg to differ.
Highwater marking may be MOST important for those financial market systems.
The impact is generally NOT found in day to day usage. For a 'normal' sequential file write no extra IO is needed. Hey, that's why the HWM is there! It's when you create the possibility to access beyond HWM that the erase is needed.
Any other real-time/oltp task on the disk with the files already open can just keep on reading/writing from those files there is no data IO lock (that concept does not even exist!).
Wim, is this question for the system you asked about for the caching? There you indicated sequential files were used a lot and indexed files where used little? Do you have a self-designed / supported data retrieval system to deal with? If so then indeed you will need to learn to deal with the 'fine print' like using the 'SQO' bit.
fwiw,
Hein.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2004 08:55 PM
06-03-2004 08:55 PM
Re: High water marking
Any create/fdl (or extent) with a substantial file, any database file creation (sybase). This is very common on our site.
>Any other real-time/oltp task on the disk >with the files already open can just keep >on reading/writing from those files
All jobs creating files, re-opening files are blocked. Also very common. I tested with dcl read/key (file was open before create/fdl was started) and found that the procedure was still blocked during 11 seconds !!!
>Wim, is this question for the system you >asked about for the caching? There you >indicated sequential files were used a lot >and indexed files where used little? Do you >have a self-designed / supported data >retrieval system to deal with? If so then ?>indeed you will need to learn to deal with >the 'fine print' like using the 'SQO' bit.
Nope. Just trying to understand things without a cause.
If you try to buy some shares on the stock exchange, close valves of reffineries or other such stuff, you can miss those suspends.
Wim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2004 09:11 PM
06-03-2004 09:11 PM
Re: High water marking
I must speculate that the slow READ/KEY was caused by the 2 to 5 IOS required to do so each had to wait in a large queue for the disk itself. ( 2 to 5 = index root, index lower-level-if-needed, data bucket, RRV-bucket-if needed)
Hein.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-04-2004 06:25 AM
06-04-2004 06:25 AM
Re: High water marking
If I may . . .
> I disagree. Performance impact yes,
> blocking activities no. Simply a bad
> implementation. Even if there are
> workarounds.
It seems like you are not accepting the reasons for blocking. We can debate the documentation's lack of clarity another time (acutally I agree with you).
Bottom line is if there is another way to do the erase without blocking other IO on the volume, how would you do it?
The goal of the erase is to prevent disk scavenging (there are other goals, but that's the one I am concerned with). What does a given application need? Does the application have to prevent disk scavenging? If one is worried about disk scavenging, then you have to erase before using the disk blocks. No other choice.
If you have another choice, how would you do it? This is not, as you claim, a "bad implementation" nor is it a design flaw or poor practice. Again, to guarantee security with the goal of disk scavenging, what would you do otherwise?
If you allow another IO to occur during the erase, you can bypass the attempted erase function, read the blocks that have not been erased (theoretically *all* of the blocks) and scavenge the data. The only kind of IO I could consider is a write IO. But the read IO's have to be blocked so to be safe, block all IO.
If this is a bad implementation, how would you guarantee security if you allow even one IO during the erase?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-08-2004 06:00 PM
06-08-2004 06:00 PM
Re: High water marking
This seems to be the same discussion as in databases. First, only table locking was implemented. Next (on certain systems like HP3000 Image) there was chain locking. And finaly there was row locking.
It seems to me that VMS must block all IO's for the part being erased, not the whole directory or disk.
Sorry, but nobody can convince me to block all IO's.
Hein : it must have been a long queue because 11 seconds is a long time.
Wim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-10-2004 10:00 AM
06-10-2004 10:00 AM
SolutionI've been looking into this. The more I find, the more questions I have. I have found an inconsistent blocking of IO when doing a file create vs. /erase; sometimes the directory, sometimes the whole disk. This should not happen so I have logged a call with HP and the support folks are scratching their heads. ;-)
When I have more, I'll post it.
This impacts our usage of highwater marking and delete/erase and the lexical functions we use as well. So I am glad you raised this question!
john
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-10-2004 05:44 PM
06-10-2004 05:44 PM