Operating System - OpenVMS
1821801 Members
3189 Online
109637 Solutions
New Discussion юеВ

question about buffered IO

 
SAMI AHMAD
Regular Advisor

question about buffered IO

can someone tell me what exactly is buffered IO in openVMS and a high rate of Buffered IO implies what?
9 REPLIES 9
Volker Halle
Honored Contributor

Re: question about buffered IO

You still seem to be looking for explanations to a (perceived ?) performance problem, right ?

Consider to install and run T4 (requires OpenVMS V7.3-1 or higher), the HP OpenVMS free system performance data collector. It is a very low overhead data collector to obtain lots of system performance counters and easily display, analyze and correleate them. This data (simple .CSV text files) can also easily be shared and the powerful TLviz tool can be used to graph them.

http://h71000.www7.hp.com/openvms/products/t4/index.html

To answer your question:

In contrast to Direct I/O (I/O done directly between a user buffer and the device), Buffered I/O needs to allocate a buffer in nonpaged pool and copy the user data to/from this buffer. The device will then do the I/O into the buffer in nonpaged pool.

I/O operations that can take a 'long' time to complete are typically implemented as buffered I/O. Examples are: terminal I/O, network I/O, file system (XQP) operations.

Volker.
Robert Gezelter
Honored Contributor

Re: question about buffered IO

Sami,

In addition to Volker's response, to which I heartily agree, there is a lot of context that is difficult to communicate in an ITRC Forum.

Outside of context, what is "high rate"? On some older systems, a hundred IOs/second was a "high" rate. On today's systems, where it is not uncommon to have many processes, a thousand IOs/second is not outrageous across the entire system. For that matter, with caching and memory-resident disks, certain rates are quite reasonable.

At the other end of the range, I remember one memorable program (admittedly this was nearly 30 years ago). This program wanted wrote tens (probably hundreds) of thousands of characters to the terminal, one character at a time using QIO (e.g., it was doing thousands upon thousands of buffered IO operations to the terminal). That activity brought a VAX-11/780 to a (pardon the pun) virtual halt. In that case the user was "persuaded" to modify their program.

Doing a simple DIRECTORY/FULL/OUTPUT=NL: SYS$SYSDEVICE:[*...] is illustrative (albeit for direct IOs). Even a simple AlphaStation 200 4/233 (certainly not the fastest system out there) can hit approximately 500 Direct IO/second with an RZ-28M internal drive. Buffered IOs, which can be any number of things, can accumulate faster.

Is this a sustained rate? What is the hardware? What are the details of the workload? What is the workload doing? All of these are critical to understanding the difference between a correctly tuned, but heavily worked system and a system that is being used efficiently.

Some of these questions are easily discussed in an online forum (e.g., type of CPU, OS release, what is the actual rate). Other questions (e.g., the details of the workload) are harder if not impossible to assess effectively in a forum such as this.

As has been suggested previously in other threads, resolving performance questions often require a great deal of background knowledge, or very experienced assistance [Disclosure: our firm does provide services in this area].

I was once called in and informed that a VAX system was promised to be fast, but took literally minutes to simply login. To say that the manager involved was upset is very much an understatement. When I arrived, and he demonstrated it for me, I was mystified at first. I could not think what would cause such a long login time on what appeared to be an idle machine. He had even volunteered that they had tried adding memory, but it had not helped performance.

Then a thought struck me. I went to a console, and used AUTHORIZE to display one of the accounts. Sure enough, the Working Set limits were set to approximately 200 pages (100Kbytes on a VAX). I reset the limit to a more reasonable number, and "performance" increased by a very large factor (1000x).

In performance, almost every question depends dramatically on the details. Sometimes, a situation is familiar enough that others have been down the same path. Other times, it is far more a question of understanding exactly what is actually happening and how it can be improved.

- Bob Gezelter, http://www.rlgsc.com
Ian Miller.
Honored Contributor

Re: question about buffered IO

Do read the OpenVMS Performance Manual

http://h71000.www7.hp.com/doc/73final/6491/6491pro.html

it explains many things
____________________
Purely Personal Opinion
Hein van den Heuvel
Honored Contributor

Re: question about buffered IO

And just to make sure you understand, most of the time a high rate buffered IO rate is desirable, as it indicates more work is being done!

Similar to a high direct-io (disk) write rate. More of those suggests that more actual work is being done. Of course that assumes that the application is tuned to do as few (write) IOs as possible for a unit of work.

Hein.

John Gillings
Honored Contributor

Re: question about buffered IO

Theoretical answer to the question as asked, but not necessarily helpful to the underlying issue (if any).

In very rough terms, an I/O operation can be performed either directly to/from the buffer provided by the requestor (DIO for "Direct I/O"), or can be transferred to a buffer in system space prior to starting the operation ("BIO or Buffered I/O").

In even rougher terms, disk I/Os are often DIOs, network and terminal I/Os are often BIOs. The choice depends on many things, including the speed of the target device and the nature of the operation.

In practice it isn't simple. Whether a particular operation is counted as a BIO or DIO is up to the writer of the driver. It's not always clear cut. For example, if a request is satisfied from a cache, should it be counted at all? There's also sometimes an alignment issue - for example, some devices require an aligned buffer, so if the provided buffer is unaligned, it may be necessary to copy the buffer to an aligned system buffer in order to proceed. Should the driver count that as a BIO or a DIO?

As others have said, "high rate" doesn't mean anything. What's high?

If a process is generating continuous BIOs it's probably reading or writing to a terminal or a network device. Most likely doing what it was asked to do!
A crucible of informative mistakes
Phil.Howell
Honored Contributor

Re: question about buffered IO

run Monitor sys and Monitor io for a while
Oracle tends to do a lot of buffered i/o
Phil
MarkOfAus
Valued Contributor

Re: question about buffered IO

Can I add my 2c worth?

It has always surprised me how the Availability Manager determines its "high buffered I/O rate" messages? Unless it has some built in benchmark for each system, surely this is what others are saying here is an arbitrary notion?

Unless of course the message means "buffered I/O is higher than zero so therefore could potentially have an impact on the system."

Regards
Mark
Volker Halle
Honored Contributor

Re: question about buffered IO

Mark,

you can customize the thresholds in Availability Manager to your liking.

Volker.
SAMI AHMAD
Regular Advisor

Re: question about buffered IO

Thank you all for the valuable insight into this topic.