topic Re: TDC in Operating System - OpenVMS

TDC

justin hales — Wed, 17 Oct 2007 15:34:39 GMT

I was wondering if anyone had information about how the tdc collects disk I/O. After extracting the data collection file there are three fields for total bytes read. DSK.TDC_DSK_RD_IOCNT
DSK.TDC_DSK_RD_LOGIOCNT
DSK.TDC_DSK_RD_PHYIOCNT

Does anyone what the difference between these three are. I cant seem to be able to generate any realistic data for disk reads and writes i have generated reads and writes and not seen any data that will recognize this.

thanks

thanks

Re: TDC

John Gillings — Wed, 17 Oct 2007 17:15:27 GMT

justin,

Logical I/O is an I/O sent to the file system. It's specified in terms of file system structures. Physical I/O is directed at the physical disk. It's specified in terms of disk structure (obviously, a LOGIO is at some stage converted into a PHYIO, but I'd guess that PHYIOs generated as a result of a LOGIO are not counted in your statistics)

IOCNT count be the sum of LOGIO and PHYIO, or it could count all I/Os that are directed to the device, even if they're serviced by a cache somewhere along the way. (sorry, I don't know how TDC defines the fields, have you looked in the documentation?).

What do you mean by "I cant seem to be able to generate any realistic data for disk reads and writes". What does your data look like, and why do you think it's unrealistic?

Make sure you remember that there are numerous layers of software between your programs and the disk surface. Thus, if you have a program that reads the same record in a file numerous times, there are potentially several places where that data might be cached, so even if you generate 100 reads, you may only see a single read I/O (indeed, you should hope that is the case!). Similarly, if you write the same record, or several nearby records, even numerous times, the updates may coalesce into a single write I/O.

Disk I/O is comparatively VERY expensive, so software does it's best to only go out to disk when absolutely necessary (pop quiz... if we scale up the time it takes for a CPU to add 2 integers, calling it 1 second. On that time scale, how long does a disk read from a locally attached drive take?)

You can trust that the data collected is accurate. What's the real problem you're trying to solve?

Re: TDC

justin hales — Wed, 17 Oct 2007 17:50:40 GMT

Thanks for your response

What Im trying to do is to use the information from TDC to graph disk I/O. This will be very usefull to see what disks are used most and when.

What ive been doing is running the data collector on a node and generating disk reads and writes in ten minute intervals(its a test node all its running is the data collector). I cant see any increase in reads or writes in the data. Also when I generate the graphs there is an average of about 20MB being written when nothing is running on the node except for the data collector.

Re: TDC

John Gillings — Wed, 17 Oct 2007 18:06:19 GMT

justin,

>Also when I generate the graphs there is
>an average of about 20MB being written
>when nothing is running on the node except
>for the data collector.

What do you suppose the data collector does with the data it collects? Ask Mr Heinsenberg ;-)

>generating disk reads and writes in ten
>minute intervals(its a test node all its
>running is the data collector). I cant see
>any increase in reads or writes in the data

Try:

$ SHOW MEM/CACHE/FULL

XFC is a wonderous thing. If there's nothing else running, there's LOTS of available memory to cache stuff. I know of a system that ran for an entire week with the system disk popped out of the storage shelf. Everything was handled out of cache.

It's VERY difficult to design a realistic benchmark program for disk I/O.