To measure bytes in/out of a File system.

[Darrell Tschakert]

Hi,
We are running Oracle on a number of 9000 K and L class servers running HP-UX 11.00 and 11.11. We are interested in knowing how many bytes of data are transfered in and out of certain file systems over a given period of time. We would like to measure the amount of data moved during one minute, one hour, and one day. If we cannot get these numbers for file systems, then perhaps the same data ported on LUN's would do. I realize that I can use glance or sar to get rates, but we would like to get absolute numbers instead. Can anyone give me any suggestions on how to go about gathering this information?

Also, can someone tell me how to measure the number of bytes going in and out of a particular file. I realize that I can watch the file offset of of a given file as bytes are writen/read to/from it. Again, I would like to know the absulute number of bytes.

Thanks,

Darrell

[Tim D Fulford · Accepted solution]

one Word MeasureWare...

That will measure the kB/s rate (in & out) for any filesystem. I would reccomend you use the newest version certainly C.03.70.

I had a similar question for Informix which uses raw deviceses. The only problem I had was the answers were 6 times larger than they should have been. by this I mean adding together all the stats for each lvol gave an answer 6x the global stats measure. I think this is because we use kilobyte stripes across 12 LUNs. I would imagine filesystem stats would be better.

Regards

Tim

[Thomas Schler_1]

Darrel,

I'd suggest to use sar to get the rates. Let run a series of sar processes, e.g. every 5 sec one sar command. Write a script that uses the sar output to get the desired absolute values. These values may not be exact, but good enough to get a good estimation on the dimensions of byte transfers.

[Michael Steele_2]

Two comments here:

a) What you start and end up with is easy enough to measure and from this you can determine the difference or changes.

b) The rate of bytes into and out of the file system will be different from a) since data is copied into RAM, updated and then over-written back into the file system.

Regarding a), Start with a cron and 'df -k'. The '-k' option reports in 1024 byte blocks. Run the cron hourly, as you indicated. One minute may be too much load. You'll be filling up your work space pretty quickly so clean out quickly.

Also regarding a), MeasureWare will be good down to 5 minute intervals, the default, but also fills up fast. Refer to 'extract' command and the different log and template files, 'logapp', 'logglob', 'logdev'. 'logapp' is probably what you want, its the application log. 'logglob' = global. 'logdev' = devices.

Get the MeasureWare Dictionaries.

[Darrell Tschakert]

Tim - I should have said "We have no money for software". We are Federal Gov. and will not be getting this years money for another month or two. My boss wants these numbers ASAP. Still, Measureware seems like something that we could use in the future and I intend to look into it a little more closely. The price is right - I saw $120 from one vendor.

Thomas - As I mentioned in my posting, I would like to avoid sar or glance. If I have to use sar and calculate rate into volume, I will do so, but I am looking for something a little more friendly and too the point.

Michael - I don't think that df -k will give me what i want. Many of our file systems stay pretty much the same size over time but we know that GigaBytes of data go in and out of these file systems daily. I should have been a little more clear on this - I am not concerned with the change in size of the file systems so much as the actual number of bytes that go into the file systems and are copied out of the file systems.

Anyone - I was hoping that a utility supplied with the HP-UX OS could handle the job without too much data manipulation. It would appear that I am not going to get what I was hoping for.

Thanks,
darrell Tschakert

[Michael Schulte zur Sur]

Hi Darrell,

if Oracle is the main item, you are concerned about, then you could use the Oracle inside measure scripts. utlbstat.sql to start end utlbestat.sql to end a measurement. You get a report, which tells you how many reads/writes you got per datafile.

greetings,

Michael

[Darrell Tschakert]

Michael Surl - I will speak to the Oracle people on Monday to see if they can do this for me. Thanks.
Darrell T.

[George E. Holland_1]

Darrell
This will take a little work but you might try a freeware program called MRTG. This uses snmp data to graph lan data in and out of the nic. The only draw back is that it will show all data transfers and not just the Oracle.

[Darrell Tschakert]

George Holland - Thank you for your reply. Although MRTG seems like an interesting program, I will not be able to use it in this case since the file systems that I am refering to are local. From what i read about MRTG, it watches for network traffic. We are trying to gather info about data going in/out of certain file systems so that we can get some estimate of whether or not we can mirror parts of the file systems with remote or NFS file systems. We do not know if our network data pipe is large enough to handle the data in/out of the file systems if they should be put on our WAN. However, if we ever do put these file systems on the Network, I think that MRTG would be a good candidate for monitoring the remote file systems. So, I am glad that you brought MRTG to my attention and I will certainly take a closer look at it when I get back to work on Monday.
Thanks,
Darrell T.

[George E. Holland_1]

Darrell
MRTG will look at any thing that has a mib and can be used on local as well as remote systems. The docs give information on how to configure the program to look at other metrics.

G.