Disks statistics for one day

[shiva27]

Hi All,

Could you please help, how can i find the disks statistics for one day.
I tried "monitor disk/per" options to get this data but need to monitor whole day and get the screen shot every one hour and find the average values.

Is there any other efficent way to find this statistics from VMS(OpenVMS 8.3) commands.

[Steven Schweda]

Define "statistics".

Do "SHOW DEVICE /FULL device" every hour?

WRITE SYS$OUTPUT F$GETDVI( device, "OPCNT")

[Jon Pinkley]

What type of statistics are you looking for? Space utililation, number of i/o operations/period or how long operations are taking.

T4 will collect a lot of info and give you the opportunity to review it at a later point in time.

But first, you need to decide what you are really trying to measure. "A screen shot every one hour". does that mean an average of the last hour, or a 3 second sample once an hour. Those will give quite different results.

I had to look up what "monitor disk/per" does.

$ help monitor monitor disk /percent


MONITOR

MONITOR

DISK

/PERCENT

/PERCENT
/NOPERCENT (default)

Controls whether statistics are expressed as percent values in
display and summary outputs. The /PERCENT qualifier is applicable
only to the DISK, MODES, SCS, and STATES classes.

It is not clear to me what this is really measuring. Percent of what?

[John Gillings]

shiva27,
Try this:

$ PIPE MONITOR DISK/PER/INTERVAL=3600/ENDING="+1-00:00" >SYS$LOGIN:DISKSTATS.TXT

Or, in a batch job:

$ MONITOR DISK/PER/INTERVAL=3600/ENDING="+1-00:00"
$ EXIT

In both cases you can view the log file at any time. Pages are written at 1 hour intervals and finish 24 hours after starting. Averages are accumulated.

What I'm not sure about is the granularity. If it's just a single point sample every hour, that might not be what you're after.

T4 gathers disk statistics. Have you tried it?

[Hein van den Heuvel]

I like T4 for that kind of work.
Just google for OpenVMS and T4 and start reading.

A very very quick way to get per disk stats over a period of time is to use:

$ SHOW MEMORY /CACHE=VOLUME=*

If you want daily or hourly, then take a snapshot every so often and SUBTRACT the counters and times to get a rate.
If you opt to do this in DCL, then be sure to watch out not to exceed 32 bit integers.

SMOP.

Hope this helps some,
Hein.

[shiva27]

I need the disk statistics in terms of disk performance like detect IO bottleneck and other parameters(if available).

i want to give data which will show the disk performance (not sure which are parameter will decide).

Thanks in advance.

Shiva

[shiva27]

Is there any way to find the bottleneck for disk in system?

Kindly let me know if you want any more information to help me on this.Thanks

[Wim Van den Wyngaert]

If you have performance advisor, you can watch the number of IO's, the thruput in MB, the aueue length and the size of the IO's.

A bottleneck is difficult to detect because stuff like Sybase or backup will do the maximum of IO's on purpose. It also depends on the hardware used. You can e.g. monitor disks on a HSG80 itself.

Wim

[Hein van den Heuvel]

>> Is there any way to find the bottleneck for disk in system?

Yes. You start by droppping all assumptions (such as a disk being the bottleneck at all)
Then you observe the system and 'see' where it hurts. Some stuff is easy, like 100% CPU usage. But network and disk may be much more tricky due to parallel versus serial activities, asynchronous versus synchroneous, forced serialization using locking (RMS bucket locks)
For mere money I'll come over explain!

Seriously, you may want to try to work with a consultant who has done this before to get up to speed in this area.

I kinda read your initial topic question wrong forcussing on "disks statistics for one day.". That is way too coarse. It will hide any and all bottlenecks.
"get the screen shot every one hour and find the average values."

That is still too coarse, and too impractical.
Check out $HELP MONI MONI /RECORD

Now capture every second or every 10 seconds for the critical business hours and REPLAY with and interval of an hour as you please, and tighten the start, end and interval as you learn more.
Something like:

$ moni disk/inter=1/record=disks.tmp/nodisp/end=...

and later, at your leisure...

$ moni disk/inter=300/input=disks.tmp/view=5

But please just get T4 / TLVIZ ( or any other of the several ready made tools ) and let it do all of that for your!
by sure to check out CSVPNG which will allow you to automate extracting T4 and find min, max, averages... even tell you when it worst time was.

If you are seriously studying disk performance, be sure to ask the help from the XFC extension to ANALYZE/SYSTEM.
SDA> XFC SHOW VOLUME/SUM
and many more.

If for some reason you want to hand craft something you using snapshots as I suggested earlier, then find below some PERL code to get you going. It is a reduced version of a larger script i sometimes find I need.
The script is appended below. Sample output:

$ show memory/cach=volum=*/out=a.tmp
$ wait...
$ show memory/cach=volum=*/out=b.tmp
$ perl show_cache_by_volume.pl a.tmp b.tmp
Sample time is 35280 seconds

Disk Virtual reads Read hits Virtual writes
_$1$DGA1: 201230 5.7 197148 5.6 19213 0.5
_$1$DGA1: 201230 5.7 197148 5.6 19213 0.5
_EISNER$DKA0: 99661 2.8 89353 2.5 2138 0.1
_EISNER$DKA0: 99661 2.8 89353 2.5 2138 0.1
:

Good luck!
Hein van den Heuvel ( at gmail dot com )
HvdH Performance Consulting

$ type show_cache_by_volume.pl
use strict;
use warnings;
use Time::Local;

my (%old, %new, $name, $disk, $counter, $dif, @disks, @counters);

# -- get data

@counters=('Virtual reads', 'Read hits', 'Virtual writes');
$name = shift or die "Please provide an OLD file:";
&read_file (\%old, $name);
$name = shift or die "Please provide an NEW file:";
&read_file (\%new, $name);

# -- process and output

my $seconds = $new{Seconds} - $old{Seconds};
printf "Sample time is $seconds seconds\n\n";
printf "%15s %20s %20s %20s\n", 'Disk', @counters;
for $disk (sort @disks) {
my $line = sprintf "%15s", $disk;
for $counter (@counters) {
$dif = $new{$disk.$counter} - $old{$disk.$counter};
$line .= sprintf (" %9d %7.1f", $dif, $dif/$seconds);
}
print $line.qq(\n);
}

# --- done.

sub read_file {
my ($array, $name) = @_;
open FILE, "<$name" or die "Could not open file $name.\n$!";
while () {

if (/^\s+Sys.* on\s+(\d+)-(\w+)-(\d+)\s+(\d+):(\d+)/) { # Time stamp line?
my $n = index(" JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC", $2)/3;
$$array{Seconds} = timelocal(0,$5,$4,$1,$n-1,$3-1900);
}

if (/^(_\S+:)\s+/) { # Disk & volume line ?
$disk = $1;
push @disks, $disk;
}
next unless /\d$/; # Line ends in a number ?
for $counter (@counters) { # a number we care for ?
$$array{$disk.$counter} = $1 if /$counter\s+(\s\d+)/;
}
}
}

[Richard W Hunt]

Concur that a pre-made solution is WAY far preferred than rolling your own, but here is an overview of how to do that.

You can set up batch jobs to run the MONITOR DISK/ITEM=xxxx command (xxx=ALL, QUEUE, or OPERATION) and capture the output to a binary file. At the end of your period of performance, you can then replay the captured data to a file in a text format. Now comes the fun part.

Read that text file, DCL is as good as any other thing for prototyping. The top of each page tells you the node and time of sample. After that date/time line, the next nonblank line tells you what wes monitored on that page. Then you get the list of disks and the CUR, AVG, MIN, and MAX - in predictable columns! So extract the values and reformat to have device, timetag, and current value in another file.

Eventually you can get to a file with a date & time stamp, device name, parameter name, and observed value. Which you can then export to something like EXCEL or any other package that has built-in graphic abilities. From there, you are on your own as to what you wanted to see.

This is what I used to call the "poor man's SPM" (after the old System Performance Monitor software that used to be standard on older versions of VMS). Is it elegant? NO. Is it easy? Not that hard, just tedious. Does it work? Most of the time. If you take this approach, build in some robustness to handle unexpected things like broadcast messages or newmail messages that might show up in the output stream.

Good luck!

[Andy Bustamante]

>>> need the disk statistics in terms of disk performance like detect IO bottleneck and other parameters(if available).

You can pull the disk metrics but if you're torubleshooting a performance issue then Hein is spot on:

>>>.. start by droppping all assumptions (such as a disk being the bottleneck at all)
Then you observe the system and 'see' where it hurts

T4 is the utlity right now. http://h71000.www7.hp.com/openvms/products/t4/index.html One of the major data collectors is monitor. You can use monitor interactively as well. In addition the examples above you can record a session, since T4 is available, this falls into the category of silly monitor tricks, but can still be useful.

$ monitor
MONITOR> monitor all_class, disk/item=all /inter=xxx/nodisplay/record=monitor.dat
The default interval is 3 seconds, select something between 3 and 60 for your system:

$ pm == "monitor /input=monitor.dat/view=1"
View controls how long a screen shot is displayed. You can record 2 hours at 1 minute intervals and play back the display in 2 minutes for example.

$ pm disk /item=que
$ pm system

T4 uses the same method to collect monitor data, you can play back the the T4 monitor files as well for an interactive session.

You can also attend a performance class or schedule time with VMS consultant. There's more than one available here.

Andy

[Steve Reece_3]

As others have commented, the use of T4 to collect data is very successful in my experience and getting someone with lots of VMS and hardware experience to look at the results with you could probably pay dividends.

The important thing that you need to consider is what bottleneck are you really looking for? What's the underlying SYMPTOM that you're looking for the CAUSE of.
The symptom could be that the users are complaining of poor performance or that the overnight batch jobs that you're running are not completing in time for users to start work.
The cause of this could be as varied as:
- volume of data has increased so that the overnight job is running perfrectly well but just doesn't have enough time;
- the disk have become fragmented so that the system is spending more time spreading data over disks and reading data from disks so that disk performance appears poor;
- the system could be short of memory so is spending time paging and swapping instead of doing real work for users.

Near 100% CPU can be a good thing or a bad thing - it's good if it means that there's no other bottlenecks and you're getting through the work that you need to do. It's bad if the CPU is spending time managing stuff that shouldn't need managing (like paging, swapping, doing I/Os that don't really need to be done if you had effective caching.
Similarly, disk I/O can mean that it's a badly behaved application that's expecting too much from the disks or it could be that there's too little memory on the box to cache data effectively.
Invariably, when I've been asked to look at client systems that are running like a dog and the client reports that disk performance is poor, there's more than one aspect to it. Look at the thing in isolation and you'll miss other causes. Look at the overall situation and you can get back to a balanced system.

In short, T4 is going to be your friend. Look at all of the data that it can collect and get a helping hand too.

Steve