topic Re: CSVPNG with a Parameter File in Operating System - OpenVMS

CSVPNG with a Parameter File

B Novak — Thu, 03 Jun 2010 16:54:21 GMT

I'm running CSVPNG to "glue" some .CSV files together using a Parameter File called CALC.LIS. I then run CSVPNG again with a second Parameter File called CALC1.LIS to generate the graphs that show what I'm interested in seeing. The graphs I'm trying to generate to show the memory items look, well, wrong. I suspect that I'm not doing something correctly with one or more of the formulas. Does anyone have a Parameter File they would be willing to share that graph memory items? I've listed the contents of the CALC.LIS and CALC1.LIS files below. Thanks!

CALC.LIS
[CALC], Free List Size, /, $MEMORY, *, 100, Memory Percentage Free
[CALC], 100, -, Memory Percentage Free, Memory Utilization
cpu busy
Direct I?O Rate
Buffered I?O Rate
Mp Synch
Process Count
Locks Total
*ORA* CpuTim
*ORA* Dirio
*ORA* Bufio
*SRV* CpuTim
*SRV* Dirio
*SRV* Bufio

CALC1.LIS
[RENAME], [*]*PU Busy,&2PU Busy
[RENAME], [*]*ree List Size,&2ree List Size
[RENAME], [*]*pRate,Disk I&2 Request Rate
[RENAME], [*]*rocess Count, &2rocess Count
[RENAME], [*]*ompute, &2omputable Processes
[CALC], $CPUS, *, 100, Total CPU
[CALC], *CPU Busy, *, 100, /, Total CPU, Percent CPU Used
[CALC], $MEMORY, -, *Free List Size, Memory Utilization
[CALC], Memory Utilization, *, 100, /, $MEMORY, Percent Memory Used
Direct I?O Rate
Buffered I?O Rate
Mp Synch
Process Count
Locks Total
*ORA* CpuTim
*ORA* Dirio
*ORA* Bufio
*SRV* CpuTim
*SRV* Dirio
*SRV* Bufio
Memory Percentage Free
Memory Utilization
Percent Memory Used

Re: CSVPNG with a Parameter File

John Gillings — Mon, 07 Jun 2010 00:43:46 GMT

B,

Can you post your exact CSVPNG commands which use these files? How are you gluing them together?

It would also help to know what you're seeing which "looks wrong". What are you expecting to be different?

Re: CSVPNG with a Parameter File

B Novak — Mon, 07 Jun 2010 12:50:07 GMT

John,
I've attached the PDF that was last generated. Check out page 6. It shows numbers down the left side that make no sense in the Memory Utilitzation graph and the Percent Memory Used graph has numbers in the negative. Since it is only one file, there was "glueing" involved, however, I've included the 2 commands that I'm using below.

CSVPNG that does the glueing:

$ CSVPNG /PARAMETER=T4$SYS:CALC.LIS -
/nologs /directory=t4_data:[t4.T4$SYS.'nodename'_data.graphs] -
'csvfile_to_graph' /append='appended_filename'.csv

CSVPNG that generates the PDF file:

$ CSVPNG /PARAMETER=T4$SYS:CALC1.LIS-
/NOLOGS-
/lines-
/novertical-
/PDF='appended_filename' -
/directory=t4_data:[t4.T4$SYS.'nodename'_data.graphs] -
'appended_filename'.csv

Thanks,
Bob

Re: CSVPNG with a Parameter File

John Gillings — Mon, 07 Jun 2010 20:53:25 GMT

Bob,

I think I understand the issue now. Your "Memory Utilization" and "Percent Memory Used" statistics are derived from several calculation steps, the results of which are suspect. Have you checked the actual numbers to make sure they're as you expect?

I'd extract the columns you're working with from the original file, open them in a spreadsheet and go through the calculation steps you've coded. Use the spreadsheet graphing functions to compare with the CSVPNG results. Even better import the columns from the CSVPNG results files and analyze any differences directly.

Check your units (are "Free List Size" and $MEMORY both the same unit? blocks, bytes, MB etc...). I'd also check that your "*100"'s don't exceed signed 32 bit integer range. Sometimes you have to do the calculations in different orders, or add extra steps to avoid overflow, loss of precision or truncation.

I'm not sure what you expect this to do:

[CALC], $MEMORY, -, *Free List Size, Memory Utilization

Why "*Free"?

All that said, I'd need some convincing that expressing the size of the free list as a percentage of total memory is a useful thing to do. OpenVMS systems are designed to use as much memory as possible for real work, distributing any available memory to various places, including process working sets, cache and modified list. In practice I'd expect a "healthy" system to show a fairly static free list size somewhere between 5 and 10%. Changes in free list size don't tell you much about what's happening. Redistributions between the other places memory can be used are probably of more interest.

Re: CSVPNG with a Parameter File

B Novak — Tue, 08 Jun 2010 10:54:56 GMT

John,

Thanks for the "light bulb" moment. I had suspected that it might be what you suggest (i.e. number of different units, etc) and now I'm convinced. I'll just have to figure out which numbers are wrong using your suggestion of a spreadsheet. Thanks for the help.

Bob

Re: CSVPNG with a Parameter File

B Novak — Tue, 08 Jun 2010 10:57:11 GMT

This thread has been closed.