Appending T4 files which might include CD-ROM statistics

[Jeremy Begg]

Hi,

We're working on procedures to generate graphs from T4-collected data. We're producing monthly CPU, Memory and Disk graphs and also a yearly graph of each.

As part of the processing we've been using CSVPNG to combine T4 data files and we've made an annoying discovery: if the disk devices in two T4 files don't match exactly, CSVPNG excludes the "extra" disks from the output file (which is understandable) but it also changes the column name of the date column.

In this case the change was a CD-ROM was mounted for a couple of hours.

Aside from manually pre-processing the data before passing it to CSVPNG, is it possible to remove the references to the CD-ROM drive from the T4 data before CSVPNG sees it? (The device name is DQAO.)

We've also noticed another bug in CSVPNG: when appending multiple input files, it reads the entire first file and seems to assume that all subsequent files will have the same number of records.

Thanks,
Jeremy Begg

[Hein van den Heuvel]

Yeah, CSVPNG has its restrictions.
May we assume you have the most recent versions?

Not that it will help you much, but I also ended up pre-processing the raw CSV files for normalization but column wise, and data value wise (replacing non-sense peaks with a prior value, or adding a record to nail down a 0 and 100 for stable predictable percentage range.

At time I then still use CSVPNG for further work, or go all the way doing my own min/max/averages spitting out a file to be used by excel.

Good luck!
Hein.

[Jeremy Begg]

Hi Hein,

Yes we have the latest versions.

We've been working on Perl programs to manipulate the data, maybe that's another manipulation we have to do :-(

Thanks,
Jeremy Begg

[Hoff]

I'm not likely going to be able to assist specifically with CSVPNG and friends; those remediations are going to be owned by the fine folks of HP OpenVMS Engineering.

That written and presuming the release of this data to folks such as myself has been cleared, can you email a zip "-V" archive of the problematic MONITOR data files and CSV files that bracket the problems you're seeing here, and I'll add them to my "T4 gremlins collection"? (And thanks for any contributions here.)

[Jeremy Begg]

Sorry Hoff, I don't think that's going to be possible.