Need an advice regarding Soft required for Performance monitoring

[Yurgen]

Need an advice regarding Soft required for Performance monitoring of

cluster computers (users and processes).

Task description:
There is a cluster consisting of HP 2-processors Itanium2 computers under

OS Linux kernel 2.4.X.
Users run their parallel tasks on it.

I need to know the on-line picture: users; computers they use; computer

resources they use.
As for resources I need to know:
1)! average Integer and floating point operations per second executed by

processor
2) average speed at which data is being transmitted via eth1 eth0
3) memory consumption
4) hard disk IO

I will also store this data for each process to receive "ps id-picture".

The idea is to define - if the program currently started by user resembles

former one with the same name.

I need preferably open source tools to measure 1,2,3,4 - especially 1!
I like Perl and Python, so if you know tools working with processor

Itanium2 counters please inform me.

Requirements for tools: 1) low overhead, 2) ability to provide data about

processes in the system "on the fly" 3) gathering debugging info and

recompiling user programs - unacceptable, 4) if tools require kernel

recompilation or module insertion that is acceptable. 5) GUI for tools is

not necessary 6) must suit for Itanium 2


By now I have chosen the following among different tools and utilities:
1) For int and fp operations it seems like Oprofile is ok.
Namely I want to monitor counters for events L2_OPS_ISSUED with masks

int_load fp_load, retired_instructions and cpu_cycles
(as for events counters, I have taken them from Intel Itanium 2 Processor

Reference Manual)
2)+3)+4) I want to use following Perl modules
Linux::Statistics, and may be
Linux::Smaps
Proc::ProcessTable::Process

To collect data analyze and visualize I will also use Perl.


So my requests for you:
1. What do you think about tools I have chosen? Your comments are welcome.
2. If you have used tools which meet my requirements to retrieve 1,2,3,4;

or if you have performed same looking tasks as mine - I would appreciate

your advices very much.

[Ivan Ferreira]

You can get some of the information with ganglia by default. You can create custom monitors and add them to ganglia.

See:

ganglia.sourceforge.net/