Benchmark Software

[Geoff Wild]

We have a RX6600 in for testing from HP.
I've run a java based test ussing SCIMARK.
I would like to know if anyone out there has used any benchmarking software (free) that will run on HP-UX and Solaris.

The Oracle DBA's will be running their own benchmarks, but I want to run some OS ones myself.

Thanks...Geoff

[Olivier Masse]

I did the same exercise last year, I needed to compare raw CPU performance between HP-UX/IA with OpenVMS/Alpha. I didn't find any free cross-platform benchmark that was free and easy to use.

I ended up writing a very simple C program that crunches data and ran it with timex a few times, while the VMS admin did the equivalent on his side. I wanted something closer to the processor than perl to have better results. That gave us a "ball park" figure, nothing scientific but at least some numbers to hand out.

I attached it here, it is very simple but it could be useful. Compile it with the native compiler if possible with the appropriate options instead of gcc to be sure to have the most optimization possible for your CPU. I can give you an IA64 copy compiled with aCC -O if you don't have the compiler.

[Ivan Ferreira]

It depends of what you want to test, there are different tools, for example, iozone, iometer, swingbench, and see also http://www.opensourcetesting.org/performance.php

[Steven E. Protter]

Shalom,

I always try and run our longest most time consuming batch job and compare the times to real production, with large test db.

That works well for me, real world, not like the tools.

SEP

[rick jones]

Be _very_ careful with small benchmarks. They tend to fit into small caches when "real" applications don't.

A bunch of possibly useful benchmarks already mentioned, I of course must add netperf to the list - http://www.netperf.org/

Caliper - http://www.hp.com/go/caliper might not be a bad tool to have on hand while you are benchmarking your rx6600.

[Geoff Wild]

Oliver - compiles fine on 11.31 on ia64, but I also want to compare against parisc, when I try to compile I get:

# /opt/aCC/bin/aCC -O raw-cpu-perf.c
Error (future) 229: "raw-cpu-perf.c", line 27 # "Ambiguous overloaded function call; a function
match was not found that was strictly best for ALL arguments. Two functions that matched best
for some arguments (but not all) were "long double log(long double)"
["/opt/aCC/include/cmath", line 279] and "float log(float)" ["/opt/aCC/include/cmath", line
91]." Choosing "long double log(long double)" ["/opt/aCC/include/cmath", line 279] for
resolving ambiguity.
result=random() * random() * sqrt(log(random())) - random() + random();
^^^
Error 203: "raw-cpu-perf.c", line 37 # Cannot assign 'char *' with 'void *'.
ptr=calloc(100, 1048576);
^^^^^^^^^^^^^^^^^^^^
Warning: 1 future errors were detected and ignored. Add a '+p' option to detect and fix them before they become fatal errors in a future release. Behavior of this ill-formed program is not guaranteed to match that of a well-formed program

Also, need to compile on Solaris 10 - just waiting for access to a box with ansi c on it.

Rgds...Geoff

[Olivier Masse]

Weird, it compiled fine on IA and OpenVMS. Here is something that might work better, no guarantees.

[Geoff Wild]

And now you know my problem - trying to get something that will run on multiple platforms :(

# /opt/aCC/bin/aCC -O raw-cpu-perf.1.c
Error 328: "raw-cpu-perf.1.c", line 25 # Function 'random' has not been defined yet; cannot call.
result=random() * random() * sqrt(log(random())) - random() + random();
^^^^^^
Error 203: "raw-cpu-perf.1.c", line 33 # Cannot assign 'char *' with 'void *'.
ptr=calloc(100, 1048576);
^^^^^^^^^^^^^^^^^^^^


Rgds....Geoff

[Geoff Wild]

okay - got that to compile - had to link in a math library (-lm).

cc -O -lm raw-cpu-perf.c

Some info on this here:

http://docs.hp.com/en/B3906-90005/apcs03.html


Rgds...Geoff

[Geoff Wild]

A couple of interesting results...

Raw Cpu Performance
Rank Seconds OS System Description
1 37.5 HP-UX 11.31 rx6600 4(dual) x 1628 Mhz
2 44.26 HP-UX 11.31 rx2600 1 x 1300 Mhz
3 61.39 SunOS 5.10 t2000 1 x 1000 Mhz
4 70.58 SunOS 5.10 V245 2 x 1504 Mhz
5 71.72 SunOS 5.10 V240 2 x 1503 Mhz
6 192.29 HP-UX 11.11 rp7410 6 x 750 Mhz

This program doesn't seem to run as expected on a Intel Linux server (DL360, 2 a 1260 Mhz scores 0.630s

Here's some Scimark results:

Scimark Small
Rank MFlops OS System Description
1 214.9344738 HP-UX 11.11 rp7410 6 x 750 Mhz java.version: 1.4.2.10
2 200.046837 SunOS 5.10 V240 2 x 1503 Mhz java.version: 1.5.0_07
3 195.9010985 SunOS 5.10 V245 2 x 1504 Mhz java.version: 1.5.0_10
4 131.8923304 HP-UX 11.31 rx6600 4(dual) x 1628 Mhz java.version: 1.5.0.03
5 106.5934764 HP-UX 11.31 rx2600 1 x 1300 Mhz java.version: 1.5.0.03
6 15.77793888 SunOS 5.10 t2000 1 x 1000 Mhz java.version: 1.5.0_10

Scimark Large
Rank MFlops OS System Description
1 106.1518022 HP-UX 11.31 rx6600 4(dual) x 1628 Mhz java.version: 1.5.0.03
2 71.87370326 HP-UX 11.31 rx2600 1 x 1300 Mhz java.version: 1.5.0.03
3 69.18737026 SunOS 5.10 V240 2 x 1503 Mhz java.version: 1.5.0_07
4 67.42025397 SunOS 5.10 V245 2 x 1504 Mhz java.version: 1.5.0_10
5 52.78626487 HP-UX 11.11 rp7410 6 x 750 Mhz java.version: 1.4.2.10
6 14.14959638 SunOS 5.10 t2000 1 x 1000 Mhz java.version: 1.5.0_10


On the Linux server I get 4.45 Mflops small and 4.51 Mflops large.

Rgds...Geoff

[Dennis Handly]

>but I also want to compare against PA-RISC, when I try to compile I get:
Error (future) 229: "Ambiguous overloaded function call; Choosing "long double log
Error 203: Cannot assign 'char *' with 'void *'.
ptr=calloc(100, 1048576);

Your source is probably C, compile with -Ae.

If you want to make it C++, you need to make sure that log()is pass a float, double or long double and not an int.

For calloc, you must cast to char*.

[Olivier Masse]

[quote] This program doesn't seem to run as expected on a Intel Linux server (DL360, 2 a 1260 Mhz scores 0.630s [/quote]

I couldn't say about Linux, but just for kicks I tried my program on my FreeBSD box at home and it does work, although it takes forever at 817 seconds on a P3-1Ghz. :)

As I said, don't consider the results as scientific, I know nothing of benchmarks (or C, for that matter).

[Rasheed Tamton]

For Solaris, http://www.brendangregg.com/cachekit.html

Also for more benchmark info.
http://www.spec.org/

Regards,
Rasheed Tamton.