Integrity Performance Comparison

[Warren G Landrum]

Hi,

Has anyone come aross a chart or doc that does a system comparison (TPS or VUPS or whatever) of Integrity Servers to each other - one sort of like the "AlphaServer system performance comparison: by performance" that can be obtained from HP's website? Also, if anyone has run across a similar chart comparing Integrity's to Alphas? Just need to have a little more ammo for cost-justifications for upcoming much-needed upgrades.

Thanks in advance!

Warren

PS - I apoloize if this is the wrong forum to be asking this type of question, but I figured if anyone would know about this, it would be one of you guys.

w

[Craig A Berry]

There are some slides in Bruden's performance cookbook here:

http://www.openvms-rocks.com/~brian/performance_cookbook.ppt

that have part of what you are looking for. I haven't seen anything as complete as the Alphaserver performance comparison you refer to.

[Hein van den Heuvel]

As with any performance question: 'it depends'.

For this particular question it depends more than ever on a larger than ever variaty of components: cpu intensiveness of the applicaton, level of SMP scaling required, cleanliness of port (alignment faults), sensitivity to memory bandwith, sensitivity to memory throughput, IO adapters. Oracle? San?

So barring a full benchmark of somthing mighty similar to your application you might as well take a very broad approach and err on the safe side.

I suggest you simply assume that a current Itanium core (1/2 montecito chip) performs better than a great Alpha chip (1150Mhz) for most applications application.
Don't worry about how much better.
That will all be positive slop.

Of course as an independent performance consultant I would encourage you to seek out a modest benchmark with professional help. But at the same time I must admit that for the price of a worthwile benchmark you might just a well build some extra slack in your calculation unless we are talking at least dozens, if not hundreds, of deployments.

So just compare your Alpha to the best of Alphas and take it from there.
And don't forget to add into your calculation that while a first Itanium deployement now may cost a little entrance fee ('port') it will get you nicely ready for future growth on the now current OS version which is likely to have a nice long lifecycle.

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Warren G Landrum]

Thanks Hein,

I know there are a lot of variables involved. That's why I was hoping to find some benchmark testing specs (ie - TPS) that had already been done in a controlled/lab-type environment where all factors were equal. Intuitively, I already know/suspected that Integrity's are going to have a higher performance than Alphas. I just wanted to see some 'hard' numbers, if possible.

Warren

[Hein van den Heuvel]

Actually... the best slides on this subject are probaly the slidesset that Greg Jordan has been presenting and maintaining.

He presented them at the bootcamp:

- OpenVMS Alpha and Integrity Performance Comparison
OpenVMS Boot Camp June 2005, Session A306

And:

- HP Technology Forum 2006
Gregory Jordan
OpenVMS Engineering Hewlett-Packard
HP OpenVMS Integrity Performance Update
Session 1152

And probably some more.

Regards,
Hein.

[Hoff]

AFAIK, no single list of performance exists.

You'll certainly find bits and pieces, and sometimes with results listings from one generation of benchmarks and other results from another.

You'll probably end up interpolating results based on other benchmarks (various of the SPEC tests, etc).

http://www.spec.org/results_search.html

Benchmarks are unfortunately not particularly representative of aggregate application performance. You're not running SPEC, after all.

As an alternative, create and run some tests of local test(s) of interest, and then re-running them on the HP Test Drive system.
I'd generally suggest you build up or package up some representative standalone application(s) or tests, and upload it all to the HP Test Drive server for a test run rather than digging for benchmarks -- and particularly given that all modern processors tend to be quite sensitive to the particular application operations.

Access to www.testdrive.hp.com is free.

Stephen Hoffman
HoffmanLabs LLC

[Robert Gezelter]

Warren,

I will concur with Hein and Hoff. The correct answer is certainly "depends".

There are many reasons for the variation. One of the most easily understood has to do with alignment faults.

The VAX architecture allowed alignment faults. On Alpha, alignment faults were processed as exceptions, at a far higher cost. On IA-64, the penalty for taking such a fault is even higher. OpenVMS Engineering has identified several cases where "poor performance" was caused by unrealized cases of alignment faults. Also, IA-64 is far more sensitive to optimization than is Alpha.

In the end, the best thing to do, IMHO, is science. Migrate (what is believed to be) a representative application, and check its performance, then work from there. As a starting point, I generally recommend that clients choose a small, entry-class server (e.g., rx1600 or rx2600). Tuning and optimization is important. If there is insufficient in-house expertise, I recommend consulting with someone who has followed the issues in this area (Disclosure: My firm does provide these services).

I have been involved in the IA-64 area since it was first announced; I was speaking on it within two months of the announcement (see http://www.rlgsc.com/alphaitanium.html ).

- Bob Gezelter, http://www.rlgsc.com

[Hein van den Heuvel · Accepted solution]

A few more observations...

OpenVMS has be notoriously, conspiciously missing in the standard benchmark arena.
If you want to use standard test with Alpha and Integrity entries you'll have to look for Tru64 work: SAP, TPC
Whether the systems would scale similary under OpenVMS in anyones guess.

Here are some TPC-C datapoints.
56,375 AlphaServer ES45 Model 68/1250
51,506 HP Integrity rx2600 â Itanium2/1.3 GHz-2p/2c  
1,008,144 HP Integrity Superdome â Itanium2/1.5 GHz-64p/64c
and the very recent Montecito result.
4,092,799 HP Integrity Superdome-Itanium2/1.6GHz/24MB iL3 64p/128c/256t

two public entries I made on this before...

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1014899

and a comp.os.vms entry which for example can be found at:

http://www.techiegroups.com/archive/index.php/t-120579.html


I like science, I'm all for science.
But as per my first reply I beg to differ on the science for this request... for a small scale deployment.
If we are talking about 100+ cpus, then sure, go for it, absolutely, but that's not how this request came accross to the.

Either go all the way with a full representative benchmark, or just wing it using some existing number as pointed to.

Not much point in taking a specint, or an 'ftp test', or any industry benchmark because none can predict how your specific application will scale unless the application is for example 90% Oracle.

I suspect a 30% accuracy in a back of the enveloppe calculation is already 'good enough' since will roudn up to whole cpu's and maybe even dual-core cpus anyway.

So maybe you find that you need 2.5 Itanium core's or 3.5 itanium cores to replace a 4p ES47. In both cases an rx2620 will do fine, in the unlikely case you can plug in enough IO. It may become an 3600 to get enough PCI slots.

To replace that GS160 you might only need to look at an rx6600.

http://h20341.www2.hp.com/integrity/cache/340168-0-0-0-121.html

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Daniel Gustafsson]

There are not that many, if any?, publicly available VMS performance comparisons with Montecito based systems yet.

Montecito does not offer huge increase in single-thread performance, but the chipset and memory performance of the Montecito based systems are much better. So to show good results you should use Montecito systems.

[Dean McGorrill]

As suggested, go hp's test drive and set up
an account for yourself. I did and was impressed. I was involved with the early
port of vms, helped do mail part of dcl etc.
the compilers weren't there then. we helped
release v8.2 before our contract with hp
ended. attached is vup.txt, rename to vup.com and do "$ @vup" it'll give a WAG!

[Warren G Landrum]

Thanks for all the feedback guys. Will go to HP Testdrive and do some testing after I get with our Apps people to give me some representative stuff to set up and test with.

Thx!

Warren

[Verne Britton]

Years ago there was a (what I call) relative performance rating of Vax and Alpha boxes... maybe it was ran under Digital Unix :-) and most models were listed in the chart ... my old notes say (in regards to my local text copy):

10-Jan-2005 newer Alpha numbers, inserted by hand... from
http://h18002.www1.hp.com/alphaserver ... PERFORMANCE ...
then Perform Comp Of Retired & Current Systems
choose By Name or By Performance

anyway I always assumed this was done using an internal benchmark suite... and thought it would be great to someday have that benchmark source (and a build kit) released ... so not only could we run it ourselves and see how close our systems came to the published figures, but also be able to run it on non-published systems (like workstations) and future systems (today that means Integrity servers).

Anyone have any recollections on the existance of this benchmark suite ?

Verne