Performance of prospective systems and TPM figures

[Yogeeraj_1]

Dear experts!

we are currently in the outlook for more performance for our Information Systems that have been running on two HP9000 servers since 3 years now.

We have come across so many performance related web pages when googling, namely:
- SAP SD Standard Application Benchmark Results
- March 30, 2005; The Forrester Wave™: Application Server Platforms, Q1 2005 by John R. Rymer
- Top Ten TPC-C by Performance at tpc.org
- TPC Results Database: www.microsoft.com/sql/tpc
- IDEAS web site: www.ideasinternational.com

Including the following TPC Technical Report Articles:

a. TPC-DS, Taking Decision Support Benchmarking to the Next Level.
by Meikel Poess, Bryan Smith, Lubor Kollar, and Paul Larson


b. TPC-W: Benchmarking An Ecommerce Solution.
by Wayne D. Smith, Intel Corporation
(http://www.tpc.org/tpcw/TPC-W_Wh.pdf )

c. ACM SIGMOD Record, 29(4) December 2000.
"New TPC Benchmarks for Decision Support and Web Commerce"
by M. Poess, C. Floyd ACM RECORD
(http://www.acm.org/sigmod/record/issues/0012/standards.pdf )

d. Issues in Metric Selection and the TPC-D Single Stream Power by Alain Crolotte
(http://www.tpc.org/information/other/pdf/tpcd_metric.pdf )

e. TPC-C and TPC-D V.1.X Overview Presentations
(http://www.tpc.org/information/sessions/sigmod/sigmod97.ppt )

f. A Recommendation for High-Availability Options in TPC Benchmarks by Dean Brock, Data General
(http://www.tpc.org/information/other/articles/ha.asp )

g. What is the TPC Good For? or, the Top Ten Reasons in Favor of TPC Benchmarks by Gary Burgess, Ideas International
(http://www.tpc.org/information/other/articles/TopTen.asp )

h. Relevance of the TPC-D Benchmark Queries: The Questions You Ask Every Day by Carrie Ballinger, NCR Parallel Systems
(http://www.tpc.org/information/other/articles/TPCDart_0197.asp )

i. Why You Should Look at TPC-C First: An Interview between TPC-C Subcommittee members and Kim Shanley, TPC Chief Operating Officer
(http://www.tpc.org/information/other/articles/TPCCart.asp )

j. TPC-D, Past, Present and Future: An Interview between Berni Schiefer, Chair of the TPC-D Subcommittee and Kim Shanley, TPC Chief Operating Officer
(http://www.tpc.org/information/other/articles/TPCDppp.asp )

j. No Analysis Required! The Performance Analyst's Paradox. by Dr. Neil J. Gunther, Performance Dynamics Company
(http://www.tpc.org/information/other/articles/NoAnalysis.asp )

k. TPC-D Interview: With Jack Stephens, Francois Raab, and Kim Shanley
(http://www.tpc.org/information/other/articles/TPCart_Q195.asp )

l. Transaction Processing: Concepts and Techniques by Jim Gray and Andreas Reuter
(http://www.mkp.com/books_catalog/catalog.asp?ISBN=1-55860-190-2 )


Since our system is more of an OLTP/DSS type, we should be concentrating more on the TPC-C and TPC-D figures.

According to the posted figures in one of the powerpoint documents, we find the following:
Competitive TPC-C Configuration (sep 1997??)
- 8070 tpmC; $57.66/tpmC; 5-yr COO= 465 K$
- 2 GB memory, disks: 37 x 4GB + 48 x 9.1GB (560 GB total)
- 6,700 users


We have also found that in general, IBM Power eservers provide better performances.

We have also been considering entry-level servers proposition from vendors: HP rp3440-4 (2 way) and IBM P5-550 (4-Way @ 1.5 GHz)

Seems like the IBM proposition is superior!

Anyone who have had to perform a similar exercise recently?

please comment.

kind regards
yogeeraj

[Arunvijai_4]

Hi Yogee,

Consider HP's support vs IBM. This topic has been discussed here in the past. Support will also play a huge role when it comes to Enterprise computing.

-Arun

[Senthil Kumar .A_1]

Hi Yogi,

I do agree with Arun. While the performance statistics are important and justified to some extent, A wise decision would be some tradeoff's in regards to service the company provides.

A percent of tradeoff in some performance parameter would not matter a lot to a application in comparision to the support metric's of two enterprise solution company as far a "24X7" support's. I would even advice to consult the documents regarding the companies, service package along with some successful case studies from both the end in the maintence of their solutions.especially there BUSINESS CONTINUATION PLANS.I will add some links in this thread,If I'm successful in googling for some.

Regards,
Senthil Kumar .A

[Yogeeraj_1]

hi Arun,

Support is just one criteria.

However, support in the region is questionable.

We are currently considering the performance criteria

thank you for your reply

kind regards
yogeeraj

[Arunvijai_4]

Hi Yogee,

Yes, i agree with you. In such cases, you may need to consider the typical system and setup not the lap where TPCs were performed.

And what about $$$$$ and ROI ??

-Arun

[Yogeeraj_1]

hi arun,

As mentioned previously, we are only looking at the performance factor right now. Price is almost comparable for both options.

thank you

kind regards
yogeeraj

PS. I would be grateful if you don't post any links unless you are quoting something specific. Have been reading so many web pages during the past 48hours that i almost dislike the google interface!! :-p

[Chan 007]

Yogee,

In fact, I advice you to go for a weight ratio. You can do that using a xls and giving marks for each.

E.g. It will look like

#. Desc - Max Marks - HP - IBM - SUN
1. HW Compatibility - 5 - 4 - 4 - 4
2. SA Skills inhouse - 10 - 8 - 7 - 4
3. DB Compatibilit - 10 - 8 - 8 - 9
4. Already Similar Conf - 10 - 9 - 6 - 2
5. Price - 10
6. Support in your area - 10

Finally total it

This is what I do it here.

Chan

[Vincent Fleming · Accepted solution]

The problem with benchmarks is that they never benchmark the configuration you are going to buy...

For example, you quote a benchmark (TPC-C) from 1997 that uses 4GB and 9.1GB disks.

Neither of those disk sizes are available anymore. You would be looking at 36GB and 72GB disks now (at minimum!). Processor and bus speeds have changed a lot also. All these things can make a significant difference in your performance.

Also, since benchmarks are synthetic, you can't really use them to determine what *your* performance will be like, even if the configuration were the same. It depends on your application. Your actual performance may vary significantly from the benchmark.

This goes for ALL vendors.

So, I agree that you should look at more than the benchmarks. Much more!

Good luck,

Vince

[Yogeeraj_1]

hi vince,

thank you for your reply.

all the information that i am getting are very confusing indeed.

The benchmarks cater for only 20% of others while us - the 80% always starve for the right information that would help us take realistic decisions.

We don't want to go for an overkill!

2 processors, 16 gb of RAM, 1 terabyte of disk space are already too much! HP is providing L2 cache in terms of KBs while IBM is offering MBs. Same for L3 cache...

I know this is not an exact science.

More comments from the Experts would be most appreciated.

kind regards
yogeeraj

[Yogeeraj_1]

Dear all,

more guidances with people who have been working with Heterogeneous systems (e.g. HP, IBM) will be most appreciated.

thanking you in advance

kind regards
yogeeraj