LoadRunner and Performance Center
cancel
Showing results for 
Search instead for 
Did you mean: 

re: What are the Goals of Performance Testing?

Randy Stevenson_1

I just want to print this Blog entry out on parchment paper and frame it on the wall of my cubicle!

Here at Deere & Company I feel like the lone evangelist for the Performance Testing philosophy you are describing. I am constantly trying to convince Management and others that our job as Application Performance Testers is not just conducting some "load tests" on an application 2 days before it gets deployed into Production. When I try to incorporate the use of monitoring during testing I have to fight for the necessary "rights" to even access the Servers on which the application runs. And I am pretty much not allowed to even think about monitoring the Databases (DB2, SQL Server, and Oracle), the Middleware (MQ), or the Network. Worse yet, trying to get the various experts in the aforementioned areas to work cooperatively with our team during testing is an ongoing struggle.

As a result of the above "fights", I am struggling to demonstrate value to the testing that we do conduct. In the past week alone, here are excerpts from two "analysis" reports from my colleagues: "We tested the application with 100 VUsers, 150 VUsers and 200 VUsers. It ran OK with 100, but crashed with the other loads." and "The application ran well for 45 minutes, but then it experienced weirdness". I'm serious! On top of this, I see constant reports that are nothing more than the LoadRunner Analysis Report in Word Doc form --- without any real explanation for what a "VUser" really is, what Hits per Second or Throughput even mean (and should we care), and whether or not the application will run at the projected loads.

The reason I am even on the HP site right now is I was looking to see if I could find more current information on several things I personally need for a project on which I am working: 1) A better understanding of how to monitor WebSphere Applications running on a virtualized Apache/Linux environment. 2) A way to monitor DB2 running on AIX. 3) A way to monitor AIX in general, with consideration for the concept of entitilement. 4) A way to diagnose Application "crashes" in WebSphere (including how to read Heapdumps and Javacores). Unfortunately or fortunately, I stumbled on this Blog, and have to give up my search again for today. Oh well, I have been trying to get the HP Diagnostics software to run in our environment since at least February, so what's a few more days? I can always open another HP Service Request, learn Spanish so I can decipher the recommendations I am offered, and give up again a month from now.

I would like to ask if you can find someone who can draft a "Performance Testing Specialist's Oath" that addresses all of the points you have made. That way I can at least give the new testers on our team something to shoot for.

Randy at Deere
0 Kudos
About the Author

Randy Stevenson_1

Comments
Randy Stevenson_1

I just want to print this Blog entry out on parchment paper and frame it on the wall of my cubicle!

Here at Deere & Company I feel like the lone evangelist for the Performance Testing philosophy you are describing. I am constantly trying to convince Management and others that our job as Application Performance Testers is not just conducting some "load tests" on an application 2 days before it gets deployed into Production. When I try to incorporate the use of monitoring during testing I have to fight for the necessary "rights" to even access the Servers on which the application runs. And I am pretty much not allowed to even think about monitoring the Databases (DB2, SQL Server, and Oracle), the Middleware (MQ), or the Network. Worse yet, trying to get the various experts in the aforementioned areas to work cooperatively with our team during testing is an ongoing struggle.

As a result of the above "fights", I am struggling to demonstrate value to the testing that we do conduct. In the past week alone, here are excerpts from two "analysis" reports from my colleagues: "We tested the application with 100 VUsers, 150 VUsers and 200 VUsers. It ran OK with 100, but crashed with the other loads." and "The application ran well for 45 minutes, but then it experienced weirdness". I'm serious! On top of this, I see constant reports that are nothing more than the LoadRunner Analysis Report in Word Doc form --- without any real explanation for what a "VUser" really is, what Hits per Second or Throughput even mean (and should we care), and whether or not the application will run at the projected loads.

The reason I am even on the HP site right now is I was looking to see if I could find more current information on several things I personally need for a project on which I am working: 1) A better understanding of how to monitor WebSphere Applications running on a virtualized Apache/Linux environment. 2) A way to monitor DB2 running on AIX. 3) A way to monitor AIX in general, with consideration for the concept of entitilement. 4) A way to diagnose Application "crashes" in WebSphere (including how to read Heapdumps and Javacores). Unfortunately or fortunately, I stumbled on this Blog, and have to give up my search again for today. Oh well, I have been trying to get the HP Diagnostics software to run in our environment since at least February, so what's a few more days? I can always open another HP Service Request, learn Spanish so I can decipher the recommendations I am offered, and give up again a month from now.

I would like to ask if you can find someone who can draft a "Performance Testing Specialist's Oath" that addresses all of the points you have made. That way I can at least give the new testers on our team something to shoot for.

I agree completely with Runner53.  I experience the same hurdles and I am at a loss to articulate the value of utilizing the LR product to the full extent.  Not allowing LR access to a server via a service account to gather perfmon stats is just one of the hurdles.   A basic understanding of the complexity of testing is another.  Scripting is a means to an end, not the end goal.

 

I am on the HP site looking for standard best practices to present to the PMs, to give them an understanding that performance testing is not just click and script and run.  It takes preparation, LOB input and interpretation of results.   

 

Having a "LR Best Practices" site would be helpful.  I would like to see HP take an active roll in assisting the performance engineer in explaining Performance testing and the value-add to the company.  The company has purchased a very powerful and useful product and under the guidance of an experienced engineer, production issues could be caught in QA.  Our goal as performance engineers is to take our years of experience in engineering and architecting combined with a knowledge of the LR product to deliver a quality application and positive end user experience.  After all, we want the customer to return to our site or application and use it, that is why we developed it.  It keeps us in business.

Events
June 6 - 8, 2017
Las Vegas, Nevada
Discover 2017 Las Vegas
Join us for HPE Discover 2017 in Las Vegas. The event will be held at the Venetian | Palazzo from June 6-8, 2017.
Read more
Each Month in 2017
Online
Software Expert Days - 2017
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all