LoadRunner and Performance Center
Showing results for 
Search instead for 
Do you mean 

How repeatable is your performance test?

‎02-04-2013 09:07 AM - edited ‎09-25-2015 02:29 PM

Guest post by Sarah Roderus,

Vice President, Consulting Services

TCT Computing Group, Inc.


It doesn’t matter if you are new to LoadRunner or Performance Center or if you are a seasoned user—understanding the sequence seed is important.


What is the purpose behind randomizing values in a load test? What is the random sequence seed in LoadRunner/Performance Center? These are questions students often ask in LoadRunner and Performance Center training classes. I want to give you the answers to these questions today, (and you won’t have to sit through a lecture to get them).


People are not robots


When designing a performance test, a big challenge is to have one single script that behaves like a group of real users. Real users are not robots. Real users are human—who move at different speeds and select different values. For example, I walked into a retail store the other day to purchase a new pair of tennis shoes for my son. Unfortunately, they didn’t have his size and color selection in stock.


So, I decided to place an order online while in the store and have the shoes shipped directly to my house. Rather than having the salesman go through the painstaking effort of typing my contact information, I asked him if I could type the information and proceed online. Being a rather fast typist, (okay, I am a rather experienced online shopper as well) I completed entering all of the information and submitting the order in approximately 1.5 minutes.

While some users may take 1.0 minute to complete this task, other users may take 3 minutes, yet others may take 4.5 minutes. Randomizing the think time values is an important step to getting that one single script to behave like real people instead of robots.


The importance of the sequence seed


When you complete the think time, pacing values and parameterized values that can be randomized; there is one additional setting to consider—j the random sequence with seed setting. By setting the sequence seed before running your test and using the same sequence seed value for a second test, you are using a mathematical technique of variance reduction. When you simulate the performance of two different systems, there are basically two sources of variation between them: 

  • Differences in the performance of the two systems
  • Differences in the (randomly generated) workload 

If you can force the simulation to use the same workload for both cases, you have eliminated one of those sources by testing the performance of the two systems against the same workload. This will increase the precision of the estimates that can be obtained for a given number of iterations.


After executing your baseline test, a piece of code is identified to be a performance issue. The normal development cycle occurs, and the modified code ends up in your environment, ready for another round of performance testing. Setting the sequence seed before executing the test again enables you to use the internal LoadRunner capabilities, allowing you to repeat random sequences of numbers.


The secret of duplication


Beware that any modifications to the test, such as: modifying the values in the parameter file that are randomly selected, think time values, adding a new business process flow, etc. will defeat your attempt to duplicate the workload from the baseline test. In order for the variance reduction technique to be properly applied, the same sequence of events with the same randomization must be identical between the runs. If you choose not to apply a sequence seed to a performance test, a different pattern of random sequence seeds could occur—potentially resulting in a better (or worse) response time.


Also, when using the sequence seed and relying on its capabilities to repeat the same sequence of values, consider the potential in your simulation for system performance to influence the workload. You also need to consider the potential for system performance to influence the workload when using the sequence seed and relying on its capabilities to repeat the same sequence of values.


As a simple example, a higher performance system might handle a number of transactions in a fixed time test. If during the baseline test, you generate a total of 1,000 transactions, but during a subsequent test, after code enhancements, you generate a total of 2,000 transactions, only the first 50 percent of the values were reproduced during the second test. Although the same random number seed was used for both runs, the workloads the two systems were tested against were different.  Depending on the random numbers generated in the second half of the test, you may over or underestimate the impact of the performance improvement over the baseline performance.     


By using the power behind LoadRunner’s capabilities and setting or not setting the Scenario Run-Time Setting value of the random sequence seed as appropriate, you increase your ability to analyze and understand differences between the baseline and subsequent test results.


What has your experience been? I would love to know your thoughts. Feel free to share them with me and the other readers in the comments section below.

0 Kudos
About the Author


This account is for guest bloggers. The blog post will identify the blogger.

on ‎02-11-2013 04:32 AM

Thanks for the insight information about PT.



NaveenKumar N
www.QAInsights.com and www.Testifications.com

on ‎04-18-2013 06:36 AM

This is a really interesting article but it doesn't explain how to set the Scenario Run-Time Setting value of the random sequence seed.


Is this feature available in Performance Center 11 or was it introduced in Performance Center 11.5?

on ‎04-23-2013 05:18 AM

Thanks for great piece of information. It was totally a different sphere of thought for me :)

Also would like to know how can we control/follow the similar random seed for next iteration of the test.




on ‎01-03-2014 05:14 AM

Thanks Sarah Roderus!

very indepth article to understand intricacy of Load testing. Also to appreciate  the capabilities in HP Loadrunner tool to best use its features!


useful....to keep in mind....during Load testing.....


Thanks again!

Nov 29 - Dec 1
Discover 2016 London
Learn how to thrive in a world of digital transformation at our biggest event of the year, Discover 2016 London, November 29 - December 1.
Read more
Each Month in 2016
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all