LoadRunner and Performance Center
Showing results for 
Search instead for 
Do you mean 

Load testing and continuous integration for Agile and non-agile environments

Anonymous_User1 ‎06-05-2013 07:10 AM - edited ‎09-22-2015 01:35 PM

(This post was written by Gal Shadeck and Yuriy Pavlovsky from the LoadRunner R&D team)


As more and more software companies and departments switch to Continuous Integration and Continuous Delivery practices, it becomes crucially important to integrate load and performance tests into the process. Developers and DevOps engineers must make sure that each new build which goes directly to QA, or even directly to the end user, doesn’t introduce performance regressions. Additionally, it’s sometimes important to include existing unit tests in load-testing scenarios, or perform additional analysis on unit test execution.


That’s why LoadRunner 11.52 contains a set of new features aimed at continuous execution of performance tests and integration with unit testing frameworks.


Jenkins Plug-in


The HP Application Automation Tools plugin for the Jenkins continuous integration server has been available since the release of UFT 11.50. Its second version, released to coincide with LoadRunner 11.52, provides a simple mechanism for executing LoadRunner Controller scenarios as part of your build script.


Anyone who has experience with LoadRunner scenarios knows that each run produces a large amount of data, which gives the full picture of the AUT’s performance and is best understood using LoadRunner’s Analysis application. However, when executing a load testing scenario as part of Continuous Integration, Deployment or Delivery, the most important thing is to get a clear “pass/fail” indication of whether the application’s performance has been hit as a result of recent changes in the code, without doing any data processing. That’s why only scenarios which have built-in service-level agreements (SLA’s) can be run via Jenkins.


Here’s how this can be achieved (we assume that if you’ve read this far, you know what Jenkins is and are familiar with its basic features):


  • Install the plugin by going to ‘Manage Jenkins’, then ‘Manage plugins’.  In the ‘Available’ tab, look for ‘HP Application Automation Tools’, and install it:

  • Select an existing build project or create a new one. Typically, functional and performance tests should be placed within the product’s build script, following the build and deployment sections.
  • Add an “Execute HP tests from file system” step:

(note that the “Execute HP tests from HP ALM” step supports executing functional tests, not load testing scenarios; we’re considering adding this feature to a later version of the plugin)

  • Set the step properties:


  1. Tests – list of paths to scenario folders or .lrs files, separated by new lines. Yes, more than one scenario can be executed in a single build! As mentioned above, only scenarios with defined SLA’s are allowed (others will be marked as “failed” without execution). Also note that the paths you supply must be absolute and reachable from different computers within your local network, as Jenkins can use slave machines for test execution.
  2. Timeout (in minutes) – timeout for the entire build step. If the scenarios included within the step are not completed after the specified number of minutes, the entire step is marked as “failed”.
  3. Controller polling interval (in seconds) – polling interval for checking the scenario execution status. This will affect the frequency of status updates in the Jenkins console during the run.
  4. Scenario execution timeout (in minutes) – timeout for lengthy parts of a scenario execution, which include running Vusers, collating results, analyzing results and shutting down the Controller. If any of these operations within any of the selected scenarios is not completed after the specified number of minutes, the step is terminated and declared “failed”.
  5. Ignored error strings – scenario execution is considered failed either if the SLA is violated or if an error occurs (in addition to the trivial cases, such as scenario not being found at the specified location). In order not to fail the scenario on insignificant errors, a list of error messages, which can be safely ignored, can be supplied. If an error occurs, and at least one of the strings in this list turns out to be a substring of the error message, the error is disregarded. For example, if we add the word “CPU” to the list, the often encountered “80% CPU use” error will be skipped. No need to use asterisks or any other special characters.
  • Define publishing of the test results as a post-build action:




  • Once the Jenkins project is ready, run it, either immediately or using the scheduler
  • During the execution, you’ll be able to view its progress using the Jenkins output console. The moment the job has finished, you’ll get an immediate indication of the tests’ status:



(note that the zip file that appears under “Build Artifacts” contains the full run results and can be processed using Analysis)



IDE DevOps plugin


To support deep and effective integration of unit or functional tests with LoadRunner infrastructure we provide addins for Microsoft Visual Studio 2010 and Eclipse with LoadRunner 11.52. You can install these from the LoadRunner 11.52 Additional Components location.


Let’s suppose that you have some existing code with business logic and unit tests for it. The following example is based on the Microsoft Visual Studio 2010 IDE with the NUnit framework. For Java-based frameworks with the Eclipse IDE, the Continuous Delivery plugin provides exactly the same functionality.



Now, extending your test with all the power of LoadRunner’s infrastructure is as easy as clicking on “Add LoadRunner API reference” in the “DevOps Vuser” menu, which is created when you install the addin. After this simple action, you can use LoadRunner functionality such as transactions, messaging, think time, etc. in your code. 



Alternatively, you can use the context menu of your unit test project in the solution:



In the code, instantiate the LoadRunner API:



.. and now you can have LoadRunner transactions in your unit test code!



It is important to note that this change doesn’t break the way the test works.  As long as the referenced libraries are available, it will work the same way it used to work before.  If you execute the test as usual, the additional LoadRunner code won’t actually do anything.  But now you can run it as a LoadRunner VUser!



The result of the running this test module as LoadRunner script is redirected to the “LoadRunner Information” output window in the IDE:




Unit test support in the Controller


To support execution of unit tests as part of LoadRunner load test scenarios, the LoadRunner Controller was extended with the ability to open not only LR scripts, but also system or unit tests:



 After the test module is opened, a temporary script is created and shown:



You can continue with scenario creation just as you did before with regular LoadRunner scripts.



Putting it all together


The features described in this article can be combined together. Take your existing unit test (or write a new one), extend it with the LoadRunner infrastructure, create a Controller scenario based on it and execute it as part of your build job.  In a few simple steps, you’ve protected your application from performance regressions.


(Note: if .NET is your technology of choice, make sure that NUnit and Jenkins are installed under the same user account)



Click here to learn more about HP LoadRunner

Click here to learn more about HP Performance Center of Excellence




Let us know what you think by leaving us a comment in the box below.



Thanks to Gal and Yuriy for providing this post!


About the Author


on ‎11-15-2013 01:23 AM



Action.c(6): Error: Failed to find SapGui component by ID "sbar"


Got error when I run oad test for SAP GUI using multiple Vusers .

Please help me to resolve this issue.



Thanks & Regards,


on ‎12-04-2013 12:10 AM

Hi venubasan,


I recommend that you open a support ticket for this issue, or post the question on the LoadRunner Support Forum.

on ‎06-17-2014 04:11 AM

Hi Malcolm,

    You mention that - “Execute HP tests from HP ALM” step supports executing functional tests, not load testing scenarios - and that this would be provided in a later version. Could you please let me know when the load test scenarois would be supported?

    We need to automate the execution performance tests on HP PC. We are working on PC 11, so the "Execute HP tests using HP Performance Center" also does not work. In the plugin link, it mentions HP PC 12 as the version that can be used:

    Alternatively are there any command line options avaible to run the performance tests through HP Performance Center 11?


Appreciate the help.




‎06-18-2014 05:20 AM - edited ‎06-19-2014 05:46 AM

Hi RS,


The current version of the plugin supports load testing scenarios.  See here for the list of supported tools and versions, and here for instructions on running LR scenarios from the file system.


I'm not aware of plans to support PC 11, but you can post on the PC support forums and ask about this and your query about command line options.

Prem Kumar Jayaseelab
on ‎06-08-2015 11:10 PM

I am using eclipse and integrating junit. I have installed the addin LREclipseIDEAddInDevSetup. Devops Vuser option does appear in eclipse but when i try to add "Loadrunner API Reference  to project nothing happens.


Could you please provide solution to this?

on ‎02-28-2016 08:13 PM

Hi, Thank You for information on Continous Integration. I am trying to use the IDE Devops Plugin option in my project for selenium scripts. Suddenly they are switching to Serenity codes instead of Selenium. Please let me know if the plugin works with Serenity code as well? Will the performacne measurements captured in serenity code will be accurate because serenity has lot of layers inbetween the call made from user to actual request to the servers.

Appreciate your quick respone.



27 Feb - 2 March 2017
Barcelona | Fira Gran Via
Mobile World Congress 2017
Hewlett Packard Enterprise at Mobile World Congress 2017, Barcelona | Fira Gran Via Location: Hall 3, Booth 3E11
Read more
Each Month in 2017
Software Expert Days - 2017
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all