3rd party performance analysis tools for LoadRunner
An ex-boss has asked me a question about analysing performance stats…I said I would ask and get back to him as I'm not a performance tester. I wondered if anyone had an answer?
“Which tool do you use to analyse performance stats from load runner volume and break testing. The tool our supplier was going to use was CA performance analyser but apparently it is going out of service and there isn't a replacement.”
Thanks to all for any feedback…
Why would you not use the built in Analysis tool, which is a part of LoadRunner and which is designed to analyze and report on test results?
Originally Posted by kristian
Note: The Analysis tool has been available since LoadRunner 6.0 and has been included in every release since version 6.5. You are able to examine response time data crossed with both standard monitors and deep diagnostic information for the analysis of tests.
Originally Posted by jpulley3
Thanks for reply...I have passed on your response, I'm not sure why LoadRunners analysis itself isn't being used. Another friend of mine mentioned NMON & Topas...have you used these tools?
I can tell you why Analysis is not being used, or being used effectively. This is a well known tool usage pattern where people using the tool have not been through training on the tool or have been provided with a mentor for a period of time when first using the tool. It is usually marked by low value for the performance test output and management seeks to offset the low value by getting feedback from platform owners during the test as to what is actually taking place. This happens when people see performance tools only as tools which "throw load," without the understanding that these same tools collect monitor data and allow for the deep analysis of test results. In shorthand, you have people without the skills and knowledge to be able to generate high value using the tool.
Are you aware that since version 8.0 of LoadRunner, an in the box version of SiteScope ships to allow for the collection of statistics from UNIX and LINUX hosts by logging in via SSH and extracting information via command line interfaces, bringing that back to LoadRunner for integration with the response time data automatically during the test?
Are you are aware that if external monitor data is collected in a file during testing that this monitor data can be integrated directly into LoadRunner results at the end of the test?
Are you aware that both historic and in-SiteScope interfaces exist to monitor all of Windows performance metrics and pull that information back into LoadRunner during the execution of a test?
Are you aware that all existing performance testing tools which leverage application tagging can be integrated into LoadRunner to allow for deep dive on transactions?
Are you aware that the results from each tests are stored natively in a database format when using Analysis? Analysis can be configured to leverage Microsoft SQL Server for storage of results as well. The Schema is public and because of this you can write your own queries. You can use statistical tools such as R and SAS for deep analysis. You can even use the database generated to underpin your own Website to display results across the organization (as well as leveraging the Analysis Tool's ability to generate a website with results and/or export to a document which can be edited in Word.
Last edited by jpulley3; 05-09-2016 at 07:20 AM.
Thanks again for your response...you time is much appreciated. I shared with my ex-boss and he's replied with the following...(Both of us aren't performance guy's so your feedback jpully3 helps)
"Can I ask you to slightly change the focus. Reason for optimiser was to simulate changes in the hardware platform. I.e take out cpu's take out web servers.
A little more clarity, capacity planning tool based on load runner volume and break tests."
Capacity planning for "what if" hardware changes are different classes of tools entirely. You will need sophisticated monitoring of your application under load to build the models. This is non trivial. Look to the Hyperformix line of products, now owned by CA, for integration. Speak to your CA rep on the current name of what used to be known as Hyperformix Capacity Planning, which was also for a time sold under the name of Mercury Capacity planning.
Originally Posted by kristian
Your models will require data points at various load levels of users for CPU, DISK, Memory and Network on all of the architectural platforms. These data points collected from your architecture will be used to project the utilization curves of resources under load. The architecture models inside of the software will then allow you to project series of "what ifs" on swapping resources. In the end, you will need to run the model live in a test since every model makes assumptions which may not be true in your environment. Nothing beats live test data.
Replace ineffective offshore contracts, LoadRunnerByTheHour
. Starting @ $19.95/hr USD.
Put us to the test, skilled expertise is less expensive than you might imagine.
Twitter: @LoadRunnerBTH @PerfBytes