SPONSORS:






User Tag List

Results 1 to 7 of 7

Thread: Tool Selection

  1. #1
    New Member
    Join Date
    Dec 2007
    Posts
    9
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Tool Selection

    Hi,

    I am evaluating a performance testing tool for one of the project. The AUT is built a web based application.

    Technology
    - .net
    - Has Telerik controls
    - Using HTTP/S protocols
    - Load balanced
    - SQL Server

    Objective:
    - Measure the page load time for each page
    - Measure the time required to complete the Acquire process
    - Measure the time required to complete the Extract Process
    - Measure the time required to complete the Transform Process
    - Measure the time required to render data on page at client side after server side operations are completed
    - Monitor the server performance on resource utilization
    - Vary the user load from 10 to 50 concurrent users.

    Can you please suggest any open source or licensed tools which I can evaluate.

  2. #2
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,030
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0
    It sounds like this may be your first foray into performance testing. There are some architectural questions to answer which can then lead you to a tool. Assuming this is your first effort, do not take the path of attempting to deliver value without training or mentoring on your first project. This will be impossible unless you are bringing two decades of refined foundations skills to the table. And, were that the case you probably would not be asking the questions you have asked.

    Answer these questions
    1. What is the communications architecture between the client and the next upstream architectural component? Once you know then then you can simply look for tools which exercise that interface.

    Once you have a subset of tools to examine then there are tougher questions

    2. Does it work in my environment? This means OS, software configuration, security policies, etc....
    3. Does it capture performance counter/metrics in my environment? CPU, Disk, Memory and Network are just the beginning. Ultimately to understand why somethings is slow or has failed, which works for one, you need to understand what finite resource pool element is being impinged, oversubscribed, etc... at the time of the event. Tools which collect monitoring data allow you to line up on a common timescale the resource use with the time of slow performance or failure to understand where things went wonky. Knowing what to monitor leverages your foundation skills in architecture and systems analysis
    4. Does this tool facilitate my analysis and reporting? You have requirements related to load and performance. There are tools all over the capability map. Some of ones which throw load and leave anything else up to your programming skills to figure out. Some will auto mark some candidates for examination. Some will report against an SAL elegantly. Others could care less. All of this leads to the concept of each tool having a labor quotient and a tool quotient to hit a given mark. Knowing how long and how much labor pins itself to the foundation skill of project management.
    5. Do I have the skills to use the tool? This could be the best tool in the world. This could be the pagani of performance tools, but if everytime you get behind the wheel there is a loud crash as you hit a proverbial tree, then the tool has zero practical value.

    Lastly, evaluate

    Hire experts on each of your remaining tools. Not people that have a checkmark on their resume, but ones which are demonstrably experts in their field. Provide them all with the same requirements for exercise and reporting of results. Bring them in for two days. Time how long it takes to complete each task to the same defined point so you understand any labor differences. Keep in mind you will not be as productive for likely nine months to a year of daily use and any differences are likely to be exaggerated.

    Or, I could just say "Use Brand X. It's the best." In a real world that would be nice but it just doesn't work.


    Quote Originally Posted by akshayj View Post
    Hi,

    I am evaluating a performance testing tool for one of the project. The AUT is built a web based application.

    Technology
    - .net
    - Has Telerik controls
    - Using HTTP/S protocols
    - Load balanced
    - SQL Server

    Objective:
    - Measure the page load time for each page
    - Measure the time required to complete the Acquire process
    - Measure the time required to complete the Extract Process
    - Measure the time required to complete the Transform Process
    - Measure the time required to render data on page at client side after server side operations are completed
    - Monitor the server performance on resource utilization
    - Vary the user load from 10 to 50 concurrent users.

    Can you please suggest any open source or licensed tools which I can evaluate.

  3. #3
    Apprentice
    Join Date
    Nov 2015
    Posts
    30
    Post Thanks / Like
    Blog Entries
    3
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0
    Totally agreed with jpulley3.

    Quote Originally Posted by jpulley3 View Post
    It sounds like this may be your first foray into performance testing. There are some architectural questions to answer which can then lead you to a tool. Assuming this is your first effort, do not take the path of attempting to deliver value without training or mentoring on your first project. This will be impossible unless you are bringing two decades of refined foundations skills to the table. And, were that the case you probably would not be asking the questions you have asked.

    Answer these questions
    1. What is the communications architecture between the client and the next upstream architectural component? Once you know then then you can simply look for tools which exercise that interface.

    Once you have a subset of tools to examine then there are tougher questions

    2. Does it work in my environment? This means OS, software configuration, security policies, etc....
    3. Does it capture performance counter/metrics in my environment? CPU, Disk, Memory and Network are just the beginning. Ultimately to understand why somethings is slow or has failed, which works for one, you need to understand what finite resource pool element is being impinged, oversubscribed, etc... at the time of the event. Tools which collect monitoring data allow you to line up on a common timescale the resource use with the time of slow performance or failure to understand where things went wonky. Knowing what to monitor leverages your foundation skills in architecture and systems analysis
    4. Does this tool facilitate my analysis and reporting? You have requirements related to load and performance. There are tools all over the capability map. Some of ones which throw load and leave anything else up to your programming skills to figure out. Some will auto mark some candidates for examination. Some will report against an SAL elegantly. Others could care less. All of this leads to the concept of each tool having a labor quotient and a tool quotient to hit a given mark. Knowing how long and how much labor pins itself to the foundation skill of project management.
    5. Do I have the skills to use the tool? This could be the best tool in the world. This could be the pagani of performance tools, but if everytime you get behind the wheel there is a loud crash as you hit a proverbial tree, then the tool has zero practical value.

    Lastly, evaluate

    Hire experts on each of your remaining tools. Not people that have a checkmark on their resume, but ones which are demonstrably experts in their field. Provide them all with the same requirements for exercise and reporting of results. Bring them in for two days. Time how long it takes to complete each task to the same defined point so you understand any labor differences. Keep in mind you will not be as productive for likely nine months to a year of daily use and any differences are likely to be exaggerated.

    Or, I could just say "Use Brand X. It's the best." In a real world that would be nice but it just doesn't work.

  4. #4
    Member
    Join Date
    Oct 2013
    Posts
    115
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0
    Given this requirement:

    Measure the time required to render data on page at client side after server side operations are completed
    I believe the only choice is using Selenium browser automation framework and running your tests using Grid.

    Other performance testing tools are working on protocol level, i.e. send programmatically built HTTP requests and receive responses, but none of them actually "renders" the page or executes client-side JavaScript.

    If this requirement can be omitted or substituted by running single-threaded Selenium test in parallel or manual client-side performance assessment using i.e. YSlow, in that case any tool will fit.

    If you don't plan to simulate more than 50 concurrent users you can consider HP LoadRunner, it will be free for that amount.

    There is also a number of free and open source load testing tools, the most popular of them seems to be Apache JMeter

  5. #5
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,030
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0
    Actually you can measure the cost of rendering with the built in developer tools in every browser with a single user. Chrome is particularly adept at this where the total cost can be illustrated

    https://developers.google.com/web/to...ine-tool?hl=en

    In short, all you need is a process change, to have all of your developers and functional testers just look at the built in tools and compare to a reference standard

    And, to be fair, all of the classical performance testing tools which can run both GUI and API level virtual users can also measure this under load. It simply requires a naming convention to do so. Login_GUI for when the full browser is executed versus LOGIN when at the API level. The difference between the two represents what most lay persons would call "rendering," but actually includes all local application processing inside of the browser, both directly related to the compositing/rendering/painting as well as local logic in JavaScript or other scripting language.




    Quote Originally Posted by glinius@live.com View Post
    Given this requirement:



    I believe the only choice is using Selenium browser automation framework and running your tests using Grid.

    Other performance testing tools are working on protocol level, i.e. send programmatically built HTTP requests and receive responses, but none of them actually "renders" the page or executes client-side JavaScript.

    If this requirement can be omitted or substituted by running single-threaded Selenium test in parallel or manual client-side performance assessment using i.e. YSlow, in that case any tool will fit.

    If you don't plan to simulate more than 50 concurrent users you can consider HP LoadRunner, it will be free for that amount.

    There is also a number of free and open source load testing tools, the most popular of them seems to be Apache JMeter

  6. #6
    SQA Knight
    Join Date
    May 2006
    Location
    Playa Del Rey, California, United States
    Posts
    2,625
    Post Thanks / Like
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0
    I'd probably won't use selenium. Depending on browser, Selenium injects tons of JavaScript and adds a large overhead to the page performance.

    It could be used more like an automated warning system. (If a page performs slower than X raise a warning), but I would not equate performance under selenium to what the user experiences.
    David Lai
    SDET / Consultant
    LinkedIn profile

  7. #7
    Member
    Join Date
    Jul 2007
    Posts
    71
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0
    I did this comparison for Open Source tools with Pros and Cons of each tool sometime back,

    https://automation-performance.blogs...valuation.html

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 8.11%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:03 PM.

Copyright BetaSoft Inc.