SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 13
  1. #1
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Perf Tool Evaluation - lower end proprietary tools

    I am in the middle of a Performance and Load tools selection process and wanted to get some feedback here.

    I currently work in a shop that uses a mix of proprietary and open source tools for web performance & load testing. The bulk of our workload and analysis is currently done using SilkPerformer. As you all probably know, there is a class of tools that is *very* expensive (including SilkPerformer). Installations and maintenance can run into 7 figures ($$$) with yearly maintenance contracts upwards of 6 figures. Since SilkPerformer is in place and we are happy with it (besides price/maintenance), there is no point in moving to a similarly priced tool. Therefore I have ruled out the class of "high end" tools from my selection:


    High-end tools
    ------------------------
    Borland/Segue - SilkPerformer
    HP/Mercury - LoadRunner
    IBM/Rational - IBM Rational Performance Tester
    Microfocus/Compuware - QALoad
    Oracle/Empirix - Oracle Load Testing For Web Applications (e-Load)


    The tool I select will be used across several web applications.. pretty straight forward HTML/AJAX/JavaScript Web UI. Here is a basic list of requirements:


    Requirements
    ------------------------
    Protocols:
    - HTTP
    - ODBC (SQL)

    Features:
    - distributed load generation
    - reporting/analytics
    - data driven testing
    - 5000+ VU


    I work on a very skilled team that is *very* proficient with programming, tools, and web technologies. Adapting to a new tool or programming language is not much of an issue.

    I've searched the Open Source landscape pretty good. There are some fantastic tools (OpenSTA, JMeter, Pylot) to augment our testing, but no open source load generation tool completely meets our criteria.


    Open Source tools
    ------------------------
    OpenSTA
    JMeter
    Pylot


    Now finally to the question/point....

    I am looking at a class of tools that I will call "low-end performance tools". This includes all proprietary tools that are not listed above as "high-end tools". They tend be cheaper and more limited in functionality than the big guns, but are substantially cheaper and sometimes sufficient for complex web performance testing. This is where my interest lies. I have scoured the web and came up with a list of tools to evaluate.


    Low-end tools
    ------------------------
    Microsoft - VSTS
    Radview - WebLOAD
    SoftLogica - WAPT
    Facilita - Forecast
    Zoho - QEngine
    Neotys - NeoLoad


    Does anyone have any feedback or experience reports using any of the "low-end" tools listed above? Are there other tools I am overlooking that I should definitely look into?

    any comments are appreciated.


    -Corey Goldberg
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  2. #2
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,018
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Where you are going to have substantial issues with the lower end tools is with the ODBC requirement. Given that you already as using open source tools you are familiar with the analysis bridges you are like to have to build versus the commercial offerings.

    VSTS may give you some approximation of ODBC, but on a programmatic level (non recorded) and working with .Net bridges to the old ODBC interfaces (i.e., .Net->OleDB->ODBC). Watch out for VSTS for as you add generators the price can begin to approximate some of the larger tools depending upon the number of generators you purchase (their pricing model is bizarre compared with the rest of the industry)
    James Pulley

    Replace ineffective offshore contracts, LoadRunnerByTheHour. Starting @ $19.95/hr USD.

    Put us to the test, skilled expertise is less expensive than you might imagine.

    Twitter: @LoadRunnerBTH @PerfBytes

  3. #3
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    thanks James. good point about VSTS cost when doing large tests.
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  4. #4
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    I have received some responses from other corners of the web where I posted this same question. I will update this thread with more info.

    the current evaluation list is looks like:

    Microsoft - VSTS
    Radview - WebLOAD
    SoftLogica - WAPT
    Facilita - Forecast
    Zoho - QEngine
    Reflective - Stress Tester
    Minq - PureLoad
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  5. #5
    Senior Member
    Join Date
    Oct 2006
    Posts
    164
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    You can definitely give it a try with Radview's WebLOAD. In my opinion, the main advantage of WebLOAD is its object connectivity (can easily integrate a COM or Java objects in the script) and their customer support also really good. .
    Even in the case of Loadrunner, we can connect to only one object at a time based on the license we have (with C Vuser - COM object connectivity / with Java Vuser - Java).

    PS: I am not associated with Radview.
    Morty

  6. #6
    Member
    Join Date
    Jul 2009
    Location
    India
    Posts
    110
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Hi Corey,

    I had worked in Adventnet QEngine for Web Performance Testing. It addresses most of the features that LoadRunner does in terms of correlation, IP Spoofing, Distributed load Generation.

    It has good reports and Graphs.

    I hope ver 6.8 has authentication - NTLM features also built in.

    Support is too good and the tool is really easy to use.

    Thanks,
    Vivek V
    Cheers,
    Vivek

  7. #7
    Member
    Join Date
    Jul 2009
    Location
    India
    Posts
    110
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Perhaps, you can try PushtoTest TestMaker. This is Opensource.

    Though i havent started my evaluation on it, which i plan to do in a few weeks.

    Thanks,
    Vivek V
    Cheers,
    Vivek

  8. #8
    Member
    Join Date
    May 2009
    Posts
    51
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Good work guys....

  9. #9
    Junior Member
    Join Date
    Jul 2004
    Location
    UK
    Posts
    23
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Just wondering - Why have you eliminated Neoload from your evaluation? It is comparable in price to Webload.

  10. #10
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Perf Tool Evaluation - lower end proprietary tools

    Webload is completely out of my evaluation, based on price alone. Very expensive for what you get. I was quoted nearly $100k for a 5000 VU license!

    If Neoload is in the same ballpark, then it would be out also.
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.38%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 02:23 PM.

Copyright BetaSoft Inc.