SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 3 123 LastLast
Results 1 to 10 of 21
  1. #1
    Apprentice
    Join Date
    Feb 2011
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Performance testing tools - where to start

    hi all,

    I suspect similar questions have been asked before, however:

    - I know very little about performance testing tools
    - I've been asked by my boss to investigate/evaluate performance testing tools

    Some details about what we need:

    - application we're testing runs in IE, but the UI is very much a 'cut-down' browser window - ie, it basically just has a toolbar with back and forward buttons, a Home button, a search box
    - the application can also be run under the 'normal' IE interface
    - we're going to need to simulate say 10 to 20,000 users using different functions

    I'm just not really sure where to start, but I guess some tips on say one to three tools I should consider would be great.

    thanks in advance for any help

  2. #2
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    Aside from tools, do you know anything about performance testing itself?

    Do you have a budget?

    Are these 10 to 20,000 simultaneous users?
    If not, how many do you expect to be using your system at the same time?
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  3. #3
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,018
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    20,000 users? Hire someone who has been there, done that, been awarded the T-shirt and lived to tell about it. The financial risk and technical acument required at the higher level are non-trivial and demands a degree of maturity in the profession. This provides you an opportunity to shadow and learn as well (keep in mind that a full on mentoring would lengthen the engagement time)
    James Pulley

    Replace ineffective offshore contracts, LoadRunnerByTheHour. Starting @ $19.95/hr USD.

    Put us to the test, skilled expertise is less expensive than you might imagine.

    Twitter: @LoadRunnerBTH @PerfBytes

  4. #4
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    "A fool with a tool is still a fool"
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  5. #5
    Apprentice
    Join Date
    Feb 2011
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    hi

    thanks for the responses guys.

    yes, I agree entirely, this is a job for a performance testing expert, not me.

    for some weird reason, my boss wants me to do this, and I have no experience in performance testing. He's well aware of this, and he's known me for some time. He does have some experience in performance testing himself, but not much. But I would've thought he should know enough to know that this is a job you don't give to someone with no experience in the area. It's little wonder I'm a bit lost.

    I've also been thinking that the best thing they could do is hire someone with the right skills/experience - a consultant, if you like. However, such a person would be expensive and this company is currently having financial problems, and they're not hiring anyone right now.

    I'm now tasked with trying to choose a performance testing tool to use to do a highly complex testing exercise, with little knowledge or experience. I guess all I can do is my best to try to do what the boss wants, but I don't understand where he's coming from, really.

    I'm not sure of the budget, but my boss has told me he has a large bucket of money that he's been given to spend on a tool, and that he needs to spend it fairly quickly...

    Actually, as it turns out, we need to simulate about 4,000 users, not 10 to 20,000. And yes, these are simultaneous users. The app is used globally by some very large companies, so we do have pretty large numbers of users online at one time...

  6. #6
    Member
    Join Date
    Sep 2001
    Location
    Sunnyvale, CA, Santa Clara
    Posts
    394
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    "my boss has told me he has a large bucket of money that he's been given to spend on a tool, and that he needs to spend it fairly quickly"

    DarkStar9999,

    Let me welcome you, your boss, his bucket of money and your question to the forum.

    If the app under test is reached via http (and your comment that the application can also be run under the 'normal' IE interface implies this), then if you have some programming experience I would look at the Grinder Open Source performance tool first.

    http://grinder.sourceforge.net/

    If you do not have programming experience but are somewhat technical then I would look at the Jmeter Open Source performance tool.

    http://jakarta.apache.org/jmeter/

    If you can create and playback scripts from either of these tools with simple requests against your application, then I can offer further assistance. It would be helpful for me if you could let me know the exchange rate between US $ and an Austrailian Bucket of money, in case I can be of further assistance.

  7. #7
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,018
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    Do not give the job of a financial audit to someone who has a cursory view of bookkeeping. Your boss is doing you and the organization a great disservice. Your boss is welcome to view the horrors of questiosn here from people in similar challenges and even ask what path management might take to reduce risk. This path is the riskiest of all. A tool will not solve a process or a skills problem.
    James Pulley

    Replace ineffective offshore contracts, LoadRunnerByTheHour. Starting @ $19.95/hr USD.

    Put us to the test, skilled expertise is less expensive than you might imagine.

    Twitter: @LoadRunnerBTH @PerfBytes

  8. #8
    Moderator
    Join Date
    Jun 2003
    Location
    Hollis, NH, USA
    Posts
    1,551
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    The current exchange rate is 1 AUS bucket of cash = 1.01 US buckets of cash.

    If you are interested in doing a test just once, you should consider out sourcing it. If you need to do this periodically going forward, consider open source tools, but above all, consider someone who can do it for you and then do a knowledge transfer to your staff. Its almost always the best mix of "getting things done right quickly" and "building the skills you need to do this yourselves over time".

    Open source generally tends to be more labor intensive but more flexible vs new protocols. I'm using OpenSTA with a silverlight application that requires NTLM right now, and frankly I am finding it easier to use than load runner. Thats not always the case (rarely actually), but don't assume that more expensive tools will necessarily be less labor intensive or easier to use until you have done an evaluation in the environment you will be testing in.

  9. #9
    Member
    Join Date
    Feb 2001
    Location
    Raleigh, NC USA
    Posts
    84
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    darkstar9999,

    Based on what you've said, including budget, timeframe and experience level, you should probably be looking for a tool oriented at beginners. Naturally, I'll put in a plug for our product, Web Performance Load Tester. But there are a number of other tools that offer a high level productivity for those without a lot of performance testing experience. A good number of the available tools (including many/most of the open-source tools) have a pretty steep learning curve. I recommend picking a test scenario that is typical for your testing goals and try it out with a variety of tools - see which one gets you up and testing withing a reasonable amount of time.

    You might consider outsourcing your first testing effort. You'll learn a lot about the process by working with an experienced vendor. We've had a number of customers to went this direction and then ended up buying our software and performing the tests themselves after they gained some experience with the process.

    Hope that helps!
    Chris
    Chris Merrill
    Load Testing Software and Services
    http://www.webperformance.com/

  10. #10
    Apprentice
    Join Date
    Feb 2011
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Performance testing tools - where to start

    hi guys

    thanks for the responses/advice

    What Chris (merrill77) says about getting a tool that's easy for a beginner to use seems to make sense, given that I am a beginner [img]/images/graemlins/smile.gif[/img]

    it seems that the scenarios we need to run are not complicated. What is a bit of a mystery to me is how we actually run them.

    I feel I'm groping in the dark a bit but my thoughts are to get a tool that enables me to fairly easily record and playback some simple scenarios and simulate a load of say 4,000 users. Since I'm not really a coder, and I don't think the scenarios we need to run are all that complicated, my feeling is that simple record and playback should be OK.

    A further thing is that my boss thinks that we should have thousands of instances of IE running, each with a different logged-in user. My feeling is that all we should really need to do to simulate a load of thousands of users is to send thousands of HTTP requests to the server (without actually running thousands of instances of IE each with a logged-in user).

    My strong suspicion is that this is how load testing is usually done - ie, you simulate browsers hitting the server with HTTP requests, but you don't have thousands of browser instances actually hitting the server.

    My boss's idea is to have say 10 client virtual machines each with say 200 instances of IE running, each with a different logged-in user. He says he wants this because our customers won't accept simulated users - they'll want to know that we've tested using thousands of actual logged-on users.

    However I'm not sure that this is feasible, even with client machines with large amounts of RAM. I have put together a VB script program that will open any number of IE instances, and login to each with a different user. But once I get up to around 100 or more instances of IE, I seem to start getting strange behaviour, seemingly due to the machine running low on RAM (eg, IE windows don't display properly or the script starts to not run properly due to the machine slowing down).

    I suppose my key question is, is what my boss is asking for feasible? Is this the way load testing is generally done? Or is the general practice just to simulate thousands of users hitting the server with HTTP requests?

 

 
Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.38%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:24 AM.

Copyright BetaSoft Inc.