SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 18
  1. #1
    Junior Member
    Join Date
    Feb 2006
    Location
    Lansing, Michigan
    Posts
    26
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    any strong opinions on Web load testing (for magazine column)?

    All,

    I'm working on my STP Magazine Best Practices column and thought I'd check in here. The topic du jour is Web load testing. I'm wondering if there any strong opinions out there on the right and wrong ways to do Web load testing?

    I'm looking for folks who wouldn't mind being quoted in the column, though I'm happy to be vague in attributions or even take stuff off the record if that's your preference. I guess one thing I'm curious about: say you've run your Web site for years on expensive and well-tested hardware and software. Now you're looking to move to low cost, commodity servers and possibly more open source software. How do you start?

    Thanks for any help on this. Deadline is the end of next week.
    Geoff Koch
    Freelance technical writer, journalist
    Lansing, MI
    AIM: geoffinmichigan
    Yahoo: geoff_pdx
    MSN: geoffarnoldkoch

  2. #2
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Geoff,

    You would do better in the performance testing forum. Report your own post to the moderator and request it be moved.

    In many ways, answers already exist to your questions.

    "RSBarber" - Scott Barber & "rstens" - Roland Stens have excellent material at their sites. Here are some links (and embedded links) to get you to their sites from which you may intimate the answers:

    http://www.qaforums.com/ultimatebb.p...c;f=2;t=000901

    http://www.qaforums.com/ultimatebb.p...c;f=2;t=002069

    Originally posted by Geoff K. in Michigan:
    The topic du jour is Web load testing. I'm wondering if there any strong opinions out there on the right and wrong ways to do Web load testing?
    <font size="2" face="Verdana, Arial, Helvetica">By web load testing, I will assume you mean performance engineering and testing!


    How do you start? Here is another link that talks in general terms. This site (my own) is still under construction and has links to some excellent material:

    http://perform-testing.com

    Originally posted by Geoff K. in Michigan:
    I guess one thing I'm curious about: say you've run your Web site for years on expensive and well-tested hardware and software. Now you're looking to move to low cost, commodity servers and possibly more open source software. How do you start?
    <font size="2" face="Verdana, Arial, Helvetica">[briefly_on]
    Are there studies that show this change will lower costs?
    How old are the servers?
    What is the cost of keeping them versus changing?
    What are the risks of keeping the old servers? Is the MTBF low on any component(s)?
    [briefly_off]

    Originally posted by Geoff K. in Michigan:
    Now you're looking to move to low cost, commodity servers and possibly more open source software. How do you start?
    <font size="2" face="Verdana, Arial, Helvetica">[briefly_on]
    Clearly stated organizational and performance objectives.
    What are the precise performance gains being sought?
    How will you know you have achieved them?
    List of problems you are trying to solve.
    List the risks - hardware, software, vendor, support and so on and so on...
    What is the cost of the undertaking?
    What is the expected downtime and associated costs?
    How reliable are the low-cost commodity servers?
    What are the known limitations of same?
    [briefly_off]

    How else do you start? By not doing what this person says:
    http://techrepublic.com.com/5100-351...ml?tag=search#


    Quoting me? "JakeBrake, big noise maker from large insurance firm in the upper midwest USA."


    Originally posted by Geoff K. in Michigan:

    Thanks for any help on this. Deadline is the end of next week.
    <font size="2" face="Verdana, Arial, Helvetica">Future solicitations? You would do better by having more lead time.

    [ 07-05-2006, 05:10 AM: Message edited by: JakeBrake ]

  3. #3
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Geoff,
    can you do a search on the forums so you can see the responses from when Esther asked the same question? she was writing the exact same article.. for the exact same magazine... maybe a year or 2 ago. why again?
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  4. #4
    Junior Member
    Join Date
    Feb 2006
    Location
    Lansing, Michigan
    Posts
    26
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Thanks to both of you. And ack, Corey, I don't know why we're revisiting the same topic. I think it's possible we might cycle through many of the same topics on this list annually: http://stpmag.com/editcal2006.htm. I'll do some more homework before I drop back in. Sorry to bug you guys.
    Geoff Koch
    Freelance technical writer, journalist
    Lansing, MI
    AIM: geoffinmichigan
    Yahoo: geoff_pdx
    MSN: geoffarnoldkoch

  5. #5
    Moderator
    Join Date
    Aug 2000
    Location
    Vancouver, BC, Canada
    Posts
    1,189
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Corey,

    This is the problem with many magazines and conferences as well. They just cater for the newbies as this typically is a larger market.

    This was one of the main reasons that WOPR started and even there we are having to deal with similar talks and topics being repeated. But the format lends itself quite a bit better for having very interesting discussions and coming up with very interesting ideas.

    Don't get me wrong, I do think that there is a place for STP and their conferences, but what was the last time you picked up something there that you were not aware of before or could not have figured out with 15 minutes? For us as experienced performance testers there are very few possibilities to learn more. Most of the time we have to invent it ourselves.

    Similar with their conference: it's good for networking and seeing my presenter buddies in action, but a big learning experience? I think not.

    One thing that I am really disappointed about is the rehashing of old topics. It really shows that content (with all due respect to the authors) is not all that important, it is merely the framework on which advertising can be hung.

    I happen to know that the going rate for the articles in STP is a bit on the low side as compared to some of the other opportunities out there. This automatically means that you are either a professional writer cranking out tens of these articles per week or you are using this medium to seek attention for yourself and your ideas. Either way, this is not an environment that is conducive to improving the art and science of performance testing.
    Roland Stens

  6. #6
    Junior Member
    Join Date
    Feb 2006
    Location
    Lansing, Michigan
    Posts
    26
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Thanks for the thoughtful, constructive post, Roland. I guess I'm not really in a position to speak for the BZ Media folks as I'm really just a freelancer. Personally, I can say that I've been trying to carve out a second career of sorts as a writer, and the column has been a decent step for me toward that goal. It keeps me writing and learning about software testing. From my questions, you can probably tell that I'm fairly green on the topic. I will say that I know I think it's common around the magazine industry -- for both trade and mass market publications -- to have standing features or sections. I'm still thumbing through the July 3 Time Magazine, for instance, which features Teddy Roosevelt. It's the magazine's fifth annual The Making of America issue. Maybe that's not the best comparison, but I hope you understand my intent. Repetition is okay. What's not okay is for me to end up rehashing something that Esther did two years ago. That would be schlocky, lazy journalism. The key is a fresh take on the issue, and so on. Anyway, enough of a ramble here. Just wanted to say that I've always appreciated the help I've received here. I've found folks to be patient with my questions and the site generally to be just a fantastic resource. I think I need to get more facile with how I use the search feature.
    Thanks again,
    Geoff Koch
    Freelance technical writer, journalist
    Lansing, MI
    AIM: geoffinmichigan
    Yahoo: geoff_pdx
    MSN: geoffarnoldkoch

  7. #7
    Moderator
    Join Date
    Sep 2001
    Location
    Boston, MA
    Posts
    4,348
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Geoff,

    I think this is a great resource for you and I'm glad you drop in to ask questions.

    I was just hoping to see the mag get into some more advanced topics.. rather than every year running similar content.

    let me guess your next article.. database load testing?
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  8. #8
    Senior Member
    Join Date
    Sep 2004
    Location
    Riga, Latvia
    Posts
    611
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    let me add some argues. I believe web architecture is changing over years improving the UI richness: html-&gt;jsp/php-&gt;ActiveX/Dynamic content. My company is obsessed with idea of rich client-side UI this year (which enable more information to be available for user without submitting additional request to the server).
    This changes performance testing both logical and technical. We have less frequent requests to server but larger and more complex result-sets with option to cancel request from client. Validation and much more happens on client side. It is possible to have background/asynchronous client-server interaction while user is able to synchronously work in his UI.
    Not to mention the new web-start technology which we use to replace browser with complete JAVA UI.
    Also the performance test tools and even the approach itself seems to hardly address the new architecture, where client-side performance could be an issue, where it is hard to determine when the request is finished and capture-playback becomes not-so-simple.
    ?:the art of a constructive conflict perceived as a destructive diagnose.
    Ainars

  9. #9
    Senior Member
    Join Date
    Sep 2004
    Location
    Riga, Latvia
    Posts
    611
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Just revisited the topic and begin to agree with Roland
    Originally posted by Geoff K. in Michigan:
    say you've run your Web site for years on expensive and well-tested hardware and software. Now you're looking to move to low cost, commodity servers and possibly more open source software.
    <font size="2" face="Verdana, Arial, Helvetica">When I’ve been in likely projects there was always an option to run both old and new one system somehow in parallel. As a result the only significant issue was data migration between two of them (which should by done dynamically all the time).
    ?:the art of a constructive conflict perceived as a destructive diagnose.
    Ainars

  10. #10
    Senior Member
    Join Date
    Apr 2003
    Location
    Wisconsin, USA
    Posts
    5,338
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: any strong opinions on Web load testing (for magazine column)?

    Geoff,

    Here are some generic points that I put together a couple 2 or 3 years ago that are still quite applicable. I use these as a basis for any presentation that I make on load testing.

    Load testing issues and considerations

    1. Load testing should not be contemplated before the application is stable and functional testing is complete.

    2. Load testing should be conducted in a hardware environment that is as close to production as possible. If possible, it is ideal to conduct the load test in the production environment prior to the application going live for the first time. Hardware setup and configuration has a HUGE effect on results of the load test. Load test results are generally not subject to extrapolation, since performance is rarely, if ever, linear.

    3. Interview users of the system to gather the following information:
    a. Functionality that they use
    b. How often they use it
    c. Peak number of users that will be on the system
    d. Acceptable outer limits for each functionality response time. Get both “desired” and “minimum acceptable” values.
    e. When the peaks occur - dates and times.

    4. Using information above, you can determine the mix of functional tests in the total load test and pass/fail criteria for the load test.

    5. It is a good idea to multiply the peak number of users that you obtained from interviews by a fudge factor to get a more realistic future peak load. Users always under-estimate their usage.

    6. Make sure that if there is system “noise” – i.e. timed routines that fire on a periodic basis – this is included in the load test. See item 3e.

    7. The load test does not have to reflect the EXACT business processes used – it can be stacked to give a very close approximation of the business mix of processes without being an exact reproduction.

    8. If the load test requires multiple protocol captures in a single session (Oracle Order Management is an example of an offender), you may have to use another tool in conjunction with the primary load tool to simulate the entire load.

    9. The first step in load testing should be iterations of a single functionality. For example, 5,10, 25, 50 users for each individual functionality. This will allow early identification of possible bottleneck routines.

    10. After iterations of single functions, build up to the total load test in steps. For example, if the peak load is 250 users, you may want to scale the entire test to 60 users, then 120 users, then 250 users, keeping the mix of functionality consistent at each level. This allows somewhat early indication of problems.

    11. Make sure that the load testing team consists of at least the following: a BSA, a tester, a test manager, a DBA and a network specialist. Problems could be found in any of these areas.

    12. Make sure that there are no “stray” users accessing the system when the load test is in process.

    13. Set up the schedule so that testers and DBAs/network specialists have priority on alternating days. Each group will want priority at all times. Testers will always want to run more tests and DBAs will always want to tune.

    14. Have (SHORT) daily meetings with the test team at the beginning of the day to go over:
    a. Activities performed the previous day and results
    b. Activities to be performed today
    c. Any conflicts or issues

    15. Data setup is by far the most time consuming part of load testing. Identify your data needs early. You may even want to have a dedicated team of people who enter/set up data for the tests. It is not unusual to burn 3 days worth of data setup in an hour (or less).

    16. If you have a good data setup prior to the first run, backup the environment so that it can be restored for the next iteration of testing. That is sometimes harder to do than you might think, since a restore may undo some code fixes. Data driven data loading scripts may get around that limitation.

    17. Realize that upgrades or patches to environments often invalidate scripts (Oracle is again a good example of an offender). Control IDs may change, requiring recapture of test scripts. Take that into account when building the schedule.

    18. Load testing data does not have to test “boundaries” like unit or functional testing. Middle of the road data is best for a load test. A load test that fails for functional reasons tells you nothing.

    19. The load testing tool may be a great scripting tool to actually do data setup and entry if the application is stable enough. Likewise your functional testing tools.

    20. Understand what metrics you want to capture when you are planning the test. DBAs want very different measurements than do the BSAs and testers. DBAs want such things as CPU utilization, file handles open, etc. Functional BSAs and testers want response times.

    21. Script out your load test cases just like you would a unit or functional test. You never know when you will have to recapture the test script.

    22. Make sure that checkpoints are identified in the test case so that they are accurately reflected in the captured script. Identify the checkpoints by a name or number so that nested checkpoints are accurately opened and closed.

    23. Level set expectations for the results of the load test before it is run. Users will want to use the exact numbers that you gather as gospel. Make them realize that any number of factors will change the results. We are measuring with a yardstick, not a micrometer. The load test is only an APPROXIMATION and a tool to help identify areas for tuning.

    24. If you have time at the end of the test cycle, keep ramping the load test to turn it into a stress test and see where the system collapses.

    And yes, you may quote me.

    Darrel

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.00%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 11:50 AM.

Copyright BetaSoft Inc.