User Tag List

Results 1 to 4 of 4
  1. #1
    Senior Member
    Join Date
    Dec 2006
    Kokan, INDIA
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    Session per hour?

    My client wants the load level to be in terms of session starts per hour rather then concurrent users.
    He says that this gives more realastic behaviour (by looking at log files they have taken this conclusion)

    He wants 750session per hour.
    Now if use 750 differnt userin 1 hour test will this give the same result?

    What you mean exactly by session/hour and how its different from concurrent users?

    I use concurrent users and what is generally used?

  2. #2
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Post Thanks / Like
    0 Post(s)
    1 Thread(s)

    Re: Session per hour?

    [ QUOTE ]
    What you mean exactly by session/hour and how its different from concurrent users?

    [/ QUOTE ]

    Let's say a session lasts for 1 minute.

    If you run 1-minute sessions, starting each one only after the previous session ends, for 1 hour, you will have 60 sessions per hour. But you will only have 1 concurrent user at any time.

    Instead, if you start all the sessions at the same time, you will still have 60 sessions per hour, but (for 1 minute at least), you will have 60 concurrent users.
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  3. #3
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    St. Louis - Year 2025
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    Re: Session per hour?

    Sessions per hour? It seems you would want a given number of users with active sessions per hour. In a way, this is concurrent users where concurrent users at any given time will equal active sessions - not http connections - AND - where an active session could be someone simply logged in or connected and not doing anything but kicking back and drinking a beer until the session times out.

    IF your customer wants realistic beavior, then it seems you and your customer need to work together to define "realistic beavior" to an acceptable degree. WebTrends logs and/or DB logs can help you determine that. One must also consider the time of day, active sessions during peaks and valleys, batch processing and a whole range of many other things including but not limited to:
    1) types of transactions,
    2) session abandonment,
    3) think times,
    4) session types as user access types,

    and blah-blah-blah.

    Again, this is something you work out with your customer.

  4. #4
    Join Date
    Apr 2008
    Cleveland, OH
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    Re: Session per hour?


    I actually appreciate your client's point of view. It's more refined (and intelligent) then trying to force a relationship between vusers and real users.

    To Jake's points, every load test is a simulation of a real load. The more time you can spend developing the realism of the simulation, the more closely the load test will mimic a real production load.

    One approach is to work with your client and develop well-understood test cases. A client that is committed to supporting your effort to conduct a highly detailed simulation will be able to help you model what a production workload looks like. This looks like a significant investment in time for discovery or scripting, but is sometimes exactly what is called for.

    An approach that can support a highly detailed model for load testing can be found here: http://www.perftestplus.com/resources/ucml_1.1.pdf

    The next level there is to model the workload. There are surges and low spots in any workload. The one thing a real workload is NOT is consistent, even though a load tool's default state is to be very consistent.

    I feel that these sorts of modelling is underserved, underdeveloped, and underappreciated. I do not think it is the right choice for every situation, but the value to the business and testing of modelling an entire workload this way could be significant, even if it is not completely simulated.

    If you do not have the time and cooperation necessary to make a highly realistic simulation, you might consider an equivalency strategy. Here, you acknowledge that you are not running a production load, but that you are going to conduct experiments that can tell you about the performance and scalability characteristics of the software you are testing. Instead of trying to match the load, you try to generate one that will stress the software to similar levels. Instead of spending significant time modelling, you might make the most of the time you have to script activities in the application.

    For satisfying the requirement you mention, I might try to cover the most common functionality in the system. I would also doing what I could to pace my test in a way that is at least somewhat like how a user would use the software. Can you go watch a user on the software and at least get an idea how fast they are clicking? If not, you will have to make your best try at imagining an end user interacting with the software. For example, the user might log in, look up some value in the software, and then complete some business activity. The time spent on the business activity is the sleep time to build in here. If the activity is going to be repeated, it could be a transaction loop, or you could use your own looping logic.

    Any kind of sign-off from a stakeholder on your test model before you simulate with it can help focus their attention on designing a good workload, or at least help C your A.

    Once you have a crude "model" like this, then you could design your load. Consider a test where you have a single script that encapsulates some of the activities in the system. With your best try at realistic think times, the script takes four minutes to exercise.

    This means each virtual user can complete 15 sessions in an hour at this rate. That further suggests that if you wanted to complete 750 sessions in an hour, you could achieve an equivalent workload by running 50 virtual users executing the test script for an hour. It is 750 sessions in an hour, whether it represents a realistic workload or not.

    This is not a stopping point, this is only a starting point. There are experiments worth doing that are not necessarily equivalent any more, but they do benefit the project by continuing to reduce risk.

    My next test might be to increase the load further; if I can get 150-250 virtual users running with decent response and no errors, I would be more confident that the system was capable of supporting surges in workload. If I could log 750 on at a time, I would be ready to risk some prestige on warranting that the system could support the desired load.

    Next, I might remove all sleep time from the script, and start ramping from 1 virtual user. When response time curves up, then I stop the test. Restart the test, ramp to the near side of the threshold that caused the response time increase. Run that for a while to get a throughput at capacity figure. Transactions per second can give you a metric you can start doing math on to generate more equivalencies.

    Are these tests ACTUALLY the same thing? No, of course not. If you are close to actual usage patterns, it may be equivalent. You will log on as many times as your client wants. You may do similar numbers of certain activities. You may be way off on some, or you may miss critically expensive functionality. It doesn't mean that you have not reduced the risk of deploying the software, or that you have not validated certain characteristics of the system. For example, you have made the system manage a certain number of simultaneous users, which can provide a chance to catch threading errors, memory leaks, and other load-based challenges.

    There are MAJOR problems in trying to make the equivalent the equal, so don't try to. Consider this approach a series of experiments, and carefully document what you tested. Be forthright about what you don't know and what you didn't test, and what the risks are. Understand that you haven't guaranteed anything, but you have reduced risk.

    And that's enough rambling for one day.

    Eric Proegler
    Performance Lead
    Hyland Software



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 06:00 PM.

Copyright BetaSoft Inc.