SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 3 123 LastLast
Results 1 to 10 of 30
  1. #1
    Member
    Join Date
    Sep 2005
    Location
    India
    Posts
    61
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Approach in identifying think time

    Please share your approaches in indentifying think time for load test. and also share about the Do's and Dont's regarding the same.

  2. #2
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    The 4th topic from the top of this forum has links to resources holding the answers to your questions. There is a wealth of information contained within this topic as well as links to other wealth.

    http://www.qaforums.com/ultimatebb.p...c;f=2;t=000901

  3. #3
    Senior Member
    Join Date
    Sep 2004
    Location
    Riga, Latvia
    Posts
    611
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    As far as I know there are two ways:
    1) Record actual user activities and get average. Problem – person should be familiar with the system already, doesn’t work if system is new one.
    2) Analyse the amount of data rendered on user interface by last transaction. Issue – you don’t know what part of info rendered user are interested in. It depends. E.g. user may sometimes requested incorrect information and figure it our in no time (even while request is processed) so reducing think time to 0.
    ?:the art of a constructive conflict perceived as a destructive diagnose.
    Ainars

  4. #4
    Senior Member
    Join Date
    Apr 2003
    Location
    Wisconsin, USA
    Posts
    5,338
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    Per comments on another thread, are we talking THINK TIME here, or PACING? Quite different animals. Not interchangeable.

    Think time is typically introduced when the person building the test is recording the script. In some tools, it is called "sleep time". There is no decision to make regarding how much think time is in the script. It is already there. The only decision to make is: turn it on, or turn it off. If you keep it on, the only decision is: do you accept the original think times introduced during recording, or do you manually modify them to some arbitrarily arrived at number?

    Now, if we are talking pacing, that is infinitely variable and very controllable.

  5. #5
    Member
    Join Date
    Sep 2005
    Location
    India
    Posts
    61
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    Darrel - Its about think time and not about pacing. As of me i never keep think time as something which is already there during scripting. I always try to modify it to something which happens in the real time.

    You can average a think time based on the statistics that are suggested by the business users. I generally dont keep the ones recorded during recording. Do u?

    To be precise, i wanted to listen to various approaches in arriving at that arbitary number.

    Ainars - As of the approaches pointed by you

    1) I dont think recording the think time by an user who knows the system will be good. In my views I would suggest that it should be an average (rather a percentile) on the think times taken by an user who knows along with the think times of an user who is unfamiliar with the system.
    2) I guess this is specific to form posts specific transactions.

    I can understand the issues invloved but would like to know the practical approaches in indentifying the think time.

  6. #6
    Senior Member
    Join Date
    Apr 2003
    Location
    Wisconsin, USA
    Posts
    5,338
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    Yes, I do keep the think times that are recorded. We usually have a knowledgeable user recording the script under normal circumstances, so the think times are what a real user will be introducing. I will ask you - you said that you modify think times to something more "real" - why are the times that are recorded not real?

    In my experience, changing think times inside a script has very little effect on the load test. What DOES have an effect is going with think times verus going with no think time and slamming the servers. That is a very telling set of tests to run.

    In the world I am in, going to the effort of analyzing what think times should be and adjusting them has very little return on the investment of time. To prove that, we did a variety of tests with varying think times. The differences in performance were in the range of 10ths of a percentage. Not significant enough to warrant time to mess with it. That is why we just go with the recorded times.

    There is an old adage - measure it with a micrometer, mark it with chalk and chop it with an ax. Is that what we are doing here? Just food for thought.

  7. #7
    Senior Member
    Join Date
    Jun 2000
    Location
    Charlotte, NC, USA
    Posts
    617
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    D.D. Your last response is correct. The actual amount of think time is less significant than whether it is used or not. I found it interesting that you get an actual user to record the scripts. That is great when you can get it, and would provide reasonably accurate think times.

    Another option is to use standard page latencies identified by a usability study, which can predict an accurate think time via a formal process. This can be done on a page by page basis, or overall. Clearly, if that is the only reason for doing a usability test, don't. But the option is there and worth the small amount of time needed to cut-paste actual timings into the script.

    Another note. Oftentimes, the performance criteria are stated as transactions per second or hits per second or some such. In that case, it is possible to run scripts w/ or w/o think time, monitoring the transaction or hit rate while incrementally adding threads until the target level is achieved. In that case, using think time or not is irrelevant as long as you have sufficient licensing to support the thread count needed to hit your target.
    Resistance is futile.

    SuperK

  8. #8
    Senior Member
    Join Date
    Apr 2003
    Location
    Wisconsin, USA
    Posts
    5,338
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    Originally posted by bkoenig:
    Another note. Oftentimes, the performance criteria are stated as transactions per second or hits per second or some such. In that case, it is possible to run scripts w/ or w/o think time, monitoring the transaction or hit rate while incrementally adding threads until the target level is achieved. In that case, using think time or not is irrelevant as long as you have sufficient licensing to support the thread count needed to hit your target.
    <font size="2" face="Verdana, Arial, Helvetica">This is the point I was trying to get at in another post where I said that decreasing think time does not increase load on the server. You stated it here much better than I did. Load tests I have run are based on a combination of thread counts and hit rates, not just hit rates alone.

  9. #9
    Advanced Member LauraScharp's Avatar
    Join Date
    Aug 2002
    Location
    Huntington Beach, Ca. USA
    Posts
    725
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    Just throwing this out...

    We don't get an experienced user to help record scripts (that would be great, though!!), so we have them document the steps and think times for us. We occasionally run into a situation where we would have recorded a much lower think time than they provide, and when we question them, we find out that in real life, they would be on the phone with a customer, having to look up information in other systems, on hold, etc...

    When the difference between what we record and what they specifiy is significant (say, 2 minutes for example!), wouldn't you see the value of substituting this??
    Laura Scharp
    SoCalGal - Defender of end user response times!

  10. #10
    Senior Member
    Join Date
    Jun 2000
    Location
    Charlotte, NC, USA
    Posts
    617
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Approach in identifying think time

    LauraScharp. Your stated methodology is excellent! and speaks to the point I made previously about usability test timings, which is essentially what you're doing.

    I would question, however, the point of load testing a process that only interacts with the server once every 2 minutes. It's probably not going to be the critical performance bottleneck.

    If the primary bottleneck is suspected to be opening and closing connections and connection concurrency, then # threads predominates when generating a specified load. If bottleneck is suspected to be processing of the transaction itself, then transaction rate predominates, regardless of # threads or think time.
    Resistance is futile.

    SuperK

 

 
Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.38%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 08:04 PM.

Copyright BetaSoft Inc.