Thanks:  0
Likes:  0
Dislikes:  0

# Thread: What is the best criteria for Response time

1. ## What is the best criteria for Response time

I wish to initiate a discussion on the following.

Suppose you are conducting a Load Testing - the Load test exercise for a particular Scenario starts with 1 User and gradually goes up 5, 20, 50, 100, 200 ...

Take for eg. if the scenario is run for 1 user then the response time might be 0.25 sec.

Now. take 5 users - When the 5 user scenario is run - the simulated users may finish the task in different time say 0.10, 0.15, 0.25, 0.08, 0.30

The same goes true for 20, 50 ,... users.

What is the best criteria to decide the response time here - Is it the lowest or is it the highest or a range.

Any statistical methods available

2. ## Re: What is the best criteria for Response time

Two books I can think of: "Scaling for E-business" and "Capacity Planning for Web Performance". Here is a great link for response times: http://www-4.ibm.com/software/develo...ty/?dwzone=web

The books will give you more formulas then you will ever need. Response times are subjective.

------------------
-- Mike --

3. ## Re: What is the best criteria for Response time

In an ideal world, you would have a Service Level Agreement (SLA) or at least some reasonable requirements in order to plan what you are going to test.

These may state response times for a given fixed number(s) of concurrent users e.g. for 100 concurrent users, 90% should to experience a response time of less than 8 seconds for transaction x.

The 'percentile' graph from the analysis tool of load testing software will give the results you need. These show the percentage of transactions that were performed within a given time for a fixed number of concurrent users.

By picking a set percentage e.g. 90% or 95% of users then you can get the response time associated and prevent the average response time being skewed due to a small number of users which have relatively high response times.

4. ## Re: What is the best criteria for Response time

I think the "7 second" rule is the most commonly referred to spec.
If it takes longer then 7 seconds for a page transfer you will lose x% of your customers.

5. ## Re: What is the best criteria for Response time

I have never been a fan of the "seven second" rule (actually it is referred to as the eight second rule, but why quibble over a second) because it is too broad of a statement. Credit card authorization may take more than seven seconds but as long as the user is notified of this, users are generally willing to wait.

It also does not take into account the fact that pages have elements that download at different times. For example, if all images have height and width tags then the text will appear first before the images and thus users will generally wait. Interlaced images will also have some users wait if they can see progress of the image filling in.

From psychology it is known that ten seconds is about the limit that the person will perceptually sit there with no visual or auditory cues. And the "rule" does not take into account another known fact which is that if users expect something to take longer, they will wait. It is when they do not expect it to take longer that they get frustrated.

For example, numerous studies have shown that users with 14.4 and 28.8 will wait much longer than even ten seconds because they expect their modem speed to be slow. Cable modem users are bracketed: some expect no wait and so do leave a site quickly but the statistics show that many will wait simply because they know they are not paying for their connection per minute spent on line and so are more willing to wait. (It is actually 56K modem users that skewed the results in almost all cases.)

This may sound esoteric but it is not: that is the point of workload characterization and determining how users will use your system and its resources.

I agree with David that it is best to consider the percentages relative to a given set of SLAs, particularly the 90th percentile measurements.
Personally I employ methods that are mentioned in the books that Mike referenced regarding queuing analysis. This is, I have found, the best way to model systems that are distributed and dynamic.

6. ## Re: What is the best criteria for Response time

Here is a chart from IBM on web performance.

[This message has been edited by mracicot (edited 02-02-2001).]

7. ## Re: What is the best criteria for Response time

That doc from IBM doesn't really tell you a whole lot however because you have no idea of the setup of the server or if any application servers were being called or database servers. And does the 17 concurrent connections mean that it was 32.33 seconds for each page or is 32.33 averaged out over the 17 concurrent connections (meaning about 4 seconds per page per user). Also were the connections unique users?

8. ## Re: What is the best criteria for Response time

Good questions Kristi. Here is the entire article (link). http://www-4.ibm.com/software/develo...ty/?dwzone=web

[This message has been edited by mracicot (edited 02-05-2001).]

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.09%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 09:24 AM.