1. ## Standard Deviation

Can someone give me an explanation or tell me where I can find an explanation of how to figure the standard deviation into our results? Also, we would like to know if there is a point in which we should start questioning the results due to the standard deviation.

Thanks!

------------------

2. ## Re: Standard Deviation

As far as when you should question the results based on the standard deviation, here is what I do:

1. <LI>Export the data to Excel
<LI>Short the data based on the Standard Deviation, from highest to lowest
<LI>Here is where it gets tricky, since it is really based on your application: The numbers that are the highest are the ones you want to take a closer look at, but only if those numbers are higher than the rest. For example, on one web application, it was common to get a SD of around 20. So, if I had SD's at around 50 or 100, I knew there was a problem.

As far as what the standard deviation is, it would be best to just look that up on google. Basically, it is a number that represents how uniform the numbers are. The bigger the number, the less uniform the numbers. So, if you had a login page, and got 1 second for it each time, the standard deviation is 0. On the other hand, if the response varied from 1 second to 20 second, then you'd have a large standard deviation. It can be looked at as the "quality" of the values retrieved.

Hope this helped.

------------------
David Genrich
david.genrich@e2open.com
david@linuxpuppy.net

3. ## Re: Standard Deviation

When you say that with a large standard deviation this could show a problem. Would you say that the problem is with the application or the way Silk Performer is collecting the results. My guess is the application because it is not sending responses uniformly or close to the average.

------------------

4. ## Re: Standard Deviation

Hi Kent,

Could be both....

Could be the application having a wide variety of response times. This could be either an application problem or it may be related to the simulation itself.

If you are using SilkPerformer's randomization features (think time, transaction order, variables, ect.) this could be sending a wide variety of traffic to the server at different times and you could be seeing the server simply responding differently to a different variety of traffic.

i.e.
You have transactions A, B, C, D
with a 100 user test each running the same transactions but with random transaction order and thinktimes you may have a situation where the server is trying to handle more transactions A+B than C+D or any combination thereof. If transaction C causes the server to work harder (or the middleware or DB) then at times when there are more C transactions being executed you could see the response times for ALL transactions to be slower. At times when there are very few transactions C being sent to the server, the response times could be faster that average.

This could cause the variations you are seeing.

------------------
James Soderborg
http://www.ameliortech.com
jamesso@ameliortech.com

5. ## Re: Standard Deviation

------------------

6. ## Re: Standard Deviation

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by jamesso:
Hi Kent,

Could be both....

Could be the application having a wide variety of response times. This could be either an application problem or it may be related to the simulation itself.

<HR></BLOCKQUOTE>

Or it could be java garbage collection...

------------------
zboss

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.