Response Time - Interpretation
I'm new to the practice of performance testing. I'd like to understand how you'd interpret Response Times reported by LoadRunner. For example, I'd like to simulate a 100 user test in a web environment in the following way:
--Login (1 Iteration)
--Transaction 1 (5 Iterations)
--Transaction 2 (5 Iterations)
--Logout (1 Iteration)
--Random Think times between iterations
Now my assumption is that, as I increase the number of iterations for Transactions 1 & 2, I'd expect to see a longer response times. Conversely, the fewer the iterations, the faster the application responds.
I'd like to understand if the number of iterations has any co-relation to response times reported by LoadRunner. I'm trying to satisfy a user requirement that reads "All transactions must complete in < 4 seconds". What would be the best way to address this requirement? Thank you.
Re: Response Time - Interpretation
Though the number of iterations have an effect on the transaction response time, it is not as simple and straightforward as interpreted by you.
Let's see. Why do you think that the response time should go higher when you increase number of iterations?
There are factors such as database level caching, connection pools etc which come into picture when you increase the number of iterations. Actually we increase number of iterations to get a better representation of average response time. We also need to take care of avoiding any abnormal caching caused by data reuse.
Cognizant Technology Solutions.
while traveling parallel to a tangent , it's difficult to be normal, isn't it?
Re: Response Time - Interpretation
What is also very important in load testing is simulating a realistic number of actions per unit of time, while also simulating realistic concurrency.
Think time, session duration and inter session duration are all very important in achieving a realistic simulation. I have included a simple, but detailed example as follows:
If you are simulating:
Logins: 1800 per hour,
Transaction A: 3600 per hour &
Transaction B: 9000 per hour
Average Session duration 2 minutes
Then one could do the following for each iteration:
Login Wait B wait B wait A wait B wait A wait B wait B wait Logoff
Which would give the correct ratio of transactions to logins.
Further more, if each “Wait” is a random amount of time averaging 13 seconds, (eg from 5 to 21 seconds) and each transaction averages around 2 seconds, then the average session duration would also be correct, at around 2 minutes.
As each iteration for each virtual user would run for an average of 2 minutes, but could run as much as three minutes, I would set up the iteration pacing to average 4 minutes per Vuser. Eg Random from 210 to 270 seconds, as this would also permit slower transaction response times for Login, transaction A and B. (if response time blew out to 10 seconds for login, tran A & B then average session duration would also blow out to from two minutes to three minutes. If an iteration needed to start each two minutes to achieve the required level of load, then a session time of three minutes would mean that the load applied would be less than required.) So by allowing extra time for each iteration, the level of load applied is directly proportional to the number of VUsers running.
So with an average of four minutes per iteration, each VUser would run an entire sequence (Login, transactions & Logout) 15 times per hour. To simulate 1800 per hour, you would then need 120 VUsers.
Now back to your original question. If you squashed more activity into each session but maintained the same number of sessions and the same average session duration, then you would be applying more load but no more session concurrency. You may then see an increase in response time, because more work needs to be done, and there is more contention for system resources such as CPU and DISK I/O.
For more information you may like to check out the information under 'types of tests' at my company's web site, particularly the "load testing" and "performance testing" pages. There is also a discussion on 'response time' at loadtest.com.au/Terminology/ResponseTime.htm that would be relevant.
[ 08-16-2003, 06:13 AM: Message edited by: Paul McLean ]