## User Tag List

Hi,

Let's assume that there is a website that needs to be tested. The customer tells you that there'll be 30 000 - 50 000 visits a day. And on average, each visit would be 8 page loads (8 clicks, links, whatever).

And, of course, the load is not even throughout the day (we people have strange habits of surfing websites at same timeperiods [img]/images/graemlins/smile.gif[/img]).

But anyway, let's create a dumb scenario that is not very good example. Let's assume that:

- 1 day has 10 active hours
- within 10 hours 50 000 visits should be simulated
- user spends time on website average of 300 seconds (making 8 requests)

Within 10 hours we could generate 120 visits with 1 virtual user (assuming that 1 visit is 5 mins... 10h / 5min = 120). According to this math, we'd need 50 000 / 120 =~ 417 vus to make this happen.

So, the question is... FINALLY! [img]/images/graemlins/smile.gif[/img]

How often have you been facing a situation that you have - let's say 500 virtual users available - but the customer asks much more than that?

Basically if I'd so script that was LOT'S faster than normal user (reduce all think time to minimum) - it generates much more load to the system than normal user. In another words... I might be able to bring the system to it's knees with 500vus although the system is normally capable of handling 10 000 users (who, of course, have quite long pauses between clicking any links, submits,...).

So, is there any "rule of thumb" how to prove that the system is ok even though I didn't have enough vus available? Of course, it's not the same... but I still think that fairly good load testing can be done with lesser amount of virtual users...

If you have any stories or opinion - I'd be interested to hear about them! [img]/images/graemlins/smile.gif[/img]

just be careful if there is session/state involved.

lets say for example that you have 50,000 unique visitors within that 10 hours. and each user creates a session that doesn't timeout for 12 hours. by the end of the test, your servers would be managing 50,000 active sessions, vs. 500 for the way you are suggesting. it is possible that these scenarios could produce very different performance characteristics.

for a stateless application, what you are suggesting would work... you could simulate similar throughput and that is all that matters.

-Corey

Depends on how session state is managed. If you are passing IDs as part of the request, you can easily make those 500 VUs look like 50k.

Might be a stupid question, but...

If I repeat 1 VU for 100 times, can't that 1 VU simulate 100 unique users - who just happen to use the system in different times?

With this logic, if I'd have 500 VUs that'd all repeat the script 100 times - simulating new user with each iteration - that'd make 50 000 unique users?

At least LoadRunner has this "simulate new user in each iteration"-thingy. Don't know about other tools - especially if I decide to use OpenSTA or JMeter (although I can't use JMeter because it's a crazy memory-eater).

---

And - if you want to punch me virtually for asking this, feel free - but how does the system know it's the same user accessing the system again when I run 1 VU for many times. I mean, if I login and logout with that user?

[ QUOTE ]
Kuka: If I repeat 1 VU for 100 times, can't that 1 VU simulate 100 unique users - who just happen to use the system in different times?

[/ QUOTE ]On the basis of this statement along and without regard to the system-under-test and all of its authentication and state-maintaining mechanisms – no.
.
.
[ QUOTE ]
Kuka: With this logic, if I'd have 500 VUs that'd all repeat the script 100 times - simulating new user with each iteration - that'd make 50 000 unique users?

[/ QUOTE ]Same as above – no.
.
.
[ QUOTE ]
Kuka: At least LoadRunner has this "simulate new user in each iteration"-thingy.

[/ QUOTE ]This feature alone does not guarantee that the VU is unique to the system-under-test.
.
.
[ QUOTE ]
Kuka: And - if you want to punch me virtually for asking this, feel free

[/ QUOTE ] [img]/images/graemlins/smile.gif[/img] There is a fee for rendering punches, otherwise non-violence is promoted.
.
.
[ QUOTE ]
Kuka: - but how does the system know it's the same user accessing the system again when I run 1 VU for many times. I mean, if I login and logout with that user?

[/ QUOTE ]If you login and logout, you have changed the dynamics of your preceding concerns. However, that alone does not guarantee the VU is unique to the system-under-test. There are many factors that come into play to determine this. A short list of factors is as follows:
1. HTTP session containers.
2. The actual implementation of Session State preservation
3. Session Timeouts
4. Is the VU coming from the same IP address AND the VU has not generated a new sessionID AND is the system-under-test being load balanced by a specific method WHERE all these factors together have inadvertently (to you and the VU) caused the VU to use the same session?
5. Pacing of the VUs and playback settings.
6. And so on.

The big caution here is in the below example and many performance tests violate this basic concept. It goes back to basics - one must consider the objectives of the performance test.

<font color="blue">Example objective/test:</font> An objective is to attempt to model a-day-in-the-life AND measure transactions AND determine pass/fail. Using your above implied test parameters, one could theoretically and easily depart the path of "a-day-in-the-life" from a system perspective. How so? By driving up the number of http session containers to an unreasonable number not normally encountered in the system-under-test during a normal day-in-the-life. This condition would certainly produce false results - even if one has modeled user abandonment. The struggling-to-keep-up state of the server will most certainly ripple into the system architecture.

This example is almost as common as an untrained performance tester (very ungood of course) simply conducting a test to kill the system without knowing why the specific test is needed! This is a classic case of not having clearly defined objectives. The result is just another bad performance test with false results and lots of ensuing and unnecessary work! It is sad that this type of thing as represented by these two latter concerns, runs rampant.

whew....this is a tough one. I'm trying to think what I would do in this situation. I guess I'd first look at what type of failures that could happen.
One test-type you can utilize is volume tests.....take those 500 users and throw a for/do loop around the actions and see how the app/server responds to tens of thousand of requests in a row.

this is honestly all I can think of though...make sure your boss knows the risk that is being taken.

unless of course you could find some way to say the unveiling of the new site is some sort of "sneak-peak" but what you're really trying to find out is if the site can handle the traffic. this is stretching it though.....

So, let me put all of this in a nutshell:

If application is
A) stateless
B) generates separate sessionID every time I login to the system (Like Mercury's example Webtours app, which stores the session into a cookie)

...I can youse fewer virtual users to simulate more "real users".

How ever - if the users tend to go to the same SessionID - this won't work.

So, how can I be sure that new VU from same load generator will get new SessionID? If I don't have IP spoofing available - for example.

If the new VU is a different thread - will it get new sessionID OR do I have to have IP Spoofing enabled to make it 100% sure that no VUs is using the same session id?

All of it depends on your app under test. IF you know your app well, and IF you are sure it tracks no session information, and IF it does not load balance by IP, then you can run the test with 1 VU if you can drive it hard enough. But those are a lot of big IFs. If you are not sure of any of the above, don't risk it.

... and you should check with your system administrators to determine the requirements for a unique session along with how it is maintained. It is also good to understand the questions about performance that your customer wants answered - whether directly from your customer or from a technically skilled representative of the customer.

Is there a database server involved?

Wouldn't the connection pool to the database be likely to be utilized differently for 10,000 VU than for 500 VU?

This or course depends on the architecture implementation.

Page 1 of 2 12 Last

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.