SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 7 of 7
  1. #1
    Junior Member
    Join Date
    Nov 2003
    Location
    USA
    Posts
    12
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Internal vs. external users

    I am nearing the completion of a lentghy performance testing cycle and have been attempting to run an "endurance" test of 72 hours of normal workload.

    In our senario we have internal users that sign on and work all day and external users that sign on and perform one business function and then sign out. Scripts for the two users types are identical with the following exceptions.

    1. The logon and logoff are in action() for the external users and in vuser_init() and vuser_end for the internal users.
    2. The external users have "Simulate a new user on each iteration" and "Clear cache on each iteration" checked while the internal users do not.

    During the "endurance" test we were seeing Java heap management issues and concluded that the test was not realistic since a user would not stay logged on for that long. To correct that we added logic to the beginning of Action() of the internal user scripts as follows.

    if (0 == strcmp("Endurance",lr_eval_string("{type}"))) // Is this an endurance run?
    {
    if (0 == (atoi(lr_eval_string("{Iteration}")) % 200)) {
    logoff_act();
    web_cleanup_cookies();
    web_cache_cleanup();
    lr_think_time(10);
    uidLogin();
    lr_think_time(10);
    }
    }

    The number of iterations was chosen for each script so that it would log off approximately every 3 hours. With that code added the users would, shortly after the first and every cycle of loggning off and on, begin getting a log in page returned to every request as if the session cookie was lost or the load balancer sent them to a different server. In thinking about what "Simulate new user ..." would do I concluded that in addition to cleaning up cookies and clearing cache it would close the connection. Keep alive timeout was set to 60 seconds so the 10 second think time would not allow it to expire. I then chenged the code as follows.

    if (0 == strcmp("Endurance",lr_eval_string("{type}")))
    {
    if (0 == (atoi(lr_eval_string("{Iteration}")) % 200)) {
    web_disable_keep_alive ();
    logoff_act();
    web_cleanup_cookies();
    web_cache_cleanup();
    lr_think_time(10);
    web_enable_keep_alive ();
    uidLogin();
    lr_think_time(10);
    }
    }

    With the addition of the disable and enable keep aslive around the logout the problem disappeared and our endurance test has been running for nearly 20 hours with very normal heap management. This is about 15 hours longer than ever before.

    I am posting this to:

    1. Ask if anyone has a better way to accomplish this.
    2. Share our technique in case others may run into the same issue.

    LoadRunner Profile
    1. What version of LoadRunner (LR) ? 9.52
    2. What is the protocol you are recording? HTML-Web
    3. If HTML - are you using HTML-Advanced with URLs HTML - yes
    4. URL mode with Concurrent groups, or No
    5. URL mode without Concurrent groups? No
    6. Which LoadRunner/PerformanceCenter feature or service packs are you using? SP2
    7. What kind of LoadRunner license do you have - Subscription, Perpetual, Virtual User days, or Evaluation? Perpetual
    8. If subscription, is subscription current or expired? Current
    9. What platform(s) and Operating Systems? Include version and service packs. Windows 2003 Server with SP2
    10. If you have filed a service request with HP/Mercury, what have they told you at this point with respect to your issue? no
    Barry C. Plimpton

  2. #2
    Moderator
    Join Date
    Feb 2010
    Location
    Europe
    Posts
    944
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    the code is not a bad idea, I've done something similar in the past, but significantly more sophisticated with allowing for users breaks and switch over between user populations in different timezones (& no I haven't got the code) - hint it was all done via think time with 0 pacing

    however, I have some questions

    do your users really work 24 hours a day? I strongly suspect that they don't

    do your users really perform the business process 200 times in 3 hours? this strikes me as extremely fast each iteration taking less than 1 minute - this is not representative of a real user

    do your users really only wait 10 seconds between clicks? again I think this is way too fast - doesn't really allow for reading of the page let alone any thinking

    I suspect that you are increasing the number of iterations (and reducing think time) in order to 'approximate' more users than you have licenses for - this in many ways fails to do what you are intending and results in a completely incorrect load profile

    how your application really behaves with the correct number of users will be completely different than in the test - imo testing like this is in almost all cases pointless

    for a quick pointer about this think about the sessions

    how many sessions?
    how long they exist (especially the ones that log off every iteration)
    what impact does that actually have on concurrency and therefore memory usage?

  3. #3
    Junior Member
    Join Date
    Nov 2003
    Location
    USA
    Posts
    12
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    Jim, Thanks for your reply.

    You are right that the 10 second think time is rather short. For this application the volumes are so low that in order to get enough samples to make the statistics usable we had to either run normal load tests for a full day or shorten the think times. The clients SLA's are based on a 90th percentile and with only a few samples that can be terribly misleading. I like to see the numbers at least in double digits to use the 90 percent response time column.

    We are using a realistic number of users. Some of the scripts require nearly a half hour to complete. I use think time multipliers and pacing on the scripts to get the correct ratios of business functions in a test run.

    In reality we are putting between 2 - 4 times the normal hourly load depending on the business functions.

    "do your users really perform the business process 200 times in 3 hours? " The script I pulled the sample from was about a four step script
    Search
    Click an action button
    Enter an action date and click the confirm button
    Return to the the main page
    Thus the less than a minute with the short think times.

    The endurance test is extreme in that we will simulate about 11 calendar days work in 3 days. The purpose being to detect memory leaks if any.
    Barry C. Plimpton

  4. #4
    Advanced Member
    Join Date
    Jun 2006
    Location
    USA
    Posts
    785
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    I generally do it (your internal users) via multiple action blocks. I will keep the "Simulate a new user on each iteration" Enabled. My action tree looks like this

    VUserInit
    LoginAction
    ---Block
    ------ActualAction x Iterations
    LogOutAction
    V_end

    the Actual Action iterations are calculated and user will always exit after finishing x number of iterations.

    It does save on the extra bit of coding . (no string cmp). However whether this is an endurance or not is controlled via the Scenrio file. That would save us the bit about setting the parameter before starting the test.

  5. #5
    Junior Member
    Join Date
    Nov 2003
    Location
    USA
    Posts
    12
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    Very interesting.
    I've used action blocks to simulate different user flows but never in this situation. It does save some code making the scripts more efficient.
    I'll certainly remember this if it ever comes up again.
    Barry C. Plimpton

  6. #6
    Advanced Member
    Join Date
    Jun 2006
    Location
    USA
    Posts
    785
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    One additional benefit is If I can simulate real user behaviour with action blocks without too much coding. eg..the users have two types of actions to do.. one is done 60% of time, the other 40% of time. and he usually does 15 iterations in total, then the blocks can look like

    vuserInit
    LoginAction
    ---Block (Block Level Settings - 15 Iterations - Random Access)
    ------ActionOne (Wtg 60%)
    ------ActionTwo (Wtg 40%)
    LogoutAction
    VuserEnd

    This is essentially two scripts in one.. extremely useful at doing this without too much coding. Script Maintenance is easier too..

  7. #7
    Junior Member
    Join Date
    Nov 2003
    Location
    USA
    Posts
    12
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Internal vs. external users

    Yeah, that's I have used the percentage in the past just never the number of iterations to force login and logout.
    Barry C. Plimpton

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:07 AM.

Copyright BetaSoft Inc.