SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 13
  1. #1
    Junior Member
    Join Date
    Apr 2002
    Location
    Helsinki, Finland
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    How are response tiems measured?

    I have a bit of an odd question....

    Consider two examples using pseudo-code:

    Example 1:
    lr_start_transaction("Prelogin");
    web_url(Page I am requesting);
    concurrent_group_start;
    web_url(downloaded resource1);
    web_url(downloaded resource2);
    concurrent_group_end;
    lr_end_transaction("Prelogin");

    Example 2:
    lr_start_transaction("Prelogin");
    web_url(Page I am requesting);
    lr_end_transaction("Prelogin");

    Now, my question is this: If I want to measure the end-user's experience which example should I use? (I would assume #1.)

    As I understand it, example 1 would measure the length of time from issuing the request until the end of all the downloaded resources being received. Whereas example 2 would emasure the length of tiem from issuing the request until the first reply from the server. However, if I try these things in vugen I see exactly the same thing in the run-time viewer. Which indicates to me, well, I don't know what... :-)

    Can anyone help me?

    ------------------

  2. #2
    Senior Member
    Join Date
    May 2002
    Location
    Scranton, PA
    Posts
    405
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    You are quite right David,

    It's as simple as this.

    Every web_url or whatever is an HTTP request to the server, to which server responds by sending an HTML page or an non-HTML resource.When the actual user does the things, all these requests apear to him as one single request that downloads everything. So if you want to measure with actual user's perspective, use method#1

    Still, I see few issues in this
    1. It is possible for both the actual user and the LoadRunner to simulate browser cache, thus not necessarily the non-html resources like gifs and jpegs are downloaded everytime
    2. The purpose of a load test , I feel, is determining how servers behaves under load, and not what feeling the end-user gets (One might contradict to this), so in that case it will be more advisable to use method#2 as it gives round trip time of exactly one HTTP request.

    ------------------
    Euc

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?
    Pankaj Bhatte
    Cognizant Technology Solutions.
    Scranton, PA
    pankajbhatte@yahoo.com

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?

  3. #3
    Junior Member
    Join Date
    Apr 2002
    Location
    Helsinki, Finland
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Thank you EUC.

    I am inclined to agree with you on your perspective towards loadtesting. The first priority should be determining where the system's breaking points are. So, the heavier load we can generate the better. (Which is why I'm generally running with cacheing off.)

    My question arose from contradictory information that I have received from an external company that is providing some hosted load for us. (They do not include anything other than the initial requests in their scripts - and insist that they are measuring the download time of the whole page. So, I wanted to see if I was completely off base with how I understand the tool works.)

    Thanks again.

    ------------------

  4. #4
    Senior Member
    Join Date
    May 2002
    Location
    Scranton, PA
    Posts
    405
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    'Client is our God', I agree fully.

    Well this kind of requirement is typically an outcome of client interested in knowing how much time his client (thatís the end user) will need to wait, when his system is under load. This is obvious. Thatís the purpose of the whole exercise we Performance Analyst do. One word 'abstraction' tell you how to work around this.
    Itís quite common for a client (who is some financial or marketing guy, to be unaware of the architectural and behavioral fundamentals of the system). Only thing he sees is, you are automating his business, and he expects ease of use and growth in profit. So give him the abstraction. If he wishes to know the response time as seen by the user, tell him that, and additionally tell him what can be done to improve on that.

    Thereís a nice feature of cascading transactions in LR 7.5, you can define parent-child relationships in transactions using functions like lr_start_sub_transaction. So you have an opportunity to keep everybody happy - the client as well as the performance analyst in you.

    One more point, I don't agree with the ideas like disabling the caching, using rendezvous points etc for increasing the load. Common, whatís the use of unrealistic things in this real world? If you want to increase the load, increase number of Vusers. Simple as that.

    Lastly in case one wishes to know good prediction of actual user response time, can it be achieved by running an (N-1) user load test form LR, and simultaneously running a test case from some functional testing tool like WinRunner for example?


    ------------------
    Euc

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?
    Pankaj Bhatte
    Cognizant Technology Solutions.
    Scranton, PA
    pankajbhatte@yahoo.com

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?

  5. #5
    Senior Member
    Join Date
    Jun 2001
    Posts
    146
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>One more point, I don't agree with the ideas like disabling the caching, using rendezvous points etc for increasing the load. Common, whatís the use of unrealistic things in this real world? If you want to increase the load, increase number of Vusers. Simple as that.<HR></BLOCKQUOTE>

    There are times when, certainly, rendezvous points are useful and should be used. I once had a test which would fail occasionally with 200 users running normally (i.e. think time, fairly random). When I ran 5 users with a rendezvous at a particular point, it failed every time. As a result it was much quicker to debug and fix for the developers.


    ------------------

  6. #6
    Senior Member
    Join Date
    May 2002
    Location
    Scranton, PA
    Posts
    405
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Agree with you Joe,
    Point I am trying to make is, unless there is a specific requirement (specific as in stated in some requiremnt spec), don't go for implanted simultaneousness.

    Its because its heavily unrealist to imagine that there are same number of concurrent and simultaneous users in a system. In real life case the simultaneous users in a system is a very small percentage of concurrent users (Suresh: no 'magic figure' here). It's my personal opinion (One might choose to disagree) that we do load testing to check whether the system works in kind of load it is expected to work when it goes live.Hence the extreme values of load should be within the realistic limits.

    There is no point of crashing a databse server by putting 10k simultaneous users when its expected to take 10k concurrent users, isn't it?

    ------------------
    Euc

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?
    Pankaj Bhatte
    Cognizant Technology Solutions.
    Scranton, PA
    pankajbhatte@yahoo.com

    while traveling parallel to a tangent , it's difficult to be normal, isn't it?

  7. #7
    Junior Member
    Join Date
    Apr 2002
    Location
    Helsinki, Finland
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Thank you for the interesting discussion.

    I'd like to reply to one comment, mainly because this an interesting topic:
    "There is no point of crashing a databse server by putting 10k simultaneous users when its expected to take 10k concurrent users, isn't it?"

    Actually, I would disagree. There are several good reasons to:
    1) You are stress testing. (Your goal is to find the limit of the system.)
    2) The lifespan of the product is unknown. (So you could end up having a much higher than expected userbase.)
    3) You might find out your system can handle far more users than expected - meaning the lifespan could be longer (and costs lower).

    There is also a practical reason for using rendezvous points, etc - you can't afford a big enough license.

    ------------------

  8. #8
    Moderator
    Join Date
    Sep 2001
    Location
    Doncaster, UK
    Posts
    5,788
    Post Thanks / Like
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Euc, its not always easy to increase Vusers with Mercury's random pricing policy (and decreasing IT budgets) so using rendezvous points may be the only answer to simulate an increased load.

    Don't get me wrong, I would increase Vusers where possible but it isn't always so.

    Mark,
    Perf. Analyst.

    ------------------

  9. #9
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,018
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Do you have an engineering document to tie the rendezvous event to? Something indicating that a certain piece of code associated with the rendevous is capable of handling n simultaneous (or nearly so) calls. If not, you will very likely be causing an artificial locking constraint in your test, one that would never occur in production due to the chaotic nature of arrival and departure of clients.

    As a rule, I never make use of rendevous unless I have an engineering document that documents that a piece/section of code is designed to handle n simultaneous connections without a semaphore-style lock occuring, and that this ability to handle n connections is a testing requirement under load. I have found it to be far less expensive to most organizations to lease the required number of users than to spend engineering time and resources chasing down artificially created peformance problems that will not occur in production ( some number of developers * a number of hours to debug and fix and retest * the average hourly rate for the pool of dvelopers and testers) . As a consultant who audits and provides second opinions on performance tests for companies, if I do not have an engineering document or testing requirement for simultaneous behavior to tie a rendezvous event to I will not be kind in my analysis. I will likely even classify the test as unrepresentative of behavior of clients in production.

    Regarding your comment about stress tests. Remember, Stress tests are tests conducted under an ever increasing load until the system fails, but this is stress within the normal and expected performance envelope of the application. It is not unlike having your exercise physiologist place you on a treadmill or exercise bike and increase your load every minute until you reach a target haert rate or go into oxygen debt/collapse.

    I know, my opinions may be the minority when it comes to the use of rendezvous but I have seen far too many instances of people believing that x rendezvous users will represent y natural users of load, when there is little to no basis for drawing that conclusion. Evidence which would support such a claim would be logs from a deployed application noting timestamps for arrival and departure of clients for certain sections of code/application with some number of simultanous instances of behavior in an hour across a larger population which has been using the application during the same period. I have yet to run across application or web server logs which support this use of rendezvous in six years of work with LoadRunner Maybe once again I am in the minortiy in this case and others have run across such evidence. If so, I would be very interested to hear about it, how you tied the observed behavior back to a system test requirement, how you defined success and failure bassed upon that requirement and if the rendezvous was inside or outside of a transaction (another common problem which distorts response time measurements).

    Something tells me I should have had my morning caffeine before submitting this post : )

    ------------------
    James Pulley
    nospam.jpulley@nospam.itestsolutions.com
    iTest Solutions, Inc
    704-243-2854 (voice)
    James Pulley

    Replace ineffective offshore contracts, LoadRunnerByTheHour. Starting @ $19.95/hr USD.

    Put us to the test, skilled expertise is less expensive than you might imagine.

    Twitter: @LoadRunnerBTH @PerfBytes

  10. #10
    Junior Member
    Join Date
    Jun 2002
    Location
    Santa Clara, CA
    Posts
    25
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How are response tiems measured?

    Again, agreed with Jim's comments.


    I would add a few more comments

    When you launch "x" number of virtual users, you create "x" number of connections to the server / app server / database server.

    If you are trying to get more concurrent users (as a test) you need to find a way to increase the number of simultaneous connections that are being opened / maintained.

    One way to achieve this (kind of) is to use iterations. If an iteration executes in a shorter time (and doesn't disconnect cleanly) than the session timeout limit, it is possible to fool the system under test to maintain more connections than there are users. But of course those unused connections aren't really doing anything work, so that's the trade-off.

    All depends on the application.


    You need to be practical as well.


    Only in very LARGE systems to you get 10,000 concurrent users. Most systems (80%) have trouble reaching 1,000 concurrent users and maintaining performance.

    Testing in the extreme is usually an isolated practice, one for which Mercury's hosted virtual-user-days licensing model will work well.



 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.38%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 04:18 PM.

Copyright BetaSoft Inc.