SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 6 of 6
  1. #1
    Member
    Join Date
    May 2008
    Posts
    155
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Handling Asynchronous - system generated events

    Hi,
    This is a general question and is not restricted to any type of application or protocol. My question is on emulating asynchornous and system generated events in the load script. During recording of an application, few of the events that were triggered at that point of time might get recorded as requests in a load script. However, during the execution of the load test the same might not repeat and the number of events generated and communicated to the client can be very huge. How can we handle this in our load tests?

    For e.g., I was testing a web application that displayed a set of records retrieved from the database as a table. Whenever an event is generated at the server, the server updates the client through several notifications. The table was getting refreshed every time when it received a notification.

    Now, how can we handle these things in our load test script? Is it wise to ignore these events and concentrate only on the functionality and its performance alone?
    The views expressed in this forum are mine. My organization does not subscribe to the substance, veracity or truthfulness of the said opinion

  2. #2
    Senior Member
    Join Date
    Oct 2004
    Posts
    1,011
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Handling Asynchronous - system generated events

    [ QUOTE ]
    Hi,
    Is it wise to ignore these events and concentrate only on the functionality and its performance alone?

    [/ QUOTE ]

    Is system genearted call not functionality?

    You can not ignore these event.

  3. #3
    Junior Member
    Join Date
    Jul 2003
    Location
    Saarbrücken
    Posts
    16
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Handling Asynchronous - system generated events


    The problem has come to us several times as well. I agree with A$H, you cannot ignore those events. The problem is how to simulate them correctly.

    All performance testing tools I know run single-threaded per VUser. This is logical, because users would try to save licensing costs by putting the actions of multiple virtual users into the threads of one licensed VUser. The correct implementation would have the virtual user start a thread which periodically makes the system call.

    I think, for LoadRunner there is at least one way to do it though: As far as I recall, Java virtual users are able to run threads. So the solution would work there.

    Now if you don't use LR in Java mode or have a different tool, you have to make some considerations about tradeoffs. I mostly used the following workaround: I cut the system calls out from the use case part of the script and extra use case. (My scripts are usually structured by use case). After baselining the scripts, I mix in the system calls into the workload so their average call frequency comes close to what it would be in a proper simulation.

    Advantages:
    - System calls don't make the use case time longer
    - Server gets as many system calls as it would get in real situations

    Disadvantages:
    - Slow system calls still slow down the VUser
    - System call is never in the middle of a use case

    However, this won't work for long lasting calls, like some AJAX based applications use them. In order to prevent massive polling, the server will answer the call with a "nothing changed" message just before the connection times out or with a status change message whenever such an event happens. If you happen to have such a behavior in your application under test, you could consider to make the call and subtract the time from the virtual user's think time in order to keep the pace. It's another tradeoff, of course, but doing user simulation for business, we must always deal with those, don't we?

  4. #4
    Member
    Join Date
    May 2008
    Posts
    155
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Handling Asynchronous - system generated events

    Christian, Yes. Since most of the load test tools and the supported VUser scripting model is single threaded, it becomes difficult to simulate push / asynchronus events.

    From your reply, I understand that you record the basic functionality along with the push events fromt the server, and mix the later one after baselining the script. Am I right? Or you are running the system calls as separate VUsers in the entire workload so that the # of calls are equal to the one in production work load?

    In the past, I was testing an application using windows sockets. (with several send + receive calls). The problem was that the application was pushing HUGE NUMBER OF ALARMS to the client using the socket. The client was receiving these ALARMS + THE DATA RELATED TO THE INVOKED FUNCATIONLITY. Now, during emulation, I had no option but to receive these ALARMS by waiting for the required data to come in. The real client was processing them using several threads and hence the response time projected was still higher than the one perceived in the real client. How can we handle these things so that we still get the accurate response time?
    The views expressed in this forum are mine. My organization does not subscribe to the substance, veracity or truthfulness of the said opinion

  5. #5
    Member
    Join Date
    May 2008
    Posts
    155
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Handling Asynchronous - system generated events

    [ QUOTE ]
    Is system genearted call not functionality?
    You can not ignore these event.

    [/ QUOTE ]
    The business process goes like this
    1) Open ALARM window (Alarm window loads)
    2) Click the HISTORY ALARM TAB
    3) RETRIEVE THE NEXT 500 RECORDS IN THE HISTORY ALARM TAB
    (Now, during this the server might PUSH the newly CLEARED ALARMS TO THE CLIENT WINDOW. The window was getting refreshed)

    Hence, even if I project the response time based on the successful completion, it would still miss the time taken by the client to process the new alarms. I didn't know how to handle them. Thought of getting some inputs on how these system generated push events can be handled during load testing.
    The views expressed in this forum are mine. My organization does not subscribe to the substance, veracity or truthfulness of the said opinion

  6. #6
    Junior Member
    Join Date
    Jul 2003
    Location
    Saarbrücken
    Posts
    16
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Handling Asynchronous - system generated events

    [ QUOTE ]

    From your reply, I understand that you record the basic functionality along with the push events fromt the server, and mix the later one after baselining the script. Am I right?


    [/ QUOTE ]

    yup

    [ QUOTE ]

    Or you are running the system calls as separate VUsers in the entire workload so that the # of calls are equal to the one in production work load?


    [/ QUOTE ]
    Yes, the difficulty is, to get the separate VU to run in the same session as the ones they perform the asynchronous tasks for. That is sometimes not possible.


    [ QUOTE ]
    How can we handle these things so that we still get the accurate response time?

    [/ QUOTE ]

    To get it accurately will be difficult. Here's a suggestion: Stop the clock for the business process while you are processing the alarms. Increase the number of virtual users but also increase the think time, so the call frequency stays the same (downside: If you have session ressources per VU on the server, your load is less accurate there AND you need more VU licenses). Subtract the time used for alarm handling from the think time, so the business process frequency isn't influenced by the alarm handling.

    Depending on how accurate the results must be and how much effort is affordable, you might even want to consider writing a specialized test driver for the application.

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 01:47 PM.

Copyright BetaSoft Inc.