| || |
Load Test Planning
Talks in general about planning the load test as well as type of scripting and configuration based information in the perspective of planning
Re: Load Test Planning
Can anyone guide me with detials of when to go for HTML based or URL based scripting ?
Re: Load Test Planning
1) Please check out the Performance and Load Testing forum located on QA Forums.
2) An excellent thread/article regarding HTML and URL based scripting was posted on the Yahoo LoadRunner forum by Steve Cheney (9697). I have reposted below.
This is kinda old; I keep it around for peoples. It was written by
Zachary Reese Whiting
A. What is meant by context sensitive mode?
HTML recording is what's referred to as "context sensitive" mode. What
context sensitive mean? Well, when you send off a request to the server and
back a page you're essentially setting yourself into a state. So executing
web_url or other action step in a script is what sets you into a state.
are some functions that when they start to execute check the state that
currently in before executing. In other words these steps are dependent on
state set by the previous step. The execution of these steps is referred to
context sensitive because their execution is dependent on the state set by
previous request, or another way of putting it is they execute in the
state ) of the previous step.
How does this effect these steps? Lets say we execute a web_url step (
which is by the way a contextless function meaning it doesn't check the
state ) which goes to www.yahoo.com. The request is successful and returns
main yahoo page, thus setting us into state A. The next step to execute in
script is a web_link step. The web_link step scans through the html which
returned by the previous request for the text of a link, extracts the url
string, and then sends a request for that url. So when it executes there
possibilities. 1) The text for the link exists in the page and thus the
finds the url string and is able to send a request off for it. or 2) the
for the link DOES NOT exist in the page and thus the web_link step can't
the url string and will fail. Now with our yahoo example, when the script
recorded the web_link step was created to handle state A in which the text
the link did exist. So when we execute the script the web_link scans
page. finds the text, extracts the url, and then sends the request for the
Possibility 1 as mentioned above.
Now lets say we replay the same script but this time yahoo returns a
in which the text the web_link is looking for does not exist. In other
once the step executes we are not in state A but instead in a new state B.
when the web_link executes it scans through the page and DOES NOT find the
and therefore can not extract the url string and fails. This type of
is what is referred to as "context sensitive" mode or HTML mode. ( If you
in the recording options you'll notice that beside the option for HTML it
context sensitive. )
B. How does this context sensitive mode make HTML differ from HTTP mode?
The most notable difference between HTML and HTTP mode is the parser
is the part of LoadRunner that scans through the html of a page. The parser
used by steps like web_link to execute and find the text it's looking for.
how does the parser make the recording modes different? It's because of the
functionality it gives us to look through the html. Because we can look
the html for information it allows us to do several things:
1. Look through the html for the text of a link and pull out the url.
2. Look through the html of a page for links to images and other files
which need to be downloaded for the page to display correctly.
3. Look through the page for text or information which would verify
the page we expected from the server was in fact the page that was
This is only a fraction of the things it allows us to do. Based on this
functionality it means when we submit a request for a page, for instance
www.yahoo.com again, we don't have do explicitly declare in the script
for images or other files which the page needs to display because the
look through the page's html and find the urls that would need to be called
do this for us. So if for Yahoo to be displayed we'd have to download 1
5 images referenced inside of that html, instead of having 6 steps in the
( 1 for each file ) that explicitly sends a request for each one the script
contain 1 step which requests the html for the page and the parser will
through the html and find the urls for the other 5 images and automatically
This also allows for another important factor with web pages, if the
for the images change or new images are added to the pages, then the parser
pick up these changes and automatically download the needed files from the
url or download or any new files needed for the page! This allows for much
flexibility in the script because the only url that must remain constant is
main call for the first html file, which is the call to www.yahoo.com in
example. This same concept allows the web_link steps to still be successful
the url that the link takes you to changes but the text for the link stays
same. For example if there was a link with the text "Mercury's Homepage"
took you to the url www. mercuryinteractive.com that was recorded into a
as a web_link("Mercury Homepage", "text=Mercury Homepage", LAST ), then if
url changed to point to www.merc-int.com, it wouldn't effect the scripts
because when the web_link executes it will extract the new url for
www.merc-int.com even though when the script was recorded that link
C. So what does this all mean for HTML scripts?
It means that the scripts can be more dynamic and simpler, allowing
possibility of a url inside a web page to change and the script to still
correctly given this. Also instead as our example above stated, instead of
recording 6 steps for 1 page ( 1 html and 5 images ), we can record only 1
which means the script will be shorted and easier to read and in most cases
D. Are there any possible drawbacks to HTML mode?
Unfortunately yes. Because for many of the steps the tool has to parse
through the html of a page this is going to use up processor time and some
memory to do these tasks and this use of resources on the load generation
machine will lower the number of vusers you can run on the system.
TOPIC 2. HTTP recording mode________________________________
A. A basic difference of HTTP mode.
One of the most noticeable differences between HTML and HTTP mode is
with HTTP mode we record EVERY request sent to the server for information.
back to the example in B of Topic 1 where we request for www.yahoo.com and
display the page needing 1 html file and 5 image files, instead of
step for the first html file a HTTP script will record the request for that
html file, but also will record the following 5 requests for the needed
files into the script. The urls for those images at the time of recording
be hardcoded into the script inside the web_url steps. Just remember about
mode the everything requested for by the client will be put into the
an individual step. Because of this there is no dependency of one step on
results of another, and therefore is not context sensitive.
B. Doesn't recording each request individually make the script less
In a way yes. Because the urls are all hardcoded if any of them should
change that step inside the script will fail possibly with a http 404 (
found ) from the server. However there are cases where you would want to
hardcoded url. Because web pages can be so dynamic it's not always possible
a tool to be able to account for every possible situation or intricacy of
the html was written. In cases like these knowing what the next url we need
request exactly is a benefit. Instead of a tool not being able to handle
specific situations, using hardcoded urls and requests for every file can
around these situations by allowing us to tell the tool EXACTLY what to do
and what to request.
C. So what does this mean for HTTP scripts?
It means that HTTP scripts allow for working with situations which may
arise because of non standard practices or new methods in coding html and
creating web sites. Because the parser is also not involved ( referred to
running in Analog mode ) http scripts don't require the extra processor
memory used to scan through html in order to find the next action or url to
take. This allows more users to be simulated on one system.
D. Are there any possible drawbacks to HTTP mode?
The script will not be as dynamic as with HTML mode. Should any of the
addresses for a file or a particular site change the script will need to be
edited to reflect these changes.
SECTION 2 - HTTP script options===============================
TOPIC 1. HTML all requests as custom requests___________________
The options for recording all requests as custom requests causes the
to have web_custom_requests recorded for the requests made by the client to
server. The web_custom_request function allows you to be able to create
requests to the server with information that is not in the standard format
specifying the different aspects and body of the request header. When this
option is checked inside the recording options for HTTP mode LoadRunner
automatically try to detect these special situations and create a
web_custom_request to handle it.
TOPIC 2. Concurrent Groups ___________________________________
The option for concurrent groups is related to the way in which the
requests are sent to the web server and how the connections for these
and transfer of information are opened. Concurrent groups allow for HTTP
to simulate the way HTML scripts open connections for files. The below
will explain how the 3 different modes ( HTML, HTTP, HTTP with concurrent
) handle connections for information.
When a HTML script replays and reaches an actions step it opens a
connection to the web server for the initial request and begins downloading
information. As the information comes in from the web server the client may
encounter other files which need to be downloaded to display the page as
At that time another request is sent to the web server on a NEW connection
is opened and the requested information is then downloaded on this new
connection. This will continue up till the maximum number of connections as
defined in the scripts run-time settings under the Network tab under the
concurrent connections option. Once a connection is closed and no longer
it is given to the next request to use if there are more requests pending.
method of downloading on several connections at once is called downloading
HTTP scripts in most cases download on 1 connection at a time because
the fact that the steps in the script only request one file and the parser
instructed by the run time settings to not execute and thus not download
files associated with a request automatically ( the parser is disabled by
turning analog mode on in the run-time settings ). As stated in A of topic
for section 1, HTTP scripts record a step for EACH request that is sent to
server. Because the script does not move on to the next step until the
step is completed the script downloads on 1 connection at a time. This
downloading is called downloading in "series".
C. HTTP with concurrent groups
HTTP scripts with concurrent groups inside of them include the new
functions web_concurrent_start and web_concurrent_end around blocks of
steps which send requests to the server. When the script is executed and
encounters a web_concurrent_start then any step following the function is
executed immediately but instead put into a queue. This queuing process
continues until the script reaches a call to web_concurrent_end. At this
the steps in the queue are then executed. The number of steps executed
on the scripts run-time settings for concurrent connections as set in the
Network tab under the concurrent connections option. Because they are
in this manner then mutiple connections are open at one time for
information and thus is downloading in parallel just as HTML scripts would.
D. So what is the purpose of having concurrent groups inside a script?
Having concurrent groups inside a script allows HTTP scripts to
the same action as a browser just as a HTML script does. Browsers download
in parallel on multiple connections at once. HTML scripts are able to
this, but HTTP scripts were not able to previously and as mentioned only
downloaded 1 file at a time. With the use of concurrent groups the HTTP
can closely emulate a browsers activities just as the HTML scripts do and
us to use HTTP scripts to solve specific situations which HTML might not be
to handle, but to still be able to closely emulate the browser with the
of connections it opens at any one time.
Changes in LR 7.0:
In LoadRunner 7.0, there will be a slightly improved HTTP recorder.
recorder will no longer record a seperate step for every request made to
server. Images and Frames will no longer be recorder into the script. These
now be handled during replay, similar to the HTML replay. This will allow
less flexible scripts, but they will handle Dynamic Data more easily. Also
that the option to switch to Ananalog mode will no longer be available.
because the HTML and HTTP replays will both be based off of the same
and there is no need to specify analog mode. This will mean that Context
Sensitive scripts can run on Unix.
Re: Load Test Planning
Thanks a lot for the detailed information.
Re: Load Test Planning
I need clarification on the following.
I need to test an enhanced version of a web site. The already existing web site supports
'n' number of concurrent users during one hour peak load time, with 'T' number of
My current objective is to ensure this support is there in the enhanced version too
and then find whether the enhanced version supports as follows
a) 150% of 'n' number of concurrent users with 150% of 'T' number of successful transactions
b) 200% of 'n' number of concurrent users with 200% of 'T' number of successful transactions
during the same peak load time.
Now my query is, for this case which of the two types (Manual and Goal oriented)
of scenario we need to choose ?
Thanks in advance for any clarification in this regard.
Re: Load Test Planning
I have never run a goal oriented scenario. This is a personal preference as I feel that the manual scenario gives me more control of the execution. Even my MI LR training brushed over the goal oriented section.
I analyze the results from the manual oriented scenario and extrapolate the required information from the relevant graphs and data.
My worry with goal oriented scenarios is that any 'spikes' may cause premature termination of the scenarios I test. In your case this may not be an issue.
Re: Load Test Planning
Since I also had the same opinion, I planned to do it with manual scenario.
Now, my scenario contains 6 different activities. Each activity has its own number of successful transactions to be supported, with total being say 'X' for 25 concurrent users. Inorder to apportion the number / % of users to perform each activity and the number iterations per user for each activity, I took the % of contribution from each activity to the total # of transactions 'X' and went ahead with the calculation.
When I ran this scenario for 25 users for one hour duration, the number of iterations exceeded more than what I actually defined in the script. The increase was exponential that I could not attribute it, only to the better performance of the enhancements but also to some miscalculation in the design of my scenario.
Here I need a clarification that whether 'pacing' is a factor. I had set 'As soon as the previous iteration ends'. Whether this is the right approach, if not how do we calculate the pacing time ?
Re: Load Test Planning
I can almost hear the rumblings from my colleagues that have been doing this stuff for a number of years (more than I have).
I'm sorry to say this, but if you don't know the answer to your own question
<font size="2" face="Verdana, Arial, Helvetica">then I question whether you know how to design the load test appropriately.
had set 'As soon as the previous iteration ends'. Whether this is the right approach, if not how do we calculate the pacing time ?
Pacing is a matter of understanding the circumstances which surround the load test, and whatever specifications relate to it. (And it's pretty basic, too).
Re: Load Test Planning
Without being derogatory, you might want to create a spreadsheet with the following columns.
Script Name, expected transaction rate, #users, Iteration rate, # iterations.
So if script A was login and perform a query that is happens 500 times an hour in the current config and is expected to happen 750 times an hour when the enhanced version is released, and the current concurrent user connection is 100 but will be 200. Put these numbers in the columns and come up with your iteration rate which for 200 users creating 750 per hour is 3.75 iterations per hour. Your pacing would be every 16 minutes or every 960 seconds.
The spreadsheet will help you perform 'what if' scenario calculations and can be used a template for making changes.
I break it down to the type of transaction per script on another sheet with the goal transaction rates. Create formulas so that when you change a value all tables/values update. It's a worthwhile effort and has always worked well for me.