User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 2 of 2
  1. #1
    Senior Member
    Join Date
    Mar 2002
    Twin Cities, MN, USA
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)
    Total Downloaded

    Create estimation baseline

    Would anyone recommend what main categories that one need to track (using a time tracking application) so that when the next release comes around, one could estimate better the amount of time would require for:
    Test Planning
    Test Scripting/Maintenance
    Executing Regression Test Suite,
    etc ...
    or if more resources are needed.
    Thanks in advance.


    BikerTech {at} SQATester [dot] com
    Thanks for your feedback.


  2. #2
    Senior Member
    Join Date
    Dec 1999
    Post Thanks / Like
    0 Post(s)
    1 Thread(s)
    Total Downloaded

    Re: Create estimation baseline

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by BikerTech:
    ... what main categories that one need to track ... so that when the next release comes around, one could estimate better the amount of time ....<HR></BLOCKQUOTE>

    Remember that the basis of all estimation is basically planning. In fact, plans are based on estimations. (Well, they should be anyway.) As far as estimating things, what we should try to do is follow this basic format:

    <UL TYPE=SQUARE><LI>Estimate the size of development of something (code, test cases, etc).
    <LI>Estimate the effort (person-months, person-hours)
    <LI>Estimate the schedule (calendar-months)
    <LI>Estimate the cost[/list]

    In other words, how long does it take to write that test plan? What is the cost? (And remember that cost can be related in terms of effort.) How long does it take to test certain areas of the application? How long does a full regression cycle take? How long does a partial regression cycle take? How many defects that are marked "fixed" routinely have to be re-opened and then re-tested? So you will want to track those things. Consider that overall productivity at some task can be given as [size / effort]. The speed with which you are to complete a given task (test plan, regression test suite) is given by [size / elapsed time].

    Now, as far as test cycle estimation, besides tracking how long a given test cycle takes, you can also track the defect metrics. In general your average test effort estimation metrics have to come from two things: test engineering time and test execution time. In reality, testing time tends to depend on several factors, some of which are the application and/or system size, the estimated number of defects to be found, the required level of reliability, and the defect detection rate across testing phases. This process tends to involve an estimate of the total number of defects in the product which can then be used to extrapolate the total testing cycle time. One such approach uses a weighted defect finding rate that is plotted over time. The result is a decelerating curve that follows the form of:

    r(t) = abe - bt

    It may sound complicated but it really is not and it is quite effective. In this relation, a is the total expected defects (and this can be based on past data or on estimates), b is the defect finding rate (again, can be from past data) divided by the total expected defects, and t is the cumulative hours of testing. This kind of process can also be used as an approach for deciding when to stop testing, particularly if you have time-to-market priorities. The idea is to plot cumulative test error rates over time and when the curve decelerates to an acceptable level, testing can be halted. In either case, a good estimate of the total testing time really cannot be made until testing is well under way or unless you have past data to fall back on for your estimates.

    Hopefully this at least provides you with a little direction. Below is an expansion of one of my main points above. I separate that out in case you are already falling asleep reading this so far.


    In general with estimation it is best to keep two things in mind: (1) It is best to understand the background of an estimate before you use it. (2) It is best to orient your estimation approach to the use that you are going to make of the estimate.

    To explain that a little better, we have to realize that estimating things like "test planning", "test scripting", and "test execution" can be multifaceted and, as such, has multiple dimensions. I basically listed these above, but here they are in list format:

    <UL TYPE=SQUARE><LI>Productivity (size/effort)
    <LI>Speed of delivery (size/elapsed time)
    <LI>Product Quality (defects/size)
    <LI>Delivery relative to time and budget[/list]

    Here the notion of "size" is being used as the measure of work-output and thus it can vary based on need. But keep in mind that these variables (dimensions) are tradable to a certain extent. For example, it may be possible to test an application faster than normal by adding in extra test personnel, which will result in lower productivity and higher cost. Product quality may or may not suffer in the process. This kind of potential problem with such an approach may be acceptable if it means the business benefits of the application are obtained earlier (i.e., you reach time-to-market and your critical defects are under a certain threshold). On the flip-side, an understaffed test effort may be efficient, because you have good testers, but take a longer time to complete.

    The point is that measuring the overall workflow of test development and test execution (as well as maintenance activities) must take into consideration to all these various aspects of the workflow and must also measure the drivers of that workflow (e.g., technology, such as automation, tester experience, time constraints, resource constraints, etc.). The point is that you need to correctly interpret what you are seeing because that is how these metrics that you track will turn into reliable estimation indicators the next time around.




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Search Engine Optimisation provided by DragonByte SEO v2.0.40 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.54%
vBulletin Optimisation provided by vB Optimise v2.7.1 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.3.0 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:25 AM.

Copyright BetaSoft Inc.