| || |
For each project, I am asked for time estimates for testing a particular function or 'story'. How do you come up with such a number? I know approximately how long it takes to test it. I know approximately how long it will take to regression and stress test it.
How do you factor in the 'out of QA time' while developers are making the fix? Some are fixed after one cycle. How do you factor in the 'multiple return bugs' that go through the cycle 4, 5 and 6 times before you can close it? I want to be as accurate as I can, but if my times are affected by others, I have no idea how I can!
Any advice would be appreciated as always
Re: Time Estimates
Digits, I've been wondering the same thing. Here's a couple of the factors I use for estimating "out of QA" test time. I know there are others.
Experience of the Developer(s) - Is this the first project they've done using a particular coding language or are they experts at the language?
Average Bug Fix Time - Pick a similar project from the past in terms of complexity/language. Review the bugs for it and come up with an average time that it took for the bug to go from "Assigned" to "Closed". Keep in mind the number of developers that were involved in the "bug fixing". Obviously if it took an average of four days for a bug to go from "Assigned" to "Closed" with two developers handling bug fixes, then estimate eight days if only one developer is involved on the project.
Re: Time Estimates
I just use the standard QA measures, like Time to Find and Time to Fix related to defects along with standard measures (such as that on average it takes about 6.3 developer hours to isolate and fix a defect). Another popular measure is Developer-Months to Find and Developer-Months to Fix in relation to defects. Both of these values are calculated by multiplying the average number of defects in a project by the average time spent fixing each defect. (This could tie into Steve's "Average Bug Fix Time.")
Since I tend to take an engineering approach and try to reduce subjective measurements, I also generate MTBF and MTTF statistical measures for proactive measurements but instead of considering these in relation to the statistical mean for a module to work without failing, I apply it to the concept of how long it will take to get the module to stop failing. With that kind of approach (or without it, for that matter) you can also break it down into specific readily applicable metrics. To wit:
Productivity (= size/effort)
Speed of delivery (= size/elapsed time)
Product Quality (= defects/size)
Here "size" is the measure of work-output from a given development team, rather than an individual. This can be based on whatever metric you want, such as normal delivery rates, complexity of modules, etc. (Obviously a weighted measure should be used in those cases.)
I also do something similar to Pert estimation in a way when factoring in "out of QA" time. Basically, I estimate the expected time delivery of fix but do so in the form of a ratio. One way is to take the smallest possible time delivery estimate (optimistic estimate). So what is the smallest interval of time in which they could potentially get the fix to you?
Then I also take the largest possible time delivery estimate (pessimistic estimate). Obviously the largest possible is some deadline by which the product will be in a "go/no-go" determination. With this, you then you have the expected value and the standard deviation given by Pert-like equations:
E = (a + 4b + c)/6
SD = (c - a)/6
where a = the optimistic estimate, b = the expected value, and c = the pessimistic estimate. (The expected estimate will be on your other means of metrics as I discussed above or as Steve discussed.) According to this technique, the actual size of the time value will fall between (E-SD) and (E+SD) about sixty-eight percent of the time. The value of SD gives you a view of the schedule and budget risks associated with the estimate based on "out of QA" time. Obviously, the larger the SD, the greater the uncertainty and thus the more risk is associated with "out of QA" time.
All this pretty much works every time for me regarding estimation techniques.