# Thread: Project Manager questions a Test Engineer

1. ## Project Manager questions a Test Engineer

Hi all,

What would be the answer when a Project Manager asks a Test Engineer "How many more defects can you find within a given time slot before shipping a project to the client?"

Thanks...

2. ## Re: Project Manager questions a Test Engineer

I would tell the PM it depends on the quality of the submitted code I have left to test.

The PM should know it's not the number of defects, but also the severity of defects, the amount of time required to fix them, and the amount of time required to retest them that determines what can be done within a given timeframe.

The best you might be able to do is give an estimate based on the percentage of error found to date. If your error rate is running at 20% (# errors found divided by #tests run), for example, you can take the number of tests left to run, multiply it by 20%, and give the PM the result.
It would then be up to the PM to extrapolate the other numbers. The PM needs to be aware, however, that such estimates are not particularly accurate.

- Linda

3. ## Re: Project Manager questions a Test Engineer

Another approach is to tell the project manager if the rate is increasing or decreasing, and by what slope. The marginal cost of finding the last bug in a given piece of code is astronomical, but to get to within 2 standard deviations might be a feasible result in 3 test cycles (95% error free) and that might be good enough depending on what the product is used for. The error rate is never a constant over time: if it was we would never be able to finish testing any product, Linda's rate only reflected the relationship between errors and number of tests performed, which is different.

4. ## Re: Project Manager questions a Test Engineer

What would be the answer when a Project Manager asks a Test Engineer "How many more defects can you find within a given time slot before shipping a project to the client?"
<font size="2" face="Verdana, Arial, Helvetica">Tell him, "The rest of the ones we will find before we ship"! Actually had you implementd the right metrics he wouldn't have to ask you. He could look at your "S" curve and make his own prediction. MTBF is a good indicator also.

5. ## Re: Project Manager questions a Test Engineer

Originally posted by ToBePerfectQA:
Hi all,

What would be the answer when a Project Manager asks a Test Engineer "How many more defects can you find within a given time slot before shipping a project to the client?"

Thanks...
<font size="2" face="Verdana, Arial, Helvetica">More can be found than there is time to fix them

7. ## Re: Project Manager questions a Test Engineer

The answer given by the Test Eng. was "As the exit criteria has been met, all the testcases have been passed and all the requirements are fullfilled there will be zero high/medium defects found if tested."

Thanks for the replies...

8. ## Re: Project Manager questions a Test Engineer

Well, this is THE topic and the issue I’m working on for some years (want to write an article). Let me share only few items here:
1) There is a problem with tester who is typically concerned only about how much defects are there right now. All our Pass/Fail criteria (see my other posts about it) defect severities/priorities, etc. are typically based on quality right now. Although testing is nowadays seen as process of improving quality instead of evaluating (as it was once, according to some books I read about testing term change over the years).
2) On the other hand and to answer JakeBrake’s "Why are you asking?". It is simple – he want to know the ship date. Defects that are going to be opened are one of the main variables that PM depends on. Also PM may consider dropping some features if see the number of defects are high.
Actually if PM don’t want to know this he doesn’t actually care about testing and quality at all.
3) My understanding that this value build up from following sources: defects that will be “implemented” along with new features, regression defects that will be implemented along with bug-fixes and finally defects that are there in product but testers didn’t find them yet (but will find later). I call them “hidden”. Both logical (defects that disable testing some cases) and psychological (defect take too much tester’s attention that s/he don’t see other defects).
4) Hidden defects mean that PM not only needs to know absolute number of defects but also – how they will be detected over the time. It is all so complicated, that simple numbers don’t help – you have either to work with PM a lot of simply be his right hand in estimating dates (if you are ready to take responsibility)

9. ## Re: Project Manager questions a Test Engineer

Ainars,

Predicting defects is a difficult task and I have yet to see any type of system that can do so reliably.

Many companies establish a quality metric up front, such as an 80/20 rule. 80% of all errors are fixed and the remaining 20% are non-critical. The system is ready to ship when all planned tests have been executed and the system passes the 80/20 rule.

It is possible, as well, to plot project progress by taking a daily measurement of defects reported vs. # tests run. There comes a point (the quality point) in a project when it is no longer economically viable to continue to look for bugs. As the test period continues, eventually the # of bugs found should decrease; a simple graph can demonstrate this. There is no point, for example, to have a 5-person test team continue testing if (in an 8-hour period), they locate one level 4 (cosmetic) error. It is up to your organization to establish what is, and is not, acceptable.

- Linda

10. ## Re: Project Manager questions a Test Engineer

Linda, you got me! To be honest in my "prediction system" one of the key roles play intuition - mine as well as each team member’s [img]images/icons/smile.gif[/img] . Root cause for success is that each team member analyses impact of each defect/issue on the risk to find more defects later. Also we don’t predict numbers, we actually manipulate them in order to fit into deadlines. I mean if it seems like we will have too much defects later we drop some development activities (fixing certain bugs, implementing features, etc.) so we decrease the number. But again – it is whole story to tell. And I’m not too good at telling them short

Page 1 of 2 12 Last

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.