# Thread: How to determine how many bugs should be found

1. ## How to determine how many bugs should be found

I work for a CTO who has little understanding of QA. He has set a goal for me to determine the "industry standard" of how many bugs per day a tester should find. Of course I have a lot I could say to him about this, but I need some solid facts. He actually expects me to come up with a number and then QA is supposed to find X amount of bugs per day! This is the tip of the iceburg at this company, there are a lot of problems, and in the 6 months I've been here, no one has wanted to listen to any of the ideas I have told them we need to implement. My boss is just now coming around to seeing that the way they do things is not correct, but he still is stuck on this idea that QA has to have a quota of bugs. I'm used to using the bell curve in the typical waterfall testing, but they don't do things like that here.

------------------

2. ## Re: How to determine how many bugs should be found

Facetious answer: 0<= X <= 5 billion per person per day. That will cover over 99% of testers in the world, at a guess (but then, of course, 67.65% of all statistics are made up )

From what you say, you don't need me to tell you that this is ridiculous and a waste of your time, however some managers won't be swayed by arguments.

Here is a possible approach. Found out why he wants this number. Is it to measure tester productivity? Efficiency? Skill? If any or all, then there are much better measures you can use - try to sell those to him.

If you really, really are forced into coming up with some bug numbers, I would try the percentage approach, something like:
In the first 10% elapsed time of testing, we will find 40% of the bugs, in the next 70% elapsed time we will find 50% and in the last 20% elapsed time we will find 10% of the bugs. Note, my numbers are made up, you could use your own history to come up with better numbers, but of course you generally find a lot of bugs at the beginning, then fewer, then when you are in regression test, fewer still. If you are not in a Waterfall environment, I guess you are in an environment where you get phased releases of some sort (XP, RAD, whatever), so you can apply percentages to each phase and then to the final System Integration test phase.

I would definitely steer away from absolute numbers as there is no way you can seriously predict these in the type of environment that you are describing - in a more mature environment it is possible, although I still am wary.

------------------

3. ## Re: How to determine how many bugs should be found

To add to Peter's comments, make management aware that the likely result of a "bug quota" for testers is that they will spend the majority of their time finding trivial bugs instead of important bugs, which is the exact inverse of what is desirable (concentrating limited test resources on finding the important bugs).

------------------
Charles Reace

charles{DOT}reace{AT}verizon{DOT}net

4. ## Re: How to determine how many bugs should be found

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by Charles Reace:
To add to Peter's comments, make management aware that the likely result of a "bug quota" for testers is that they will spend the majority of their time finding trivial bugs instead of important bugs, which is the exact inverse of what is desirable (concentrating limited test resources on finding the important bugs).

<HR></BLOCKQUOTE>

Too true, and equally once they have found their quota they'll feel they can go home for the day. Human nature will also say that if they have a quota of (say) 20 bugs per day and they find 30, then 10 will be held over to the next day in case they don't find so many - not exactly what you want.

------------------

5. ## Re: How to determine how many bugs should be found

What first comes to mind is "YIKES!"

Let's see if we can put this into a slightly different perspective. There's a quote from the following: http://testing.com/writings/status/status.html that spells out to me what should be done. "Report on product, not people."

------------------
Jason Trebilcock
Cyberentomological Detection, Prevention, and Eradication Specialist
Wells Fargo

6. ## Re: How to determine how many bugs should be found

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by cpgirl:
I work for a CTO who has little understanding of QA. He has set a goal for me to determine the "industry standard" of how many bugs per day a tester should find. Of course I have a lot I could say to him about this, but I need some solid facts. He actually expects me to come up with a number and then QA is supposed to find X amount of bugs per day! This is the tip of the iceburg at this company, there are a lot of problems, and in the 6 months I've been here, no one has wanted to listen to any of the ideas I have told them we need to implement. My boss is just now coming around to seeing that the way they do things is not correct, but he still is stuck on this idea that QA has to have a quota of bugs. I'm used to using the bell curve in the typical waterfall testing, but they don't do things like that here.

<HR></BLOCKQUOTE>

Maybe the CTO really does understand that this hunt for an "industry standard" bug count quota is foolish?

Maybe he/she is just trying to see if you'll be able to come up with convincing arguments as to why such folly shouldn't be implemented?

(here's hoping)

------------------
- Joe (strazzerj@aol.com)

7. ## Re: How to determine how many bugs should be found

Wouldn't a high number of bugs-per-day have more to do with bad requirements, design, coding, and configuration management than with good quality assurance?

8. ## Re: How to determine how many bugs should be found

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by testgeek:
Wouldn't a high number of bugs-per-day have more to do with bad requirements, design, coding, and configuration management than with good quality assurance? <HR></BLOCKQUOTE>

I agree, large volume of bugs clearly indicates bad requirements,design,poor coding,absence of standards and procedures and bad process (Software Development Process or SDLC). In some companies the project managers don't even know SDLC

qa

------------------

9. ## Re: How to determine how many bugs should be found

I would just tell the powers that be that this is "Quality Assurance", not "Quantity Assurance".

------------------
Don

If you don't like my driving then stay off the SIDEWALK!!!!!

10. ## Re: How to determine how many bugs should be found

I think this is an impossible number to come up with.

This would be like saying that the Tester who found 15 low priority defects in an obscure / infrequently used portion of the app did a better job than the Tester who spent all day characterizing 1 killer defect in the main part of the app. Ridiculous!

------------------

Page 1 of 3 123 Last

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.