Thanks:  0
Likes:  0
Dislikes:  0

# Thread: Using Lincoln Index as a total bug count estimate?

1. ## Using Lincoln Index as a total bug count estimate?

http://www.johndcook.com/blog/2010/07/13/lincoln-index/

Has anyone actually ever used a Lincoln Index as a way of estimating the number of remaining bugs?

A while back, I posted a book review, and wondered if the Lincoln Index would be useful in this context as well.
http://strazzere.blogspot.com/2010/0...ring-more.html

[ QUOTE ]
Everyone knows that typographical errors (called typos by those in the trade) are sometimes difficult to spot, so a printer might ask two proofreaders to read through independently to look for errors.

Suppose the first reader finds E1 errors and the second finds a different number, E2. They now compare their results, and discover that some of the errors, a number S, were the same ones. How many errors might they expect there to be in total?

There is a way of making a good estimate, known as the Lincoln Index. This says that the total number of errors in the manuscript will be roughly:

Expected Errors = (E1 * E2) / S

For example, suppose the first reader found fifteen errors and the second twelve, and that ten of the errors were found by both. The Linoln Index predicts (15 x 12) / 10 = 18 errors in total. Of these only seventeen have been found so far - ten found by both readers plus five more that only the first reader found and two more than the second found.

[/ QUOTE ]

2. ## Re: Using Lincoln Index as a total bug count estimate?

Love it and hate it all at once. It's kinda cool, but it's still dangerously contextual. I mean what build is this? Like how many have we already seen? What version is this? Is it version 1 or 10? Is it a minor release or major release? What is the experience level of the tester testing it? What is the severity of the bugs being entered? Are these bugs which we're going to fix or defer? What percentage are bugs which need to be fixed?

Estimation or not, these are extremely important questions to ask because, in the end, when you try to sell someone on the release, they will either ask these same questions or they won't, in which case you have duped yourself because you haven't asked yourself the tough questions.

It's like if I were to ask you a very simple scientific question, "How long will it take for the water to freeze at 0 degrees fahrenheit?"

This is a simple task. We have enough variables to generate a number which could answer the question, but the answer will also vary largely depending on other key variables like what is the starting water temperature? How much water is there? What is the shape of the container? What is the container made from?

So the real answer to the question can't REALLY be determined unless you have a knowledge expert who can answer these questions with regards to the situation at hand. Even if you do formulate an appropriate response based on the external criteria, this value is only valid within these conditions. Under other conditions the answer will vary slightly. Since software is ever-changing and there are an unending number of variables changing on any given build, this sort of hard number becomes a liability, especially if it's used as a major contributor to the final release decision.

3. ## Re: Using Lincoln Index as a total bug count estimate?

So, Brent - I'm guessing that you have not actually tried using this as an estimation technique?

I was wondering if anyone had.

As John Cook said "Does the Lincoln Index actually provide a good bug count estimate? That depends on how well the assumptions are met."

4. ## Re: Using Lincoln Index as a total bug count estimate?

I haven't tried, but doesn't the basic formula expect that testers will overlap? What if two testers check entirely different aspects of the system with no bug overlap, the results can't be valid since S is now 0 and we all know what happens if you divide by 0.

5. ## Re: Using Lincoln Index as a total bug count estimate?

Nope, and I wouldn't either. Michael does make a good point also. In addition, though, is that the estimation is using unintelligent data (defect tracking information) to determine the status of a project while there is a knowledge center, or in this case multiple knowledge centers, who are standing by with an opinion which SHOULD weigh 10 times more heavily than the number that I can generate off this formula.

If I've had two people crank through a system and neither can tell me what state my project is in, I'm screwed. I really am. Because not only can they not tell me what state my project is in, the simple fact they don't know the state of the project would, for me, devalue any data that I would be using to generate this estimation? What would their test results mean? The same can be said if I can't trust their evaluation of the software.

Now I know that a thumbs up isn't something I can sell to management, but maybe that's the problem. Maybe we're too busy trying to generate numbers that make the line go up and down that we're not seeing the full picture. It's a schooner!

6. ## Re: Using Lincoln Index as a total bug count estimate?

[ QUOTE ]
What if two testers check entirely different aspects of the system with no bug overlap

[/ QUOTE ]

The premise is that both testers/proofreaders test/read the same thing, not distinct sections.

7. ## Re: Using Lincoln Index as a total bug count estimate?

I've never used this method.

Is your interest simply to prove/disprove/test this theory as applied to defect detection or do you see some value in using it?

As mentioned, there are plenty of assumptions associated with trying to make a test of this nature valid. Having said that, I don't see how that would be any different from any other defect metric. It would be fun/interesting to 'test' this method, but I think I'd have a hard time convincing my client that I want them to pay for 2 testers to test the same thing in parallel.

Assuming this algorithm 'worked' within a reasonable level of fluctuation (+/- x%), what would you do with the information? We found 57 defects and you probably have 7-8 more going into production? Would you use it to justify an extended testing phase?

8. ## Re: Using Lincoln Index as a total bug count estimate?

[ QUOTE ]
Is your interest simply to prove/disprove/test this theory as applied to defect detection or do you see some value in using it?

[/ QUOTE ]
I've tried and discarded many other metrics as not useful, impractical, or misleading. I'm always checking out new possibilities.

Short of trying the experiment out with my own team (something I haven't found the opportunity for in the 2+ years since I first encountered the concept), I was simply wondering if anyone had actually tried it, and if they had been successful or not.

[ QUOTE ]
Assuming this algorithm 'worked' within a reasonable level of fluctuation (+/- x%), what would you do with the information? We found 57 defects and you probably have 7-8 more going into production? Would you use it to justify an extended testing phase?

[/ QUOTE ]
Yes! Although in my case, I'd really only have to justify it to myself.

9. ## Re: Using Lincoln Index as a total bug count estimate?

Well you can read Michael Bolton's view on the Lincoln Index which just came up on my feed reader a little while ago.

10. ## Re: Using Lincoln Index as a total bug count estimate?

[ QUOTE ]
[ QUOTE ]
What if two testers check entirely different aspects of the system with no bug overlap

[/ QUOTE ]

The premise is that both testers/proofreaders test/read the same thing, not distinct sections.

[/ QUOTE ]

I guess it depends on how you define "same thing", is it the system itself "we're both testing Project X but A is testing the GUI and B is testing the API Interfaces" or do they need to test the exact same thing? I wasn't making the leap from two proofreaders to exact same testers, but rather two testers on the same project, who may or may not have overlap. To me I am not sure where I would use it, and I haven't had occasion to do this sort of thing in the past where I have had multiple people test the same thing over a period of time.

I don't know if it works historically, but if you had two people testing the same set of code over a multi-year period, if you knew all the duplicates they found (and not entered because it was already there) you might get data. Although I don't know what use it would give me in knowing what I haven't found, I usually expect "one more bug" to pop up at any time.

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.