# Thread: Calculating the Cost to Fix a Defect

1. ## Calculating the Cost to Fix a Defect

There is a good article on Sticky Minds this week from Johanna Rothman about cost of fixing defects.

This is a question often posed, as so many times, the common 'it costs more to fix a defect the later in the life cycle that it's discovered' is not well-explained or documented.

Metrics like those she discusses could help define this specific to an organization.

------------------
~ Annemarie Martin ~
annemarie[dot]martin2[at]verizon[dot]net

~Those who dream walk in stardust, and dance to the rythm of the heart~

2. ## Re: Calculating the Cost to Fix a Defect

I saw that article.
It was interesting, but I'm surprised she didn't talk at all about the cost of NOT fixing a bug.

In my experience, the cost of not fixing a bug is often a lost sale. That many times far outweighs the cost of fixing it.

------------------
- Joe (strazzerj@aol.com)

3. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by jstrazzere:
In my experience, the cost of not fixing a bug is often a lost sale. That many times far outweighs the cost of fixing it.
<HR></BLOCKQUOTE>

I would agree. I think that part of the problem, however, is that it's more difficult to quantify the costs of not fixing a defect. It's true there could be loss of sales, but how many? You could also lose current customers, or see a decrease in renewal of licenses, but again, how many? Long-term, you could track that sort of thing from Customer Support and Sales tracking systems, but it's sometimes much harder to 'get your hands around' than are metrics for the costs of fixing defects..

------------------
~ Annemarie Martin ~
annemarie[dot]martin2[at]verizon[dot]net

~Those who dream walk in stardust, and dance to the rythm of the heart~

4. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by QAGirl:
This is a question often posed, as so many times, the common 'it costs more to fix a defect the later in the life cycle that it's discovered' is not well-explained or documented.<HR></BLOCKQUOTE>

I agree that this is the case (in terms of being not well-explained) but I have never understood it. (I would maintain that it is well-documented, but not always in popular sources. IEEE routinely publishes articles on this as do software engineering or software development magazines/Web sites, etc.) For myself, I have never had trouble explaining to management why it was harder to correct downstream defects as opposed to upstream defects. Sometimes I have to resort to analogies about houses, bridges, and space stations but I can always convey the point. It is always a common-sense argument that never fails even though I do sometimes have to repeat my points a couple of times.

If I want to go a more matehmatical route, I can also use some of the equations of Barry Boehm, Philip Crosby, or Larry English which show you various ways that you can calculate downstream defect costs, in terms of various measures, whether that be in terms of lost revenue, lost employees, lost customer satisfaction, etc. Those can be more-or-less vague depending upon how you implement the equations. Actually, come to think of it, you could easily modify the equations that I showed in Project Effort and Value to handle downstream efforts after a project. Alternatively, you could widen the scope of what "downtime" means in my talk of metrics in Cost Associated with Software Virus/Bug for different types of defects.

Personally, I do this all the time, and, as QAGirl mentions, you can also build up trends for historical data. If you implement these kinds of metrics at a lot of organizations you will also see that they apply pretty much across the board and that is another quiver in your sling in terms of what you can show to justify the statement.

[NOTE: Due to QA Forums re-design, I have updated the link in this post.]

[This message has been edited by JeffNyman (edited 06-26-2002).]

5. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by QAGirl:
I would agree. I think that part of the problem, however, is that it's more difficult to quantify the costs of not fixing a defect. It's true there could be loss of sales, but how many? You could also lose current customers, or see a decrease in renewal of licenses, but again, how many? Long-term, you could track that sort of thing from Customer Support and Sales tracking systems, but it's sometimes much harder to 'get your hands around' than are metrics for the costs of fixing defects..

<HR></BLOCKQUOTE>

I agree that it's more difficult to quantify things like lost sales.
But clearly, it's an important factor in such ship/no ship decisions.

That's why I tend not to like "mathematical" approaches to these decisions - only the "easy to measure" factors are even considered.

------------------
- Joe (strazzerj@aol.com)

6. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by jstrazzere:
That's why I tend not to like "mathematical" approaches to these decisions - only the "easy to measure" factors are even considered.<HR></BLOCKQUOTE>

That depends on who utilizes the math and what is being utilized. I have been very successful in providing mathematical measures (and not just for what some consider "easy to measure" factors) but tempered, also, with realism that some numbers are fuzzy or some numbers are more in the realm of statistical possibilities or potentialities. Of course, most people I run into in QA are pretty much ignorant of statistics as well and so you run into the same issues that you are probably talking about: a reliance on the easy to measure.

However, quantification to any degree implies numbers and numbers, to greater or lesser degrees, imply "mathematical" approaches such as measures (less than, greather than, equal to, average, etc) or manipulations (taken away from, added to, multiplied over segments, sum over histories, etc).

It is actually very easy to quantify lost sales if you have viable expectations about the assumptions you bring to the quantification measures and, further, if you model this based on potentialities and opportunities. Beyond all that, which would require more detail than I want to go into, you can, at the very least, consider the estimated target market and the ratio of those people who you assume will buy your product. And here is where the fuzzy part comes in, but the point is you can quantify the lost sales relative to that figure if your assumptions are more or less accurate. Or you can take them as a starting point and then refine your metric model. (Something peole often forget to do and then later simply blame it on the math.) And if you do not want to think of it in terms of direct sales, you can simply subsume that within the mesaures of gross profit margins and revenue. You can also measure this relative to competition. I have done this routinely and thus I am speaking from experience.

There can always be ambiguities and vagueness, and that is part of the mathematical nature of statistics or fuzzy measures but, in fact, it is not just the "easy to measure" factors that are considered. What is often the case is that people do not know how to measure other factors and either (a) assume it is not possible, or (b) use metrics and math incorrectly, such as using inappropriate measures (such as fuzzy measures to get presumptively exact values).

------------------

7. ## Re: Calculating the Cost to Fix a Defect

I remember a previous company where I always saw sales projections that placed a 50% chance of landing a particular sale in the current quarter.

It usually stayed that way until the end of the quarter - when it was promptly rolled over to the next quarter at a 50% chance of closing.

------------------
- Joe (strazzerj@aol.com)

8. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by jstrazzere:
I remember a previous company where I always saw sales projections that placed a 50% chance of landing a particular sale in the current quarter.

It usually stayed that way until the end of the quarter - when it was promptly rolled over to the next quarter at a 50% chance of closing.
<HR></BLOCKQUOTE>

And, in that case, by what I said above, someone with a metric-mindset should have realized, at the end of that quarter, that the sale was not made and thus counted that as a lost sale relative to that quarter. What happens in the roll-over to the next quarter is irrelevant in the sense that, for that first quarter, you gained no revenue from that sale. Thus your metric is a sale loss for that quarter.

If the sale did not happen in the second quarter, you, again, did not have revenue from the sale and, in terms of the bottom-line of the business, you had a lost sale. This was a particularly easy example to determine because the sale simply was not made.

Of course, a good proactive approach to this situation is also to determine the expectations for this sale. If someone just keeps saying "We will get it this time!", then someone has to look at the expectations for this sale. Has the particular client that the organization hopes to sell to expressed great reservations about the product? Is that why it is not selling? Is there some other reason?

However, if no one bothers to look into this, then, of course, sales projections have no intrinsic value except as wishes. This, however, really does not speak to mathematical methods only looking at the "easy to measure" factors. This sounds to me like someone not bothering to (a) measure the current situation (no sale relative to a given quarter) and (b) no one looking at the details as to why the negative measure was happening.

------------------

9. ## Re: Calculating the Cost to Fix a Defect

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by JeffNyman:
Of course, a good proactive approach to this situation is also to determine the expectations for this sale.
<HR></BLOCKQUOTE>

In this case, the VP of Sales felt that he was publishing his expectations (by indicating 50%). But in truth, the number had nothing to do with reality. It was just there because he was told he had to come up with a number to make the sales projections work.

<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>
However, if no one bothers to look into this, then, of course, sales projections have no intrinsic value except as wishes.
<HR></BLOCKQUOTE>

Agreed. Often though, this is exactly the type of projection I see.

From these projections, Engineering is supposed to decide where to allocate their resources. We must attempt to decide to fix or not fix defects that will impact the sales that are projected.

The decisions get made, the project goes on. Some sales are made. Some are lost.

------------------
- Joe (strazzerj@aol.com)

[This message has been edited by jstrazzere (edited 02-20-2002).]

10. ## Re: Calculating the Cost to Fix a Defect

Joe, since you seem to have a stonger opinion than some as to the cost of *not* fixing defects, do you have ideas as to how you'd go about calculating that type of cost?

I guess I'm having trouble coming up with one, based on your comments - if the projections of sales are incorrect or 'fuzzy' to begin with, then it would be difficult to build any real numbers off those.

The only thing I can think of would be to track customer service and sales issues or responses from customers that can be tied directly to a defect in the application, but even then, you'd only be getting a sub-set, because that wouldn't cover the people who left for the same reason, but didn't say so..

------------------
~ Annemarie Martin ~
annemarie[dot]martin2[at]verizon[dot]net

~Those who dream walk in stardust, and dance to the rythm of the heart~

Page 1 of 2 12 Last

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.