I have been tasked with performing defect cost analysis here at work. Here's where I'm currently at with my initial rrequirements gathering & brainstorming efforts (it is my hope that others will benefit here):
- The defect value will be generic/average defect costs
- We want as much data as possible for a "defensible" cost analysis
- Cost assessments will be across various phases in the SDLC (Requirements, Unit Testing, Integration & System Testing, Production)
- Amount of work needs to be determined which "may" include the following time variables: testing, daily defect calls (includes triage/management resolution), development, retest & closure
- Defect history can be used to detect time periods (when opened, assignments, status change, etc)
- We will probably go back and analyze the past 2-3 releases
- Phase durations will be determined based upon release dates
- Average cost to fix a defect will be based upon the following formula:
(# of People <times> # of days) <times> (cost person-day) <divided by> #Fixed Defects
There has been some discussion on using costing units which uses industry standard multipliers (the multiplier is applied to the actual numbers we get in our cost assessment). I'm not familiar with costing units & maybe someone can elaborate on this?
Q1: Is it critical to have specific applications in mind with this analysis or should I just pick random defects and track the values?
Q2: What significance can be found with defects when analyzing product versus project? (if this is even necessary given Q1 above)
Q3: Are there other industry resources available for defect cost analysis? (ie., good web sites or white papers)
This is good for me just to do a brain dump on this issue. If I'm missing something above or if anyone has any wisdom on approaching this issue, please fire away. (please be professional in your responses)
Re: Defect Costs
This is a massive undertaking. Your organisation will need to have tracked data to a very detailed level for things such defect frequencies per stage, severity, man effort per defect... the list of factors goes on. If you do not have this data, the best you can do is guess.
The other thing that makes this sort of thing difficult is that it is near impossible to quantify the "hidden cost" of the defects i.e. how many sales opportunities did we lose because a bug was found in a trial period. How did the defects that went live cost the company in terms of potential revenue etc.
Many people have tried this sort of thing but in the main the costs involved in getting an accurate measure tend to result in these types of project being canned.
On the upside, although published nearly 20 years ago there is the "Boehm Model" which is used by pretty much everyone to present the relative costs to resolve defects per stage. Taking a look at this and applying similar principles to your envrionment will likely save you a lot of time and effort