Defect-removal efficiency is the percentage of software “bugs” that are removed from a product prior to the product being released into production.
This quality metric has been in use at “best of class” companies since the 1960s. These companies are considered “best of class” because they maintain high customer satisfaction rates by consistently producing high quality software. Defect-removal efficiency is a method used to measure “quality”. The method is based upon the number of defects that escape detection by a development team. Defect-removal efficiency is one method that can be used to measure the effectiveness of a development team.
A project's defect-removal efficiency is very simple to compute. During the development effort, all defects that are found are recorded. After the software program is released into production, any problems that are found by the customer are also recorded. After a specific period of time, the total number of defects found during the development effort is divided by the number of defects found (pre and post release). For example, if a project records 180 defects during the development effort and 20 defects post-live, then the project's defect-removal efficiency would be 180/200 or 90 percent.
Well-known Software Productivity Researcher, Capers Jones, has published data that documents the U.S. average for defect-removal efficiency is only 85 percent.
He also documents that there are some companies that achieve rates as high or higher than 99 percent on their “best” projects. These “best” projects use formal pre-test inspection methods of design, code and other major deliverables to prevent issues escaping from the development phase and entering into the production product.
Mr. Jones has published several papers with ranges for defect-removal efficiencies for various development activities. These ranges are based upon research derived from thousands of projects. Mr. Jones’ research illustrates:
Ø System Testing alone has an efficiency range of 25% to 55%
Ø Formal Code Inspections alone have an efficiency rating of 45% to 70%
Ø Formal Design Inspections alone have an efficiency rating of 45% to 65%.
When these three practices are used together, Mr. Jones’ research proves that a project can achieve an efficiency rating of 85% to 99%.
Of course, a high defect-removal efficiency rating does not guarantee success. It is still possible to have dissatisfied customers for other reasons. However, Capers' research shows that there is a strong correlation between a high defect-removal efficiency rating and customer satisfaction.
----source material is from an article by Capers Jones in Computer Mag.