The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Quality Engineering >> Defect Tracking

Pages: 1
Ganesh78
Newbie


Reged: 09/06/06
Posts: 5
Defect metrics Benchmarks or industry standards
      #491774 - 06/16/08 03:48 AM

Hi,
Is there any industry standard or benchmark for defect metrics like Defect density, Defect Rejection ratio, defect leakage etc. Are there organization/common bodies which publish these types of benchmark for QA. The rational being, wanted to compare our metrics with industry standard to know where we stand. Primarily looking for Web based applications.


Post Extras: Print Post   Remind Me!   Notify Moderator  
brentpaine
Veteran


Reged: 03/09/07
Posts: 3755
Loc: Waterloo, Ontario, Canada
Re: Defect metrics Benchmarks or industry standards [Re: Ganesh78]
      #491808 - 06/16/08 05:33 AM

I don't think there are really any standards that would give you the answer you're looking for, no.

Also, I don't think that this type of statistic would be useful unless each of the projects reported were reporting each of the metrics mentioned.

Also, there are outside factors related to environment (aren't controlled) for Rejection ratio, so you really can't take someone else's number for this metric as some organizations could be more strict than your own, for instance. In which case you would not likely see the same rejection ratio.

Also, there are organizational facotrs surrounding defect leakage. What is the deadlien for the project? How many QA people are working on it? How many test cycles has it been through?

I think you'll find it difficult to find any sort of statistic, like you're looking for, but even if you do, you'll need to take it with a grain of salt because even if you're on par with them you might find that it's not good enough in your organization. Or, maybe you simply can't live up to the metrics that are being reported.

This doesn't mean you're software is any worse or better than the ones reported in the statistics, just that the numbers aren't in your favor. I generally review how everyone on the project feels about the software. You might think it's funny, but the people who are actaully testing the software will usually have a better idea of how the software feels than any sort of metric. This should determine whether the project is done. Sometimes the metrics are better used for toilet paper than anything else

--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------


Post Extras: Print Post   Remind Me!   Notify Moderator  
jmg
Advanced Member


Reged: 06/11/08
Posts: 583
Re: Defect metrics Benchmarks or industry standards [Re: brentpaine]
      #491911 - 06/16/08 08:50 AM

Hi,

I also don't know of any industry benchmark or published metrics for this sort of data.

On the other hand I've had very good experience using these types of metrics and benchmarks. I started using it on a company I worked about 4 years ago, where we deployed this system/process on all the Internal Teams. We started by comparing the numbers from consecutive versions for each team, and also by benchmarking similar products been released by different teams.

Since we didnt find any initial numbers, the way we started was by sitting with the development, customer support and product marketing, and defining arbitrary (but logical!) numbers that we though should be achieved by all our projects (e.g. 5% rejection, 5% reopened, 0.7% escaping over 6 months that jumped to 1% over 1 year, etc). We started reporting on these numbers once a month and tracing the results.

After 6 months we did the first phase of fine-tuning, then another one after 1 year. By then we understood that we had the magic numbers for our company.

I really recommend doing this, but only if your development process is mature and people will start doing productive changes based on the information. It is very easy to take the numbers and start making excuses for why was the team of their mark.

You can read a bit more about what we did with escaping defects, rejected and reopened from my Blog; but the essence is here.

--------------------
-joel
9 times out 10, less is actually more

PractiTest - QA and Test Management Tool
PractiTest QABlog - Testing Blog


Post Extras: Print Post   Remind Me!   Notify Moderator  
LonnieFranks
Newbie


Reged: 03/23/10
Posts: 1
Re: Defect metrics Benchmarks or industry standards [Re: jmg]
      #619120 - 03/23/10 06:44 AM

A book by Caper Jones, Applied Software Measurement, contains almost all of these benchmarks. Most of the measure are in terms of xxx/function point. Of course for Java there are about 50 lines of code per function points.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Yolonda
Junior Member


Reged: 07/16/03
Posts: 4
Loc: Plano, Texas
Re: Defect metrics Benchmarks or industry standards [Re: LonnieFranks]
      #715919 - 09/05/12 10:55 AM

Is there an acceptable defect leakage to UAT?

--------------------
YKennedy


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: Defect metrics Benchmarks or industry standards [Re: Yolonda]
      #715936 - 09/05/12 12:48 PM

Quote:

Is there an acceptable defect leakage to UAT?



Sometimes, yes.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
John_Ruberto
Newbie


Reged: 07/11/12
Posts: 19
Re: Defect metrics Benchmarks or industry standards [Re: Joe Strazzere]
      #716119 - 09/07/12 04:06 PM

Capers Jones has a report from 2011

Here is the webinar: http://blog.smartbear.com/software-quali...th-Capers-Jones

--------------------
My Blog


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 7 anonymous users are browsing this forum.

Moderator:  AJ, Daniel_S, swt88 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 31366

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5