| || |
Test Effectiveness Metrics
here are some test effectiveness metrics that i have collected.. let me know if they are useful to you..
send in your comments to :-->
( I ) Process to measure the Efforts spent on a Project
we can use the Time Sheet for tracking the effots
Name of Employee Project Code No. of Hours
xyz P1 150
abc P1 40
Total Efforts spent on a Project P1= 150 + 40 =190
( II ) Process to measure the Quality of Testing work done on Project.
There are 3 types of costs that are associated with a Process.
a) Failure costs: the costs of diagnosing a failure, making necessary repairs, and getting back into operation.
b) Appraisal costs: the costs of evaluating the product to determine its quality level.
c) Prevention costs: the costs associated with identifying the causes of the defects and the actions taken to prevent them in the future.
The PSP(Personal Software Process) Cost of Quality Measures -->
Failure COQ=100*(compile time + test time)/ (total development time)
Appraisal COQ=100* (design review time + code review time)/(total development time)
Hence, Total COQ= Appraisal COQ + Failure COQ
Product Quality Metrics
Defect Density Metrics = Defects / KLOC
KLOC can be measured as :
(i) executable lines
(ii) executable lines + data definitions + comments (IBM)
Customer Problems (Defects ) Metric
PUM ( Problems per user month)
= Total problems (defects) customers reported for a time period / Total number of license months
Number of license months is measured as:
number of install licenses * number of months in calculation period.
Metrics to Measure Test Effectiveness
I guess every situation is different. but..
We simply assign a Business Impact to an open defect. We then relate that Impact to the revenue lost if this defect went into production. In a large telecom environment this can easily be a million dollar defect. or "thousands of dissatisfied customers" which will lead to an est. revenue loss. This is how we sustain our arguements for testing.
Test Effectiveness Metrics
Defect No. Project Code Business Impact Revenue Impact
101 P2 High High
109 P2 Low Low
Customer Satisfaction Description
High No Complaints, Customer buys more company products,asks for more services from the company.
Medium Customer Satisfied,does not have option/money to change the product/system so uses it.
Poor Customer logs complaints against the product/system,demands free servicing,maintenance or enhancements to the delivered system,stops future payments.
Impact Loss of Revenue Per Quarter Estimated Revenue Earnings Per Quarter
High $20,000 – $50,000 $70,000-$90,000
Medium $10,000 - $20,000 $35,000- $45,000
Low $2,000 - $5,000 $10,000 - $18,000