The online community for software testing & quality assurance professionals

 Web SQAforums.com
Calendar   Today's Topics
 You are not logged in. [Login] Main Index · Search · Active Topics New user · Who's Online · FAQ · Calendar

Quality Engineering >> Estimation and Planning

Pages: 1
abhi1234
Member

Reged: 05/17/07
Posts: 70
Loc: Nashville, TN
How to calculate total % of Regression Testing
#425907 - 10/19/07 11:28 AM

Folks,
I create weekly QA Summary report which include test status, showstopper defects, QA schedule and total System Testing complete percentage. Here is how I calculate my system testing %......

Example: Software name XYZ, which has three modules: A, B and C.

We write test cases for module A, B and C. Let's assume we have total of 90 test cases. 30 for A, 30 for B and 30 for C. Having said that, everytime new release comes out. We run those 90 test cases. For first release, out of 90 test cases, 5 of them failed and 85 passed. Then I calculate System Testing % by calculating: 85/90 * 100 which equals to 94%. Hence, system testing is at 94%.

Now, for the second release those 5 failed bugs will be fixed. Also, technically we will be running "regression testing" for that cycle. Now, question is how do I calculate regression testing percentage?

Also, we run the system testing cycle second time for this release. Let's say this time around out of 5 bugs, 3 of them are fixed and 2 are unfixed. Then total System Testing is 88/90 * 100 = 97%.

But, I am still confused how to calcuate regression testing. Techically second cycle of system testing is regression testing. Does that means Regression testing is 97% OR Regression testing is 3/5 * 100 = 60%. Which one is correct? I know that regression testing doesn't mean retesting of fixed defect but QA also has to run entire system testing again. That's why I am confused about how to express regression testing % on QA Summary Report?

Thanks

--------------------
Lean and Mean machine

 Post Extras:
Richard_Hunter
Member

Reged: 08/16/04
Posts: 388
Re: How to calculate total % of Regression Testing [Re: abhi1234]
#426168 - 10/22/07 02:05 AM

Hi,

The way you discribe it is that the whole thing is regression testing, you always run the same test cases, so I would say you are 97% through regression testing. System testing would be testing any new functionality, so I would lay it out as below:
System Testing Cycle 1 - Run everything
System Testing Cycle 2 - Re-run the failures and any test cases not run due to other failures
System Testing Cyclle 3-x - Re-run failures until 100% or good enough

Then start on regression testing.

Richard

--------------------
Richard Hunter

 Post Extras:
Super Member

Reged: 12/03/06
Posts: 1937
Re: How to calculate total % of Regression Testing [Re: Richard_Hunter]
#426174 - 10/22/07 02:28 AM

Well ,it depends if you have version control process in place, if A is dependent on B and so , then i feel that you might be correct in doing so, but still you need to take sub modules and their dependency on other modules in consideration for regression testing,
with my limited knoweledge on functional side, i would say that you figure the dependency among the modules and decide the number of test cases to be executed, i am sure it will not be more than 20 or 30 % of overall test cases.but you need to have tight version control system in place for that.
i am assuming that you are fixing some issues in previous run and retesting it rather than testing if new functionalities are added to the bulid.

 Post Extras:
abhi1234
Member

Reged: 05/17/07
Posts: 70
Loc: Nashville, TN
Re: How to calculate total % of Regression Testing [Re: Richard_Hunter]
#426260 - 10/22/07 06:17 AM

Quote:

System Testing Cycle 1 - Run everything
System Testing Cycle 2 - Re-run the failures and any test cases not run due to other failures
System Testing Cyclle 3-x - Re-run failures until 100% or good enough

Then start on regression testing.

Richard

According to you, I should continue running System Testing cycles until all the defects are fixed (all the high/critical defects NOT the low ones or enhancements)? Then, I should execute one cycle of regression testing. Did I understand it correctly?

I was thinking that I should run one cycle of System Testing and then whatever fixes are deployed, they all will be tested under "Regression Testing" plus the entire cycle of Systems Testing.

--------------------
Lean and Mean machine

 Post Extras:
Richard_Hunter
Member

Reged: 08/16/04
Posts: 388
Re: How to calculate total % of Regression Testing [Re: abhi1234]
#426488 - 10/23/07 03:11 AM

Hi,

The way ew do it is as you describe, run system testing until I have passed everything at least once(or agreed that outstanding issues are going live). Then re-run everything as a regression test.

I think this is the most efficient way, re-running everything each release takes a lot of time, and in many cases is not required. Obviously when you retest you do a little but of regression testing around the area but not a full go. We have found that this is good enough to then be able to confirm everything is working with a final regression test.

However this is possibly not a good fit for every project. If you find that when you go through a whole system test you are finding sicnificant errors in places that previously worked then perhaps retesting everything is required, I suppose you have to try and work out the risk of not re-testing everything against the time this takes.

--------------------
Richard Hunter

 Post Extras:
TigerEye
Member

Reged: 02/26/07
Posts: 78
Loc: USA - Maryland
Re: How to calculate total % of Regression Testing [Re: abhi1234]
#483947 - 05/16/08 11:56 AM

You seem to be confusing the passing rate with test completion rate. The testing is considered completed when the test cases are executed. Unless, there were some blocker level items that prevented from execution of even one single test step, thus stopping the test effort. So, if your team had been able to execute all test cases, then you can report that 100% of testing is complete.

To determine the test completion rate, I think, there are two approaches. The first one is to multiply the total number of test cases by 100 and divide by the number of executed test cases. The second one is to assign weight to each test case, according to their overall mission importance, and then apply the same formula. For example, the three test cases that test content spelling, graphics, may be even less of importance than the single test case that tests database data retention accuracy.

Edited by TigerEye (05/16/08 11:57 AM)

 Post Extras:
SQA_Freak
Newbie

Reged: 07/01/08
Posts: 8
Re: How to calculate total % of Regression Testing [Re: TigerEye]
#499347 - 07/11/08 11:54 PM

From what i understand, I would suggest you take this approach:

i. In System testing I - you have only 5 failed cases out of the possible 90, which means you have 94% as pass rate.

ii. Those 5 cases which have failed (assume there are 5 defects that are attached to these 5 cases). You would retest these 5 cases (5 defects).

At this point you would give the retest pass rate rather than add these 5 cases to your regression number.

Always remember, retest and regression are different.(eventhough they are a part of confirmation testing)

Retest - Always associated with checking if the defect is fixed by the team incharge.

Regression - This is to check if the unmodified/unchanged parts of the code are affected due to defect fixes.

In System testing II - Below would be the split (based on assumption)

i. Retest Test cases : 5

-- 4 Passes
-- 1 Failes
----------------
Pass % for Retest - 80%
----------------

ii. Regression (assume you have identified 40 regression test cases from the remaining 85 which has passed from the first instance)

Total nr. for regression - 40

Passed: 38
Failed: 2
----------

Pass % = 38/40 * 100= 95%

It is always better to highlite retests and regression separately since they are a lot different from each other. Giving individual numbers for these activites would give a better perspective to the management.

 Post Extras:
Inder_P_Singh
Member

Reged: 06/18/08
Posts: 115
Loc: India
Re: How to calculate total % of Regression Testing [Re: TigerEye]
#499360 - 07/12/08 03:02 AM

Quote:

The first one is to multiply the total number of test cases by 100 and divide by the number of executed test cases.

This should be the other way round i.e. multiply the number of executed test cases by 100 and divide by the total number of test cases.

Inder P singh

--------------------
Inder P Singh

For more of my thoughts, visit the blog, Software Testing Space @ http://inderpsingh.blogspot.com/

 Post Extras:
Member

Reged: 07/25/07
Posts: 43
Re: How to calculate total % of Regression Testing [Re: Inder_P_Singh]
#500688 - 07/17/08 05:30 AM

I totall agree with SQA_FREEK. What we are following here is like this.

System Cycle 1 - Execute all test cases - ie 90, and in that 85 got pass and 5 got fail. We will log the 5 defects, if its are different problems.

System Cycle 2 - Determenine or shortlist areas, that can have impact due to the fixes of 5 defects, and add those to the System Test Cycle 2 Test Suite. Suppose I identified 20 test cases.

So i will execute 20 TC + 5 failed TC. Suppose now 1 TC got failed and others all passed.

System Cycle 3 - Determine or shortlist areas that can have impact due to the fix of that lonely deect, and add those to the System TEst Cycle 3 Suite, Suppose i identified 3 TC.

like that cycle will go on and on...

 Post Extras:
JakeBrake
Moderator

Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to calculate total % of Regression Testing [Re: madhuqtp]
#500727 - 07/17/08 07:00 AM

Quote:

... System Cycle 1 - Execute all test cases - ie 90, and in that 85 got pass and 5 got fail. We will log the 5 defects, if its are different problems...

Aside from upstream test design/case review practices, should there be a step here prior to logging defects that would require one to examine the test to see if it is current and correct?

 Post Extras:
SQA_Freak
Newbie

Reged: 07/01/08
Posts: 8
Re: How to calculate total % of Regression Testing [Re: JakeBrake]
#505259 - 08/01/08 09:09 PM

Hi Jake, Unless or until the test cases are basedlined you wouldnt start executing the same. (unless otherwise specified in SoW)

There is a possibility that these test cases needs to be modified based on CM (Change management). If defects raised out of the failed test cases arent valid due to change in the initial test requirements (eventually the test cases) then these defects needs to moved to deferred till the test cases are modified and then retested after CM is approved.

Once the the retest is successful, you close the defect with the defect catogery as "Change requirement" or "Requirement error" or anything that would indicate that this defect belongs to the requirement phase raised during other phases (SIT,UAT,UPT etc.,)

Edited by SQA_Freak (08/01/08 09:10 PM)

 Post Extras:
TnkerBell
Active Member

Reged: 09/01/08
Posts: 1051
Loc: Neverland
Re: How to calculate total % of Regression Testing [Re: SQA_Freak]
#528444 - 10/31/08 05:11 AM

To calculate the regression testing :

Count of the test cases/Average effort for each task for each type of complexity

You can have a Run Chart for Number of Test Cases run per each task for each type of complexity

Regards,
Tinker

--------------------
Regards,
Tinker

"The most wasted of all days is one without laughter."
-e e cummings

 Post Extras:
Pages: 1

Extra information
0 registered and 5 anonymous users are browsing this forum.

Print Topic

Forum Permissions
You cannot start new topics