The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Quality Engineering >> Estimation and Planning

Pages: 1
abhi1234
Member


Reged: 05/17/07
Posts: 70
Loc: Nashville, TN
How to calculate total % of Regression Testing
      #425907 - 10/19/07 11:28 AM

Folks,
I create weekly QA Summary report which include test status, showstopper defects, QA schedule and total System Testing complete percentage. Here is how I calculate my system testing %......

Example: Software name XYZ, which has three modules: A, B and C.

We write test cases for module A, B and C. Let's assume we have total of 90 test cases. 30 for A, 30 for B and 30 for C. Having said that, everytime new release comes out. We run those 90 test cases. For first release, out of 90 test cases, 5 of them failed and 85 passed. Then I calculate System Testing % by calculating: 85/90 * 100 which equals to 94%. Hence, system testing is at 94%.

Now, for the second release those 5 failed bugs will be fixed. Also, technically we will be running "regression testing" for that cycle. Now, question is how do I calculate regression testing percentage?

Also, we run the system testing cycle second time for this release. Let's say this time around out of 5 bugs, 3 of them are fixed and 2 are unfixed. Then total System Testing is 88/90 * 100 = 97%.

But, I am still confused how to calcuate regression testing. Techically second cycle of system testing is regression testing. Does that means Regression testing is 97% OR Regression testing is 3/5 * 100 = 60%. Which one is correct? I know that regression testing doesn't mean retesting of fixed defect but QA also has to run entire system testing again. That's why I am confused about how to express regression testing % on QA Summary Report?

Thanks

--------------------
Lean and Mean machine


Post Extras: Print Post   Remind Me!   Notify Moderator  
Richard_Hunter
Member


Reged: 08/16/04
Posts: 388
Re: How to calculate total % of Regression Testing [Re: abhi1234]
      #426168 - 10/22/07 02:05 AM

Hi,

The way you discribe it is that the whole thing is regression testing, you always run the same test cases, so I would say you are 97% through regression testing. System testing would be testing any new functionality, so I would lay it out as below:
System Testing Cycle 1 - Run everything
System Testing Cycle 2 - Re-run the failures and any test cases not run due to other failures
System Testing Cyclle 3-x - Re-run failures until 100% or good enough

Then start on regression testing.

Does that add up?

Richard

--------------------
Richard Hunter


Post Extras: Print Post   Remind Me!   Notify Moderator  
kiranbadi1991
Super Member


Reged: 12/03/06
Posts: 1937
Re: How to calculate total % of Regression Testing [Re: Richard_Hunter]
      #426174 - 10/22/07 02:28 AM

Well ,it depends if you have version control process in place, if A is dependent on B and so , then i feel that you might be correct in doing so, but still you need to take sub modules and their dependency on other modules in consideration for regression testing,
with my limited knoweledge on functional side, i would say that you figure the dependency among the modules and decide the number of test cases to be executed, i am sure it will not be more than 20 or 30 % of overall test cases.but you need to have tight version control system in place for that.
i am assuming that you are fixing some issues in previous run and retesting it rather than testing if new functionalities are added to the bulid.


Post Extras: Print Post   Remind Me!   Notify Moderator  
abhi1234
Member


Reged: 05/17/07
Posts: 70
Loc: Nashville, TN
Re: How to calculate total % of Regression Testing [Re: Richard_Hunter]
      #426260 - 10/22/07 06:17 AM

Quote:


System Testing Cycle 1 - Run everything
System Testing Cycle 2 - Re-run the failures and any test cases not run due to other failures
System Testing Cyclle 3-x - Re-run failures until 100% or good enough

Then start on regression testing.

Does that add up?

Richard




According to you, I should continue running System Testing cycles until all the defects are fixed (all the high/critical defects NOT the low ones or enhancements)? Then, I should execute one cycle of regression testing. Did I understand it correctly?

I was thinking that I should run one cycle of System Testing and then whatever fixes are deployed, they all will be tested under "Regression Testing" plus the entire cycle of Systems Testing.

--------------------
Lean and Mean machine


Post Extras: Print Post   Remind Me!   Notify Moderator  
Richard_Hunter
Member


Reged: 08/16/04
Posts: 388
Re: How to calculate total % of Regression Testing [Re: abhi1234]
      #426488 - 10/23/07 03:11 AM

Hi,

The way ew do it is as you describe, run system testing until I have passed everything at least once(or agreed that outstanding issues are going live). Then re-run everything as a regression test.

I think this is the most efficient way, re-running everything each release takes a lot of time, and in many cases is not required. Obviously when you retest you do a little but of regression testing around the area but not a full go. We have found that this is good enough to then be able to confirm everything is working with a final regression test.

However this is possibly not a good fit for every project. If you find that when you go through a whole system test you are finding sicnificant errors in places that previously worked then perhaps retesting everything is required, I suppose you have to try and work out the risk of not re-testing everything against the time this takes.

--------------------
Richard Hunter


Post Extras: Print Post   Remind Me!   Notify Moderator  
TigerEye
Member


Reged: 02/26/07
Posts: 78
Loc: USA - Maryland
Re: How to calculate total % of Regression Testing [Re: abhi1234]
      #483947 - 05/16/08 11:56 AM

You seem to be confusing the passing rate with test completion rate. The testing is considered completed when the test cases are executed. Unless, there were some blocker level items that prevented from execution of even one single test step, thus stopping the test effort. So, if your team had been able to execute all test cases, then you can report that 100% of testing is complete.

To determine the test completion rate, I think, there are two approaches. The first one is to multiply the total number of test cases by 100 and divide by the number of executed test cases. The second one is to assign weight to each test case, according to their overall mission importance, and then apply the same formula. For example, the three test cases that test content spelling, graphics, may be even less of importance than the single test case that tests database data retention accuracy.



Edited by TigerEye (05/16/08 11:57 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
SQA_Freak
Newbie


Reged: 07/01/08
Posts: 8
Re: How to calculate total % of Regression Testing [Re: TigerEye]
      #499347 - 07/11/08 11:54 PM

From what i understand, I would suggest you take this approach:

i. In System testing I - you have only 5 failed cases out of the possible 90, which means you have 94% as pass rate.

ii. Those 5 cases which have failed (assume there are 5 defects that are attached to these 5 cases). You would retest these 5 cases (5 defects).

At this point you would give the retest pass rate rather than add these 5 cases to your regression number.

Always remember, retest and regression are different.(eventhough they are a part of confirmation testing)

Retest - Always associated with checking if the defect is fixed by the team incharge.

Regression - This is to check if the unmodified/unchanged parts of the code are affected due to defect fixes.

In System testing II - Below would be the split (based on assumption)

i. Retest Test cases : 5

-- 4 Passes
-- 1 Failes
----------------
Pass % for Retest - 80%
----------------

ii. Regression (assume you have identified 40 regression test cases from the remaining 85 which has passed from the first instance)

Total nr. for regression - 40

Passed: 38
Failed: 2
----------

Pass % = 38/40 * 100= 95%

It is always better to highlite retests and regression separately since they are a lot different from each other. Giving individual numbers for these activites would give a better perspective to the management.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inder_P_Singh
Member


Reged: 06/18/08
Posts: 115
Loc: India
Re: How to calculate total % of Regression Testing [Re: TigerEye]
      #499360 - 07/12/08 03:02 AM

Quote:

The first one is to multiply the total number of test cases by 100 and divide by the number of executed test cases.




This should be the other way round i.e. multiply the number of executed test cases by 100 and divide by the total number of test cases.

Inder P singh

--------------------
Inder P Singh

For more of my thoughts, visit the blog, Software Testing Space @ http://inderpsingh.blogspot.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
madhuqtp
Member


Reged: 07/25/07
Posts: 43
Re: How to calculate total % of Regression Testing [Re: Inder_P_Singh]
      #500688 - 07/17/08 05:30 AM

I totall agree with SQA_FREEK. What we are following here is like this.

System Cycle 1 - Execute all test cases - ie 90, and in that 85 got pass and 5 got fail. We will log the 5 defects, if its are different problems.

System Cycle 2 - Determenine or shortlist areas, that can have impact due to the fixes of 5 defects, and add those to the System Test Cycle 2 Test Suite. Suppose I identified 20 test cases.

So i will execute 20 TC + 5 failed TC. Suppose now 1 TC got failed and others all passed.

System Cycle 3 - Determine or shortlist areas that can have impact due to the fix of that lonely deect, and add those to the System TEst Cycle 3 Suite, Suppose i identified 3 TC.

like that cycle will go on and on...


Post Extras: Print Post   Remind Me!   Notify Moderator  
JakeBrake
Moderator


Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to calculate total % of Regression Testing [Re: madhuqtp]
      #500727 - 07/17/08 07:00 AM

Quote:

... System Cycle 1 - Execute all test cases - ie 90, and in that 85 got pass and 5 got fail. We will log the 5 defects, if its are different problems...



Aside from upstream test design/case review practices, should there be a step here prior to logging defects that would require one to examine the test to see if it is current and correct?


Post Extras: Print Post   Remind Me!   Notify Moderator  
SQA_Freak
Newbie


Reged: 07/01/08
Posts: 8
Re: How to calculate total % of Regression Testing [Re: JakeBrake]
      #505259 - 08/01/08 09:09 PM

Hi Jake, Unless or until the test cases are basedlined you wouldnt start executing the same. (unless otherwise specified in SoW)

There is a possibility that these test cases needs to be modified based on CM (Change management). If defects raised out of the failed test cases arent valid due to change in the initial test requirements (eventually the test cases) then these defects needs to moved to deferred till the test cases are modified and then retested after CM is approved.

Once the the retest is successful, you close the defect with the defect catogery as "Change requirement" or "Requirement error" or anything that would indicate that this defect belongs to the requirement phase raised during other phases (SIT,UAT,UPT etc.,)

Edited by SQA_Freak (08/01/08 09:10 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
TnkerBell
Active Member


Reged: 09/01/08
Posts: 1051
Loc: Neverland
Re: How to calculate total % of Regression Testing [Re: SQA_Freak]
      #528444 - 10/31/08 05:11 AM

To calculate the regression testing :

Count of the test cases/Average effort for each task for each type of complexity

You can have a Run Chart for Number of Test Cases run per each task for each type of complexity

Regards,
Tinker

--------------------
Regards,
Tinker

"The most wasted of all days is one without laughter."
-e e cummings


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 9 anonymous users are browsing this forum.

Moderator:  swt88, AJ, corklad, blueinatl 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 29806

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5