The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Software Testing >> Unit Testing

Pages: 1
oliviabean
Member


Reged: 01/08/06
Posts: 56
How to measure the Unit test
      #420402 - 10/01/07 01:24 AM

When doing unit testing in your projects, how do you measure your unit testing. Usually people collect test coverage data, are there other metrics?
I'm a QA doing functional tests, the developers do the unit tests. But I collect metrics for both functional test and unit test.
Our manager insist on getting the data of "unit test cases required","unit test cases defined" for each class/method, to measure the completeness of the unit test work done.
But I don't think it's a good way to measure unit test. For functional tests, we can say that for each transaction, there should be at least one positive test case and one negative, so the minumum test cases required for each requirement/transaction is two. For unit testing, how can a QA tell how many unit test cases are required for a class/method. That requires thorough reading of the code, and that's a actually a code review/walk-through, I don't think there's time for us QA to do so.
What do you think?

--------------------
Olivia,Dou


Post Extras: Print Post   Remind Me!   Notify Moderator  
JakeBrake
Moderator


Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to measure the Unit test [Re: oliviabean]
      #420421 - 10/01/07 02:51 AM

Quote:

Olivia: When doing unit testing in your projects, how do you measure your unit testing. Usually people collect test coverage data, are there other metrics? I'm a QA doing functional tests, the developers do the unit tests. But I collect metrics for both functional test and unit test. Our manager insist on getting the data of "unit test cases required", "unit test cases defined" for each class/method, to measure the completeness of the unit test work done. But I don't think it's a good way to measure unit test.


Olivia, It appears that the goal is to reduce the introduction of defects correct? It also appears that your manager wishes for a statement of test coverage or exposure - not to be confused with code coverage. If you have existing defect metrics, the following suggestions would help, and if not it would be good to implement the means by getting the data for some of the following:
Suggestions for identifying units that must be tested assuming all cannot be tested:
= Identify the most complex (complex algorithms, etc.)
= Identify any timing-critical units
= Identify those most subject to change most used
= Identify those most prone to defects
= Identify those with critical interfaces
= Identify those subject to failure as a result of changes to other components, be it within the application, O/S, or other third party components

Then identify which components have unit tests and at this point in the process, not the quality of the unit tests. This is just a coverage or risk exposure tool at this point. It seems that the development manager should be charged with making this happen. At this point, then one could be concerned with detail metrics.
.
.
Quote:

Olivia: For functional tests, we can say that for each transaction, there should be at least one positive test case and one negative, so the minimum test cases required for each requirement/transaction is two. For unit testing, how can a QA tell how many unit test cases are required for a class/method. That requires thorough reading of the code, and that's a actually a code review/walk-through, I don't think there's time for us QA to do so. What do you think?


I think your manager is somewhat correct however, I think your manager is also missing the mark somewhat. Code reviews/walk-throughs are an effective means by which to catch defects. In many cases these activities at this point in the SDLC represent another first opportunity to catch defects. To miss this opportunity is to allow defects a chance of showing up when you touch the code and therefore costing you more time. QA as it appears in your case should not require you to do code reviews, etc. This should be required of the development team and at a minimum, per the list above. In short, omission of code reviews/walkthroughs is incomplete work that shows up later on someone elses doorstep who may or may not be qualified to deal with it. Anyway, here is a very good site that provides the bases for the detailed metrics.

http://www.bullseye.com/coverage.html


Post Extras: Print Post   Remind Me!   Notify Moderator  
SteveHeidstra
Member


Reged: 10/28/04
Posts: 280
Loc: The Netherlands
Re: How to measure the Unit test [Re: JakeBrake]
      #420444 - 10/01/07 04:57 AM

Quote:

not to be confused with code coverage.




It seems to me you are in an excellent position to introduce code coverage! These are extremely useful metrics when using unit tests!

Ofcourse you should find out if code coverage is an option in your organisation. For this you should see what programminglanguage is used and what programming environment. Also: you should know how unit testing is done at this moment.
With this information you should be able to find out if the environment supports automated unit testing or can be modified to do so. If yes: you might want to go ahead and propose the introduction of a code coverage tool. This can provide management with lots of clear metrics and almost always improves the code from the start.


Post Extras: Print Post   Remind Me!   Notify Moderator  
JakeBrake
Moderator


Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to measure the Unit test [Re: SteveHeidstra]
      #420447 - 10/01/07 05:01 AM

Steve, the actual statement was, "It also appears that your manager wishes for a statement of test coverage or exposure - not to be confused with code coverage."

That was intended as a differentiator between a test coverage inventory and "code coverage". The code coverage suggestion is implied in my post by providing the link to an excellent document regarding same.


Post Extras: Print Post   Remind Me!   Notify Moderator  
SteveHeidstra
Member


Reged: 10/28/04
Posts: 280
Loc: The Netherlands
Re: How to measure the Unit test [Re: JakeBrake]
      #420452 - 10/01/07 05:09 AM

Hi Jake,
I did not mean to imply otherwise... your remark promted my response, that is all...

And as for the wishes of management... who knows what they actually want...? Weird as witches to me.... :-)


Post Extras: Print Post   Remind Me!   Notify Moderator  
JakeBrake
Moderator


Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to measure the Unit test [Re: SteveHeidstra]
      #420455 - 10/01/07 05:12 AM

... resulting in a good thing for Olivia - more clarification/details. (team work)

Post Extras: Print Post   Remind Me!   Notify Moderator  
AlfredPeterson
Newbie


Reged: 08/17/10
Posts: 10
Re: How to measure the Unit test [Re: JakeBrake]
      #640419 - 08/29/10 10:40 PM

Well recently i had gone threw the another very good tool the Jester for measuring the code coverage...

and some one had told me its some benefits..like this


Error seeding: changes source code; then recompiles

Does not instrument code

Upside: really finds out if code is tested, not merely exercised

Downside: Takes a long time to run.
Prerequisites

If tests break, that line is tested; otherwise it's not.

How all these implies ....

--------------------
[URL="http://www.dozenvideo.com/Video-Downloads-FLOP/emulecom.html"]emule[/URL]


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 0 anonymous users are browsing this forum.

Moderator:  AJ, Jeanj 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 6356

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5