Eric_Zhao
Newbie
Reged: 12/04/08
Posts: 15
|
|
Hello,
Is there a code coverage analysis tool for manual testing? I know there're many tools for unit test. But it needs instrumented builds, which can be done by developers only.
I'm looking for a tool that testers can run to analyze code coverage. Testers don't have access to the source codes - can only see the .exe files.
If there's no way to do code coverage without access to source code, how do you do usually?
Thank you 
Edited by Eric_Zhao (07/29/10 10:13 PM)
|
TestingMentor
Member
Reged: 12/28/06
Posts: 201
Loc: Seattle, Washington
|
|
Hi Eric,
Once your project is instrumented for coverage analysis you can collect code coverage information via manual or automated testing.
You can gather code coverage information from .NET assemblies without instrumenting the builds by using tools such as .NET Coverage Analyzer http://www.softwareverify.com/dotNet/coverage/index.html, but you will at least need the .pdb file.
I am not sure how effective this tool is as I have not used it. Different tools may measure coverage differently (e.g. block coverage vs. function vs. condition, etc.)
The greatest value of code coverage is that it tells us what code has been untested. Of course, if we know what code is untested then we need to analyze the code in order to determine whether we need tests to hit that code, and how to design tests to exercise those untouched areas of code.
If you don't execute a line of code then you don't know if a bug exists or not. But, just because you exercise a line of code doesn't mean a bug doesn't exist.
--------------------
- Bj -
I.M. Testy blog
Testing Mentor
|
Eric_Zhao
Newbie
Reged: 12/04/08
Posts: 15
|
|
Hi BJ,
Thanks for your reply. I have downloaded .NET Coverage Analyzer and tried for several small .NET projects. Emm, it's good.
Actually i'm looking for some tools that tester (without code access) can use directly to analyze code coverage. Tester doesn't have the instrumented products. In this case, tester is not able to see the .pdb file. So .NET Coverage Analyzer is not actually what I need.
Now I'm wondering if any tools do exist that tester can use without instrumented builds?
Thanks again!
Eric
|
Peter Ruscoe
Veteran
Reged: 03/18/02
Posts: 6913
Loc: Tampa Bay
|
|
I can't see how that would be possible, Eric. Compiled code (without embedded tags designed for the purpose - i.e. instrumented) has no way to indicate meaningful information about what has been tested and what hasn't. (Unless, of course, the original app was hand-coded in machine code - or at least assembly language!)
The only thing any such tool (if it existed) could tell you is where chunks of object code (which you clearly couldn't relate back to source code easily) have not been covered.
Actually, I'm having a hard time understanding the concept of (or even the necessity for) code coverage if you don't have access to the code itself. Code coverage only has a meaning when related to the original source.
What would the tester (who has no access to the source) do with the output of a code-coverage tool?
|
Joe Strazzere
Moderator
Reged: 05/15/00
Posts: 10116
Loc: Massachusetts, USA
|
|
Quote:
What would the tester (who has no access to the source) do with the output of a code-coverage tool?
Magic metrics numbers?
--------------------
- Joe
I speak only for me. I do not speak for my employer, or for anyone else.
Visit my new blog All Things Quality
|
Peter Ruscoe
Veteran
Reged: 03/18/02
Posts: 6913
Loc: Tampa Bay
|
|
Hey, now there's an idea 
|
Walen
Active Member
Reged: 05/09/01
Posts: 1064
|
|
Harry? Are you here Harry? Where's he gone?
--------------------
P. Walen
|
Shane_MacLaughlin
Super Member
Reged: 09/22/05
Posts: 1502
Loc: Dublin, Ireland
|
|
Quote:
Compiled code (without embedded tags designed for the purpose - i.e. instrumented) has no way to indicate meaningful information about what has been tested and what hasn't.
Not quite true Peter. As per BJs post, some tools can use an uninstrumented executable and an external debug file such as a PDB or detailed map file. i.e. the tags aren't embedded, they're in a seperate external file. From a developer perspective, this is great as you don't need a seperate instrumented build. Sometimes instrumentation will given a more accurate result, but for many situations it is neither necessary or desirable. Outside of the tool BJ lists, AutomateQA's AQTime works this way and from my experience works pretty well.
--------------------
My LinkedIn profile
|
Peter Ruscoe
Veteran
Reged: 03/18/02
Posts: 6913
Loc: Tampa Bay
|
|
OK, I was a little sloppy in my statement, Shane. But the fact remains that code coverage is only meaningful if something ties the executable code to the source code and (b)my point about a tester with no access to the source still stands.
|
Previous
Index
Next
Threaded
Edit
Reply
Quote
