SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 8 of 8
  1. #1
    Points for Confirmed Friends
    Guest

    Exploratory vs. Planned testing metrics

    Here's the scenario.

    A field has been added to my team's defect tracker called "Source." Two of the choices are "Planned testing" and "Exploratory testing." We will see if this will provide us with some useful metrics.

    Now, here are the questions.

    1) If I run a test case and it passes (ie the result matches the expected result) but I do find a defect, would the source be exploratory testing or planned testing?

    2) If I run a test case and it passes and then I try ONE more step and find a defect, would the source be exploratory testing or planned testing?

    Just want to gather some opinions on this.

    Thanks!

    ------------------


    [This message has been edited by gsh (edited 11-22-2001).]

  2. #2
    Senior Member
    Join Date
    Mar 2001
    Location
    UK
    Posts
    484
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Exploratory vs. Planned testing metrics

    It depends what your metrics are supposed to be useful for.

    Think of metrics as milestones on a process roadmap - to tell you how far you've got and in what direction. Where are you heading?

    ------------------
    Slowly, Slowly catchee monkey...
    Softly, Softly catchee monkey...

  3. #3
    Member
    Join Date
    May 2001
    Location
    Shrewsbury, Shropshire, UK
    Posts
    47
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Exploratory vs. Planned testing metrics

    perhaps you need another source "Vigilant Tester"

    ------------------

  4. #4
    Member
    Join Date
    May 2001
    Location
    Shrewsbury, Shropshire, UK
    Posts
    47
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Exploratory vs. Planned testing metrics

    thought i would add a slightly more thoughtful reply!
    I do personally record what I refer to as "test type" in my bug tracking database, the possibilities I use are:
    Exploratory/Functional
    Formal/Functional
    White Box
    Inspection or Review
    Exploratory/UAT
    Formal UAT

    the idea is to compare defect statistics in a way that includes how they were found, incredibly useful for test process improvement and for evaluating the effectiveness of formal vs exploratory testing, especially where defect risk is also taken into account.

    Bearing in mind the type of person often considered adequate for execution of formal test scenarios (I tend to get given any service support personel available) it is possibly a good idea to classify those defects given as examples above as exploratory, as they would not have been found if the formal tests were simply followed robotically - you had to use your skills as a tester to find them.

    ------------------
    edit: p.s. is post count locked????? 46 twice doesnt add up to me..


    [This message has been edited by kiaora (edited 11-23-2001).]

  5. #5
    Points for Confirmed Friends
    Guest

    Re: Exploratory vs. Planned testing metrics

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by kiaora:
    it is possibly a good idea to classify those defects given as examples above as exploratory, as they would not have been found if the formal tests were simply followed robotically - you had to use your skills as a tester to find them.
    <HR></BLOCKQUOTE>

    kiaora - thx for your input. Good point!
    The reason I was worried about this was that I figured if I put "Exploratory," a pointy headed boss somewhere might look at these metrics and see that "Exploratory" bugs outnumbered "Planned" bugs so we should stop planning our testing. Your point shows me that our metrics might, for example, show that our planned test cases are not good candidates for automation yet as a human eye is needed. Maybe we need another source "Planned Exploratory testing"!

    Jules - I guess what I said above covers what you said - it depends on what they're useful for. Unfortunately, I do not really know what the metrics are going to be used for. However, do we necessarily need to know how metrics will be used before taking them? Don't the results dictate what they'll be useful for? Your thoughts on this would be appreciated.

    Thanks!


    ------------------


    [This message has been edited by gsh (edited 11-23-2001).]

  6. #6
    Senior Member
    Join Date
    Dec 2000
    Location
    Pune, Maharashtra, India
    Posts
    1,037
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Exploratory vs. Planned testing metrics

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by gsh:
    Here's the scenario.

    1) If I run a test case and it passes (ie the result matches the expected result) but I do find a defect, would the source be exploratory testing or planned testing?
    <HR></BLOCKQUOTE>

    I dont understand - If the expected met the actual results, how can there be a defect ?


    2) If I run a test case and it passes and then I try ONE more step and find a defect, would the source be exploratory testing or planned testing?


    My vote goes for exploratory testing. It isnt part of the steps of your test case and you went ahead anyway. Thats the clincher.

    ------------------
    - Suresh
    [Speed, quality, price - pick any two !]
    Suresh Nageswaran, CQA, CSTE
    Lead Consultant
    Kanbay Inc.
    Pune, India.
    sureshnageswaran@yahoo.com

  7. #7
    Senior Member
    Join Date
    Aug 2001
    Location
    Atlanta, GA
    Posts
    2,693
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Exploratory vs. Planned testing metrics

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by gsh:
    Jules - I guess what I said above covers what you said - it depends on what they're useful for. Unfortunately, I do not really know what the metrics are going to be used for. However, do we necessarily need to know how metrics will be used before taking them? Don't the results dictate what they'll be useful for? Your thoughts on this would be appreciated.
    <HR></BLOCKQUOTE>

    I'm not Jules, but will give you my opinion anyway.

    I don't necessarily agree with the theory that tracking things in the hopes that the metric will one day become useful providing anything of benefit to you and your team. In your particular instance, you are spending time to define these areas, and your team will spend time trying to define each of the defects they find when you're not even sure you're going to get something valuable out of it! It doesn't make sense, then, to track it at all, IMHO.

    That being said, the metric you've mentioned is not necessarily one that won't be used for something, but I'd take the time to analyze if the cost and effort of implementing it, determining the usefullness, and putting it to good use are worth the return. Keep in mind, also, that in most cases, metrics are designed so that you can determine a standard, or an 'acceptable range', and they are designed to be 'improved upon' in other cases - does the metric you wish to implement do any of those things? And again, analyze the cost versus benefit before implementing any tracking and metrics, to guard against tracking solely for the sake of tracking.

    ------------------
    "I want the fairy tale..."
    Annemarie Martin
    Secretary
    Association for Software Testing

  8. #8
    Points for Confirmed Friends
    Guest

    Re: Exploratory vs. Planned testing metrics

    punekar - you ask how there can be a defect if the test met the expected results - the expected results may occur but another result that was not expected also occurs - this result was not anticipated maybe due to lack of documentation or other factors (in complex systems, I don't think you can really write down every expected result as there may be thousands and you can only test one thing at a time).

    QAGirl - the reason I brought this question up in the first place (which I probably should have made clearer) is because I am not the person tracking this, just the person providing the info. My boss has provided me with pretty vague instructions when I asked him which I should use in these situations so I figured I'd see what people here thought. Maybe he does have a reason for measuring this or maybe it's coming from above him, I don't know.

    ------------------

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 11:41 PM.

Copyright BetaSoft Inc.