SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 11
  1. #1
    Senior Member
    Join Date
    Jun 2008
    Location
    The Land of Snake Charmers
    Posts
    212
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    ET and Late Cycle Defects

    Hi

    I have been involved lately in doing some exploratory testing. (more or less for projects in which little or no requirements are provided). I just play with the product for a while and then use common sense to find defects.
    On discussions with developers, I learn new functionalities with the project and then test them too.

    However, I have observed that I am able to find critical/high risk defects very late in the testing cycle?
    Though no one has frowned on me (in my knowledge) for this, however, I frown on myself.
    How would you suggest I would be able to better handle this?
    Is there a better way to perform ET?

  2. #2
    Senior Member
    Join Date
    Aug 2008
    Posts
    239
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    Start it earlier?

    Due to it's nature (stressing integration in new and interesting ways being a strength of ET) you can find some nasty deep-seated issues that may not be found with scripted tests. Also software becomes 'immune' to the same scripted tests run over and over with no variation.

    tbh I wouldn't beat myself up too much about finding critical defects late in the testing cycle. I'd worry if customers found them _after_ the testing cycle. That said - I do apply structure to ET, employing some session based techniques (this is vital as I need to be able to organise many hands testing - it may not be so important if you are alone/part of a small test team with visibility on what everyone is focussing on).

  3. #3
    Senior Member
    Join Date
    Jun 2008
    Location
    The Land of Snake Charmers
    Posts
    212
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    [ QUOTE ]
    That said - I do apply structure to ET, employing some session based techniques (

    [/ QUOTE ]

    I guess, I should understand what applying structure to ET would mean?
    Do we have any techniques for this?

    I thought ET was very "dependent" on the knowledge and experience of the individual tester?

  4. #4
    Senior Member
    Join Date
    Aug 2008
    Posts
    239
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    some reading here:

    http://www.satisfice.com/sbtm/

    ET is very dependent on the individual tester, but that doesn't preclude some structure. What I need to avoid in my ET is 10 well intentioned testers scratching the surface of the AUT in the same place.

    Bad metaphor here, but imagine blindfolding 10 people, pointing them all in the same direction, and saying "Go!". Now imagine taking the same 10 blindfold people, putting them in a circle, all facing outwards and saying "Go!". You're likely to get better coverage from the latter, although there will invariably be some overlap.

  5. #5
    Senior Member
    Join Date
    Jun 2003
    Location
    Tampa, FL
    Posts
    372
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    The results you get and success you have with ET do depend largely on knowledge and experience, although I would challenge anyone to say something different about automation. However, everyone starts somewhere, and whatever you have when you start, you can be successful to some degree. (This of course presumes you are NOT hiring complete idiots for testers)

    "Structure" and "ET" are not directly correlated in any way...which is to say, not negatively either. So, ET can be as structured or unstructured as you like; structure does not negate exploration, and exploration does not negate structure.

    My first recomendation is to read up on Session Based Test Management as a great start to bringing structure to your ET, you can find a good paper here...

    http://www.satisfice.com/articles/sbtm.pdf

    My favorite way to bring structure personally, especially when I am not yet familiar with the application, is to take a feature tour. I start with the first menu item, typically "File", and I click every menu, button, toolbar item, link, etc, from left to right and top to bottom, in as close to a linear order as possible. At this point, I am leaning heavily in the "Learning" direction of ET...I am building a model of the application in my head, what it does, what it does not do, implied orders of operation, levels of complexity, etc. You would be amazed how many defects you can turn up this way, but they will likely not be the deep nasty ones. However, armed with this new information and model, you can then begin to attack business process flows and functionality with a vengeance, and your ET will be much more focused and purpose driven; and all of this presumes a complete lack of any guidance or documentation at all. If you have any of that, so much the better.

    Finally, finding deep bugs late is, as has already been said, better than not finding them at all. Rather than say "ET caused me to find these late, and that is bad" say "Look what ET found that all our other processes missed...this is GOOD!"

    Good Luck.

  6. #6
    Senior Member
    Join Date
    Mar 2007
    Location
    Waterloo, Ontario, Canada
    Posts
    3,628
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    [ QUOTE ]
    However, I have observed that I am able to find critical/high risk defects very late in the testing cycle?


    [/ QUOTE ]

    I will actually confirm this with my own story. We ran a pilot project which integrated Exploratory Testing into our first, small, project a while back during some downtime that we were waiting on, of all things, artwork.

    We were finding very few new bugs on a daily basis with manual scripted testing and our team was made up, primarily, of co-ops with a senior overseeing them. We introduced them to the ET concept and how we would like to see it integrated into our testing process. In the next two weeks we found about the same number of bugs as we had with our last month worth of testing using manual scripted testing.

    It's something that is difficult to sell to management, but if you get a chance it will show you some great results. Manual, scripted tests are linear. Also, those tests become even more linear if we do something dumb like link them to our performance. You have your steps, and you have your expected results. You're never asked, though, what if I do this afterwards?

    If you've ever known a customer to go, File>>New, File>>Open, File>>Save As... then let me know. In my experience, though, they will usually do something in between those tests. 1 + 2 + 3 is simple math. It's only when someone thinks that the result should be 7 that you're in real trouble. You might never find that without ET though.

    I would suggest running both scripted tests and ET to please management, but in my opinion, well executed ET can do just about anything that scripted tests can and more. It's more just the burden of proof that scripted tests can sometimes help more with.
    Brent
    --------------------
    9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
    --------------------

  7. #7
    Senior Member
    Join Date
    Jun 2008
    Location
    The Land of Snake Charmers
    Posts
    212
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    This has been a good discussion. To add it further.

    1. http://www.satisfice.com/articles/sbtm.pdf
    >> This talks about test management. On how a test manager can improvise ET in the team.
    >> Any suggestions for individual contributors?

    2. One of the traits of a good tester is a find high risk defects early in the testing cycle!
    ET does have it advantages (finding defects late, as compared to not finding them at all).

    However, my specific query still remains unanswered - How best can be implement ET to find high risk defects early in the testing cycle.

    Can this be considered a limitation of ET? or do we have best practices?
    If ET is really that great! Than how different would it be monkey testing? or ad-hoc testing?
    As per my limited knowledge, they all mean testing the application without scripts.

  8. #8
    Senior Member
    Join Date
    Jan 2006
    Location
    Stockholm, Sweden
    Posts
    371
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    [ QUOTE ]
    However, my specific query still remains unanswered - How best can be implement ET to find high risk defects early in the testing cycle.


    [/ QUOTE ]

    Prioritize high risk areas in your ET testing (applies to scripted testing as well). I also try to do End-to-end testing as early as possible.

  9. #9
    Senior Member
    Join Date
    Jan 2005
    Location
    UK
    Posts
    876
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    [ QUOTE ]

    However, my specific query still remains unanswered - How best can be implement ET to find high risk defects early in the testing cycle.



    [/ QUOTE ]

    I'm not trying to be a smarta** but isnt the answer to test the high-risk areas as soon as you can ?
    whether it's ET or scripts, do you know the high-risk areas of the program ?
    what am I missing from your query ?

    [img]/images/graemlins/confused.gif[/img]

  10. #10
    Senior Member
    Join Date
    Aug 2008
    Posts
    239
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: ET and Late Cycle Defects

    [ QUOTE ]
    If ET is really that great! Than how different would it be monkey testing? or ad-hoc testing?
    As per my limited knowledge, they all mean testing the application without scripts.

    [/ QUOTE ]

    Hmmm.... you start the thread saying you perform exploratory testing, then suggest that you do not understand the definition.

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 8.82%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 12:36 PM.

Copyright BetaSoft Inc.