SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 6 of 6
  1. #1
    Junior Member
    Join Date
    Jun 2004
    Location
    US
    Posts
    3
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Validation of Automated Testing Tool

    The question I like to post for you out there is, should and does an automated testing tool be validated? I've got mixed apinions, What's yours?

  2. #2
    Moderator
    Join Date
    Mar 2004
    Location
    West Coast of the East Coast!
    Posts
    7,756
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Validation of Automated Testing Tool

    If the tool was created for or by your company (home grown) then yes it should be validated. But if it is a shrink-wrapped tool from a reputable company such as Mercury, then validating it might be a waste of effort and resources as well as down right hard to do without the original requirements. if you use the tool as it was designed, and you run into an issue based on the requirements for the AUT then it is either a real bug in your app, a bug in the tool, or an error introduced by the test script creator.

    Rich
    Personal Comment

    Success is the ability to go from one failure to another with no loss of enthusiasm.
    ~ Winston Churchill ~


    ...Rich Wagner

  3. #3
    Junior Member
    Join Date
    Jun 2004
    Location
    US
    Posts
    3
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Validation of Automated Testing Tool

    Yes, I understand what you mean. However, since we operate in a regulated environment, isn't it counter productive if we are to use as you put it "a shrink-wrapped" tool to perform testing of a regulated system; how do we know for sure the tool will do what it is intended to do? What assurance do we have that it will perform the testing and playback for any test scripts? Your thoughts...

  4. #4
    Moderator
    Join Date
    Mar 2004
    Location
    West Coast of the East Coast!
    Posts
    7,756
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Validation of Automated Testing Tool

    In your case I would say what you mean is to evaluate the tool not validate it. Of course there is a fine line there. But I assume you don't mean to look for bugs in the tool, right?

    In that case, by all means evaluate it, and any others which might be able to handle your needs. But I would advise you to list the tools that can do what you are lookig for and only evaluate the top 3, as evaluation can be a lengthy process and resource intense. To set the requirements for your new tool it might be best to have a meeting with all involved and prioritize the requirements for the tool.

    Rich
    Personal Comment

    Success is the ability to go from one failure to another with no loss of enthusiasm.
    ~ Winston Churchill ~


    ...Rich Wagner

  5. #5
    Senior Member
    Join Date
    Sep 2002
    Location
    Chanhassen, MN, usa
    Posts
    188
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Validation of Automated Testing Tool

    You can perform some of the validation by verifying that the expected results in both pass and fail conditions are the same on limited validation runs by comparing the test tool to manual testing.

    The next step is then to show by analysis that the test tool cannot obtain false positive passes, this is most often achieved by specifying a particular logical sequence that you follow in the verification steps to ensure that if you are having the test tool look for something, it cannot obtain false data that results in a pass. i.e. if you are searching for the text string "hello world" on the application, you have to show that the failure conditions that can occur in the test tool do not result in any sort of reference that would have invalid data. Some of this analysis has to be performed against your test system architeture.

    the final step then can be to perform code reviews on the test scripts to verify that the correct test logic is used to ensure that the validation logic from above is correctly implemented.

  6. #6
    Junior Member
    Join Date
    Jun 2004
    Location
    US
    Posts
    3
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Validation of Automated Testing Tool

    Thanks to both Rich W. and Mkrause. I will take these suggestions under advise when I am meeting with the team. Both suggestions seem reasonable and logical.

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 12:05 AM.

Copyright BetaSoft Inc.