User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 3 of 3
  1. #1
    Junior Member
    Join Date
    May 2001
    OKC, OK, USA
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)
    Total Downloaded

    Automation Tool Criteria

    Hi all!

    I'm evaluating tools for a new company. I decided to go with a standard school grading system so the stakeholders can easily understand my report. I've written the portions that matter most to my projects, and the projects we know about in the future. Some of them are blue sky type criteria that I don't think have been implemented in a tool currently available. I thought this would be a good point of discussion in regards to what you'd add for your company's projects. Some point of knowledge for evaluating testing tools would be extremely helpful to new automation groups. We should begin with a description of the environment we will utilize the tool to test to give a little background on our thoughts.

    Anyway, my projects are all web delivered apps with 95% of the Java functionality running on the backend. It's supposed to be displayed utilizing IE 4.01+, NS 4.7+, and Opera 5.0. We're going for W3C HTML 4.01 Transitional compliancy.

    Automation Tool Criteria

    Ease of Use
    A. Tool has an intuitive interface.
    B. Tool has an average interface, keep the manual handy.
    C. Read manual before installation, after that you’re okay.
    D. Tool was designed by a very sadistic people
    F. . . . and so was the manual.

    Support of Browsers
    A. Tool fully supports IE, Netscape, and Opera.
    B. Tool fully supports IE and Netscape.
    C. Tool supports only old releases of IE and Netscape.
    D. Tool supports only IE or Netscape.
    F. Tool doesn’t seem aware of what a browser is.

    HTML Support
    A. Tool has settings to allow tag validation on a custom level, from fully W3C HTML 2.0 to HTML 4.01 Transitional compliancy to anything goes.
    B. Tool has a fixed method for dealing with tag validation and both tells you what it is and does it accurately.
    C. Tool does no tag validation, but works with page elements correctly.
    D. Tool does tag validation, but doesn’t tell you what document type compliancy it’s going after.
    F. Tool does tag validation, doesn’t tell you what compliancy it’s looking for (or has a vendor defined non-standard compliancy) and is inconsistent in its reports.

    Java Support
    A. Tool not only is not confused by Java, it points out coding mistakes.
    B. Tool is not confused by Java.
    C. Tool is sometimes confused by Java.
    D. Tool is not aware of what Java is.
    F. Tool reports any Java/Javascript as an error.

    Mock Test One: Navigation
    (This test checks the tools ability to handle different navigation methods. Straight hypertext linking, drop down nav, dynamic link nav (those that change depending on the hour of the day, for example), java toolbar nav, back & forward & refresh buttons on the browser, images as links, multiple page navs, across multiple domains and page based redirection)
    A. Tool is never confused. The tool recognizes each type of navigation and categorizes it appropriately.
    B. Tool sometimes misinterprets a navigational item but always recovers and performs the expected action.
    C. Tool lacks support for one of the not-commonly used navigation methods.
    D. Tool lacks support for any navigation method that doesn’t involved an A HREF tag
    F. Tool was designed by a monk in the mountains of Tibet who had no clue what the Internet is.

    Mock Test Two: Forms
    (This test covers the tools ability to deal with different form elements and implementations of those elements. This includes drop down menus, list boxes, radio buttons, checkboxes, textboxes, password boxes, text areas, file browse text boxes, custom buttons, submit buttons and reset buttons.)
    A. Tool deals with all forms of forms without a problem. Tool correctly categorizes each.
    B. Tool sometimes deals with form items incorrectly, such as labeling a file browse textbox as a normal textbox, but the mislabeling doesn’t hurt the tools functionality.
    C. Tool can’t deal with one of the form items, but there is a workaround.
    D. Tool can’t deal with most form items, workarounds are tedious and time consuming.
    F. Tool is unaware forms exist. Tool is actively hateful towards any user that attempts to deal with them.

    Conditional Support
    A. Tool can perform if…then…else’s, do while’s, for each’s, with no problem.
    B. Tool has a more limited set of conditional support, but what it does have works beautifully.
    C. Tool has one conditional that usually works.
    D. Tool has no straight forward conditional support, but you can see a way to hack around it.
    F. Tool doesn’t appear to have a language at all you can access.

    Dynamic Data Sets
    A. Tool supports multiple data sets and can basically turn anything hardcoded into a variable to be looked up in an easy manner.
    B. Tool supports a single data set but can turn anything hardcoded into a variable.
    C. Tool supports data sets but has limited support for variables.
    D. Tool supports data sets somewhat and variable insertion is somewhat difficult.
    F. Everything is hardcoded. Tool will make anyone from a development background leave the country and live in an Eskimo village.

    Error Recovery
    A. Tool can deal with any network, page, or script anomaly with the grace of a dancer.
    B. Tool can deal with most errors without aborting.
    C. Error recovery must be dealt with heavily by the automation person, but it can be done.
    D. The tool aborts the script almost any chance it gets.
    F. The tool has the grace of a dancer that has been drinking tequila heavily for several hours.
    QA Automation Outsourcing. The Lab @ www.11ag.com

  2. #2
    Points for Confirmed Friends

    Re: Automation Tool Criteria

    For learning curve size, you might include an evaluation of its scripting language in terms of familiarity to other programming languages:

    A: It's virtually identical to {C | Perl | Visual Basic | other common programming language}.
    B: It's very similar to some common programming language.
    C: If you know some common programming language, you'll pick it up fairly quickly with many referrals to the manual.
    D: It has a unique syntax, but an experienced programmer should get used to it eventually.
    F: You'd be better off writing your own assembly code test programs than using this tool's scripting language.

    Charles Reace

    Software Testing (n): 1. The art of trying to increase your confidence in a piece of software by finding everything that is wrong with it.

  3. #3
    Junior Member
    Join Date
    Jan 2001
    Las Vegas
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)
    Total Downloaded

    Re: Automation Tool Criteria

    Browsers: Does the tool keep up with current releases? For example, Compuware's QARun wouldn't work correctly with IE5.5 until recently.




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.11%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 05:39 PM.

Copyright BetaSoft Inc.