User Tag List

Results 1 to 3 of 3
  1. #1
    Junior Member
    Join Date
    Nov 2001
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    evaluation comments of QuickTest 5.5

    These are my Astra QuickTest (AQT) evaluation comments.
    I evaluated the downloadable AQT [not the Professional
    variant], version 5.5, during the first week of
    November, 2001. I spent 25-30 hours in this evaluation,
    so I would not consider it extensive. I would have
    spent more time, but I really was not impressed with
    this products features, as well as its pricing relative
    to its limited capabilties.


    AQT is a very good product for rapidly producing web
    application testcases developed through the Microsoft
    IE browser, if/when you only need the testcases for the
    duration of a single software release, and you are
    not attempting to do team development of those tests.

    Rapid development is achieved through the traditional
    capture/playback technique. The resulting testcase
    is nicely presented in an iconic Tree View [by
    default]. The "initial" recording is generally easy
    to read and understand, as long as AQT selected a
    sensible Logical Name for each Page and Object
    interaction, which are recorded as Steps in the
    Tree View. But as issue #9 below indicates things
    get pretty messy in the Tree View when you attempt
    to add more recording steps after the initial
    recording has been completed.

    While several versions of the Netscape browser are
    also supported in addition to IE browsers [version
    5.0 and above] it is not clear that you can record under
    one browser and execute under another [and there are
    very few programming things which you can do to
    accomodate cross-browser differences at runtime].

    The underlying scripting language [which you can easily
    use by switching to the Expert View] is vbScript.
    This is much more powerful than WinRunner's TSL
    [roughly equivalent in power to Segue's 4Test]. But it
    is difficult to exploit this scripting power due to
    the AQT vbScript runtime environment [discussed in
    greater detail below], as well as the difficulties of
    dealing with objects that are [or are not!] in the
    Object Repository.

    Tests are constructed in a hierarchy of Actions.
    Actions include Steps. Steps can and will be nested
    as the user navigates the application under test
    during recording. The developer decides if and how
    to use multiple Actions. Actions from one test can
    be called/reused in another test[parameter passing
    between Actions is generally done through the
    Global DataSheet--which is very primative]. The
    contents of an Action live in their own [vbScript]
    namespace, so functions defined in one Action are
    not reusable in another test's Action [code
    reuse is discussed in greater detail below in
    issue #8].

    The Object Repository is roughly equivalent to WinRunner's
    GUI Map, but much less flexible. You can't manually
    add Pages/Objects to the Repository. And more importantly
    you can NOT change the Logical Name of an object [which
    is assigned by AQT when the object is learned during
    a recording session].


    Some specific issues, along with examples, are listed
    [in no particular order] below:

    1. Tests which include Actions from another test, reference that test's file structure with an embedded fully qualified pathname reference. Therefore multiple developers, each on their own PCs, must work in the same drive letter/dir structure in order to share actions. [Also refer to item #6 on issue related to source code control].

    2. You can't define/modify the logical name of an object/step's property. The one selected by AQT is the one you have to live with. For example when recording against the Search Flight Results page on the Mercury demo web application clicking on a destination flights radio button recorded a "buttonX" logical name [where X is 1 - the max number of buttons in the list of flights], rather than the name of selected flight.

    3. Error messages are not very informative. A example is the fact that if the Page property is incorrect [which can intentionally be messed up for eval purposes using the Object Property dialog] AQT consistently failed in the first object reference on that page [for example pushbutton click] indicating that the page's object failed [when in fact the Page could not be found].

    4. The manuals fail to contain information about many things you might want to dig into. For example you can't find any description of the "miclass" Property, and therefore you can't figure out what are legitimate Value settings. [This is especially puzzling when the tool records references to all Pages as Property: miclass, Value = Page; it just seems to be saying that any loaded page will suffice for object recognition purposes].

    In some situations you are left to ponder how you access data with some of the AQT objects, using their object model documentation. For example it is not clear [at least to me] from the documentation that you read a setting object's value with a default reference of setting("settingName"), while the documenatation provides descriptions on how to create an new value and delete an existing one.

    5. If a series of Actions are created in a testcase, you can only have one expanded at a time. When the testcase is played back you can't see which steps are being executed as each action is played back [you only see the execution arrow pointing to the action]. This is very frustrating when waiting with timeout for something to fail.

    This is even true when you have expanded an Action for visual display. If you then select the Test:Run Action From Step... command AQT colapses the Action and then starts execution.

    6. Tests created by AQT create a signficant number of files and directories "behind" the scenes. WinRunner also does this but AQT creates many more. For example I create a very simple test with only 3 user interactions which simply logs into the Mercury demo website. The following dir structure and files were created under the tests name in the ..\Tests\loginTest dir:
    This list grows rapidly as more pages and objects are recorded and new Actions created. This will make it very difficult to put tests under source code control.

    7. A test's source code can only be viewed using the the AQT editor [i.e. the *.tsp files are stored in a proprietary binary format]. This may make it difficult for those who need to document tests. You should not need a licensed copy of AQT just to view code.

    8. The AQT hosting environment [sometimes referred to as a vbScript "host platform" like Microsoft ASP and WSH] does not include an "include file" feature. This means you will have to perform non-intuitive workarounds to hack together reusable code libraries.

    9. Maintenance of scripts: attempting to record additional Steps within a previously recorded Action leads to puzzling results. Many additional and unnecessary pages are placed in the Object Repository as well as several Steps in the Tree View.

    For example record a script which simply logs into the Mercury demo web application that brings up the app, enters the login ID and password, and then clicks the LOGIN button. You initially have two pages recorded, the 1st Page ["Mercury Tours"] has 3 objects while the 2nd page ["Welcome to Mercury"] has just the Sync() method recorded. If you then highlight either the 2nd Page or the Sync() Steps and re-record a single SIGNOFF button click on the 2nd page, AQT [consistently] records the following Steps/objects:

    a) another Browser Step [this makes no sense]

    b) under which is a "Welcome to Mercury_2" Page [whichis identical to the original 2nd Page]

    c) under which is the "Signoff Button" Click() step.

    d) also under the new Browser Step is a new "Mercury Tours_2" page [which is identical to the original 1st Page] and it contains a Sync step.

    Therefore by adding 1 additional button click Step [i.e. clicking on the 2nd Page's SIGNOFF button] resulted in 7 new items being added to the Object Repository!
    The Tree View is also cluttered with these additional Steps/objects. And the test will not run if you just move the button click Step back into the original 2nd Page [where it belongs!] and delete all the other un-needed Steps--I think this might be feasible if you could directly edit the object definition into the Object Repository, but that is not permitted [you can only fiddle with existing object Property values using the Object Repository dialog].

    The bottom line: design a testcase with lots of small Actions. Then if you need to re-record a portion of the test delete all of the Actions steps and re-record all of those interactions/Steps again. [From the NEWs groups it appears this is what Mercury recommends].

    10. Pricing is absurd: as of 11/2/01 AQT is priced at $4400 a seat while AQT Professional is $6600 [which is the cost of WinRunner with WebTest]! I can't imagine why any customer would pay the same price for AQT Professional, rather than buying WinRunner/WebTest.

    -The Dog

    [This message has been edited by qadog (edited 11-05-2001).]

  2. #2
    SQA Knight
    Join Date
    Jul 1999
    Grand Rapids, MI
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    Re: evaluation comments of QuickTest 5.5

    For another [more positive] slant on this tool checkout David Gilbert's impressions, based on 3 years experience, in the following post on the WinRunner board:


  3. #3
    Junior Member
    Join Date
    Oct 2001
    Melrose, MA, USA
    Post Thanks / Like
    0 Post(s)
    0 Thread(s)

    Re: evaluation comments of QuickTest 5.5

    Included are supported by VBScript but it is tricky because there is no function include.

    The include source code work well while debugged!

    Try a search engine i am sure you will find how to do it...




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 08:59 AM.

Copyright BetaSoft Inc.