SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 7 of 7
  1. #1
    Junior Member
    Join Date
    May 2006
    Location
    London
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Fitnesse AND/OR STAF/STAX

    I am working with a system developed in Delphi, which is in a client-server setup. We are looking at doing automation testing on this system. Aside from going to Rational/Mercury I would like to investigate other methods. We are quite fortunate that the system has been developed in a macro style fashion. In the sense that there is a console for the application that records current actions triggered by using the GUI and where you can execute these actions/macros and see the resulting action in the GUI. The console can also be accessed via telnet, so you can essentially control the application entirely via issuing commands over telnet to it (the client pc).

    This poses two questions:
    a)Is this a sufficient level of interaction for functional testing? I.E. does controlling the application in this way equal clicking on the actual button like the user would?
    b)What framework can we use to accomplish this?

    Our primary testing goals are to be able to create a test system that will:
    1)automatically pick up new builds and install them on both client and server machines
    2)execute tests
    3)report back the results.

    Due to some existing testing code being written in python, I have investigated both fitnesse (pyfit) and STAF/STAX (here on in referred to as STAF).

    Fitnesse
    Fitnesse has the advantage that it is easier to design new test cases and get an overall picture of what is being tested in a nice web/wiki interface. Yet it does not (to my understanding) allow remote execution. Which proses a problem with testing goal 1), to pick up new builds and install them on the machines. Testing goals 2) and 3) can be accomplished by sending commands via telnet to the applications and reporting back the results in fitnesse. Are there ways to install applications remotely over a simple lan connection?

    STAF
    STAF does allow distributed execution, meaning that from the one STAF control machine, tests can be executed that will be sent to either the client or server machines and executed locally on the machine and then return the results back to the control machine for reporting. From what I have seen of STAF, writing in xml/python can be a bit messy, and although you can abstract most of the python code into libraries you still have to deal with the xml side of things. Also, I wonder if you would be able to obtain a clear overview of the test design/plan or would you need to use a test design tool to record your test cases so that you had a clear view of what your tests covered? (or maybe running a documenter, like phpdoc for php, on the python code will give a report of the test cases)
    It does however offer a great little control and reporting application built in java.
    http://agiletesting.blogspot.com/200...-tutorial.html gives a nice overview of something similar. Yet we would want to use it for much more than just a smoke test.


    I have even contemplated using BOTH Fitnesse AND STAF together.
    •Either use Fitnesse to control the execution of scripts and storage of test data while leveraging STAF to be able to execute code remotely (by sending commands from fitnesse to STAF), or
    •To use STAF to control the execution of scripts, using Fitnesse simply as the data store for test data, leveraging any built in functions to retrieve the data from its wiki template and execute them via STAF.


    What I am looking for is some input/opinions on any of the approaches defined above that may assist me in making a more informed decision based on the tools listed above (or any other tool that may be better suited).

    Thanks in advace
    Alex

  2. #2
    Super Member
    Join Date
    Dec 1999
    Location
    Washington, DC
    Posts
    1,273
    Post Thanks / Like
    Blog Entries
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    Hi Alex,

    We have been using STAF/STAX successfully for over a year now and I highly recommend it. With STAX you can automate and monitor your entire test environment; with many features available out of the box, such as
    <ul type="square">
    System/product setup
    Testcase distribution
    Testcase execution
    Testcase results collection and analysis
    and more[/list]
    Elfriede

  3. #3
    Junior Member
    Join Date
    May 2006
    Location
    London
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    Hi Elfriede,
    How do you find editing the STAX scripts in xml?
    I have read in some places that it can be a bit fiddly as there is no set plugin for eclipse yet etc?

    Also, do you have a test plan document or is there some way to extract what test cases you have from the STAX scripts to get a list of test cases etc?

  4. #4
    Senior Member
    Join Date
    Jun 2007
    Location
    Raleigh NC
    Posts
    509
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    "Are there ways to install applications remotely over a simple lan connection?"

    I bet you could do it with WSH/powershell. Depends on what you mean by "simple", but I think either of these would work.

    Sounds like a good idea to use both tools in combination, although if you're using STAF just for remote execution you might be better off just rolling your own on that. I created a lightweight remote exec/performance framework in a couple weeks on my own. It was in a unix based system and I called junit tests on the remote boxes.
    Cheers,
    B^2

    "Imagination is more valuable than knowledge." (AE)

  5. #5
    Super Member
    Join Date
    Dec 1999
    Location
    Washington, DC
    Posts
    1,273
    Post Thanks / Like
    Blog Entries
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    [ QUOTE ]

    How do you find editing the STAX scripts in xml?

    [/ QUOTE ]
    The test scripts are written in xml and that works fine, no issues

    [ QUOTE ]

    I have read in some places that it can be a bit fiddly as there is no set plugin for eclipse yet etc?

    [/ QUOTE ]

    STAF/STAX is an Eclipse independent framework. What are you trying to accomplish? Regarding the use of xml within Eclipse we have found an xml tool that can be integrated (let me know if you are looking for an Eclipse/xml integration)

    [ QUOTE ]

    Also, do you have a test plan document or is there some way to extract what test cases you have from the STAX scripts to get a list of test cases etc?

    [/ QUOTE ]

    The test cases are listed within a directory and can be pulled up within the STAF client. If you want to create test case reports, that can be done within STAF/STAX also.

    -- Elfriede

  6. #6
    Junior Member
    Join Date
    May 2006
    Location
    London
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    [ QUOTE ]
    I bet you could do it with WSH/powershell. Depends on what you mean by "simple", but I think either of these would work.

    [/ QUOTE ]

    Yes I have seen these, and the possibilities are there to accomplish this. I think we even have some python scripts written somewhere that can do this.. which is why I was thinking of using STAF for the remote execution/installation of the test software. Saves me having to re-write it all.

    im not too keen personally on creating a new one.. I guess i am looking for the simplest way of doing this. so maybe, initially either STAF and existing python scripts or the powershell/WSH will do.

    thanks

  7. #7
    Junior Member
    Join Date
    May 2006
    Location
    London
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Fitnesse AND/OR STAF/STAX

    Thanks Elfriede,

    With regards to the Eclipse plugin, yes, I was referring to an eclipse/xml integration for the editing of the scripts.

    I guess one of the things that I really like about the Fitnesse is the visual representation of test cases available in the fitnesse wiki. Which can also be used as a sort of requirements store. I am working in a scrum environment and having the developers able to add pages/requirements as they develop code allows me to be able to keep track of areas to test, and what to test. So I guess that is the other thing i am looking into, an effective way to be able to map requirements through to the automated tests.
    -Alex

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.54%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:21 AM.

Copyright BetaSoft Inc.