SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 3 of 3
  1. #1
    Junior Member
    Join Date
    May 2000
    Location
    Auckland, New Zealand
    Posts
    27
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    How to keep automated testing when the target app is always changing

    I used to work in a large corporation, our department used 12 developers who were constantly (daily) checking in code. I implemented and maintained a test automation process where the test tool (qarun by Compuware) was used to make the daily 'integration build' of the software and then run a series of automated tests over it.
    This worked well in theory but the trouble is that due to the amount of development going on things (eg GUI) were constantly changing. I found a huge amount of my time was taken up chasing 'failed' tests, only to discover that the failure was the result of new functionality. Getting appropriate communication (information) from development and management about the changes was difficult.
    I have been trying to keep the tests running day by day, which means constantly updating sources of checks and altering scripts. I think I have been blowing too much hot air for not enough result.
    When we 'freeze' our code every 2 months I also freeze the test scripts and data. Other departments find this very useful for smoke testing. But I wonder if my 'grandularity' of updating the tests has been far too fine?

    I am now beginning a new job setting up an automation lab and wish to avoid the same thing happening. Suggestions?

  2. #2
    Member
    Join Date
    May 2000
    Location
    Nottingham, UK
    Posts
    83
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: How to keep automated testing when the target app is always changing

    IMO I would set up a Software release process from development and make sure that management have a "buy in" to this. I would also set the expectations of the testing lab to both these groups, i.e. You can't have daily changes to functionality and expect a quality product. You should then be able to tailor your existing test processes(?) accordingly.

    Hope this helps, good luck.

  3. #3
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: How to keep automated testing when the target app is always changing

    Having these types of automated tests to run after each build is the best way to go. However, as with most good things, there is a price - you must be willing to keep up with the UI changes of your application.

    I strongly suggest that you try to budget time for this effort, as I believe it yields the highest quality results.

    However, if you do not feel you can afford the time each day to run and "fix-up" the test suite, there are a few things you can do to mitigate the time investment.

    1) Try to write scripts which avoid the User Interface. Often, you can test a lot of the underlying functionality without hitting very much of the user interface. Sometimes, a specialized testing harness UI can be built to test the underlying layers of your application. I used this method a few times to test a "language" that was embedded in my application-under-test. The language was one of the more important aspects of the profuct, but it could be tested without any real UI. We wrote a driver for the language portion, and I wrote my test cases against that. No matter how broken the app's UI was, I could still test the language layer.

    2) Even though you build daily, you don't always need to run regressions daily. Perhaps weekly would suffice?

    3) Depending on the nature of the UI changes being made, you may need to modify the way you identifiy UI elements in your application. If a minor UI change is breaking your tests, the test may not be using the optimal object recognition method. Most test tools have some flexibility in this regard. For example, if you recognize a pushbutton by its text, and the developers change the text often, you could consider recognizing it by something that doesn't change as frequently (by its control ID, for example).

    4) Often, "cleaning up scripts" can be delegated to a more junior QA Engineer, or perhaps a Build Engineer. It's often easier to make these types of minor changes to test scripts than it is to create them. If they are well-documented, that can help.

    5) You could divide the scripts into two categories - those that will be run after each build, and those that will be only run periodically. The post-build scripts should be made much lighter - more of a smoke test than a full regression test. That way, you'll have fewer scripts to clean up each time. You should also attempt to isolate those portions of your scripts which must be changed often - perhaps into separate functions. These function should be well-documented, and made as easy to change as possible.

    Best of luck,
    -joe
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.54%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 11:49 AM.

Copyright BetaSoft Inc.