Page 1 of 3 123 LastLast
Results 1 to 10 of 24
  1. #1

    Why do you automate?

    Chapter 2 of our upcoming book Implementing Automated Software Testing is titled "Why Automate?" and it consists of 20+ pages explaining why many tests should be automated (that includes the need to evaluate a test to see whether it makes sense to automate it, etc.).
    What is your reason for automated software testing? Maybe we can start a "Why Automate" collection here.


  2. #2

    Re: Why do you automate?

    In order to liberate time for high quality manual testing...by reducing time consuming tasks which can helped by...

    Automated regression test suites
    Tailor made test tools (file creation, API testing, random test data etc)
    Test environment daily maintaince (survailance, automated batch jobs, test data handling etc.)

  3. #3

    Re: Why do you automate?

    Hi Elfriede,

    I automate out of a requirement to maintain a fixed cost ratio between testing and development for a mature product with regular new releases. This is based on the assumption that regression testing is a necessity, i.e. new additions to the system can break existing functionality, and I originally gave an example here.

    Manual regression testing does not scale with development over a long product life cycle. I have not found a reasonable alternative to automation in this role, specifically for end to end scenario based regression testing. Unit testing is excellent from a code stability perspective, but IMO does not replace this. The issue arises where you have a flexible COTS program that supports a broad range on end user work practices. A well-written 'improvement' to the product might provide a perceived enhancement to one group of users, while breaking the workflow of another group. Having test scenarios covering these various workflows allows us to detect and resolve these potential issues.

    While this might not be important to a large company like MS who seem willing to change UI paradigms at the drop of a hat, it is critical to a small company such as my own where the maintenance revenue stream accounts for a significant part of our turnover.

    Hope this all makes sense,


  4. #4

    Re: Why do you automate?

    We have a lot of test cases for the filter driver I test, and we run them many, many times over the development cycle before we get to the release cycle. Automation helps us to run the tests, the same way over and over, while maintaining a log of results - this way we have a small variance in how we conduct the tests and the results are saved so we have history.

    Because filter drivers tend to be fragile in some respects this helps us maintain a good Regression Suite, but also add in new cases as new functionality is delivered. It saves us time during the test cycle and while a test is running I can be doing something else. I think that is the only thing that has not been covered already.
    - M

    Nothing learns better than experience.

    "So as I struggle with this issue I am confronted with the reality that noting is perfect."
    - Unknown

    Now wasting blog space at QAForums Blogs - The Lookout

  5. #5

    Re: Why do you automate?

    This thread Philosophical discussion is related.
    Am trying to collect various inputs and then will add to the FAQ.

  6. #6

    Re: Why do you automate?

    I do performance testing, which is usually impossible to do manually. Different realms of testing automate for different reasons.
    Corey Goldberg
    Homepage: goldb.org
    Twitter: twitter.com/cgoldberg
    Google+: gplus.to/cgoldberg

  7. #7

    Re: Why do you automate?

    We automate because it gives us the ability to run our regression tests in days instead of weeks. It also helps to free up time for us to do manual testing on the actual changes/enhancements that were made while the automation verifies that all of the existing functionality is still working. This allows us to respond to customer needs quicker because we have shortened the test cycle without losing any quality.

  8. #8

    Re: Why do you automate?

    [ QUOTE ]
    I do performance testing, which is usually impossible to do manually. Different realms of testing automate for different reasons.

    [/ QUOTE ]

    Good point, Corey. I also use automation tools for certain types of performance testing and bug hunting. On the performance side, I use automation in conjunction with a profiler to repeat a set of steps while tweaking the app, to check the effect on performance. Similarly, if I have a hard to reproduce bug, i.e. lots of steps and the bug doesn't always show itself, I'll create a one-shot record/playback script to be able to more easily reproduce and analyse it. Thinking about it, these are a couple of scenarios where R/P is very useful, i.e where the expected life of the script is less than a day.

    I also monitor execution times on regression tests, to see how the application performance is impacted by internal changes, and external environment changes such as new O/S, virus scanners, disk defragging, etc... FWIW, my regression tests run about 20%-30% faster with the virus scanner turned off. Disk defragging doesn't seem to have much effect, even though the AUT has a lot of disk activity. Reducing the number of files in indexed folders, e.g. old base lines, does improve things quite a bit.

  9. #9

    Re: Why do you automate?

    Because manual regression testing is by nature a repetitive chore that can quickly grow out of hand. With each subsequent release the regression suite grows larger and, unless your company is prepared to allow testing time to grow indefinitely with it, you will end up having to prioritise and run only portions of your test suite.

    Also, the old adage "to err is human" applies here - a tester who has to run through the same set of tests on a regular basis is likely to start making assumptions or cutting corners after a while.

    If you have a set of tests you are going to run regularly, it's worthwhile investing the time upfront automating them. If they're only going to be run once, then it's probably not worth the pain. My rule of thumb in the past has been that it takes about five times longer to write an automated script, so I ask myself whether I am likely to run the test more than five times.

  10. #10

    Re: Why do you automate?

    I have yet another point of view. Some time ago I was leading a team responsible for localization testing of a number of products on Spanish, German and French locales. Initially we were trying to hire bilingual testers to do the work. It was however too expensive and engineers with both foreign language and software skills were difficult to find. Hiring was a nightmare and we could not fill in some of the positions for about half a year.

    Facing such problems we changed our strategy and decided to automate. The test suites had to be robust enough to handle GUI changes caused by localization. We identified two suitable test tools/frameworks and we automated part of the testing. Test reports were designed to include screenshots with localized UIs and console/web output. Instead of having bilingual engineers we employed linguists (students) to review the results. The feedback was provided to engineers who fixed the issues with the help of linguists. This model was so successful that we ended up running the automated tests for 4 Asian locales which were originally tested by teams in China and Japan.

    The point here is that automation helped us to effectively separate areas of expertise. We didn't have to look for bilingual engineers any more. We were suddenly OK with experienced QA engineers who just designed robust automated tests on English locale, ran them on other locales and submitted the results to language experts. Such approach can be effectively applied anywhere you need a special expertise to evaluate functionality of your product, whether it is science, medical systems or any other specialized product.
    Robert Pes
    T-Plan Robot, open source cross-platform automated testing tool based on remote desktop technologies


Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
BetaSoft Inc.
All times are GMT -8. The time now is 06:04 PM.

Copyright BetaSoft Inc.