Automated Testing and Cost Savings--Article
I'm a reporter working on a story for CNET's TechRepublic.com Web site. I'm looking for a couple of people who have used any automated testing tools and can talk briefly about the project they use/used it on and how much money they save by using the automated testing tools. Please reply before May 23, 2002 here on the forums or to firstname.lastname@example.org.
CNET's TechRepublic.com Contributor
(You can read my author bio and view some of my previous articles at the following link, but you must be a member of TechRepublic.com--which is free--to be able to see this page.) http://www.techrepublic.com/authorbi...l?authorId=dcs
Re: Automated Testing and Cost Savings--Article
While there are numerous white papers describing the cost savings of using automated testing tools, the ROI of using automated software testing tools for testing efforts is difficult to quantify.
This company here www.imbus.de (see their PIE 24306 "GUI-Test") conducted a one year research study using Mercury’s Winrunner. You can see their research findings on their website.
However, there are many caveats and numerous issues to be considered and some of the many pitfalls and drawbacks (and benefits) of using automated testing tools are described in my book “Automated Software Testing,” (Addison Wesley, 1999) and also in this article “Lessons in Software Test Automation” at http://www.stickyminds.com/sitewide....nction=edetail
Automated Software testing is not a silver bullet, and if implemented correctly, it requires software development.
Automated testing tools are merely a part of the solution – they aren’t a magic answer to the testing problem. The term automated test tool seems to bring with it a great deal of wishful thinking that is not closely aligned with reality. An automated test tool will not replace the human factor necessary for testing a product. The proficiencies and analytical thinking of test engineers and other quality assurance experts will still be needed to keep the testing machinery running. A test tool can be viewed as an additional tool from the tool box used in support of the release of a good product. Analytical skills are required to make the testing process most efficient, no automated tool can replace this work, and in most cases, additional staff and skills will be required to maintain the automated test suite. Automated testing has to be seen as an enhancement to the manual testing effort, not as a replacement.
In many cases, the testing team might be using a capture/playback tool with which they have automated 30% of the testing, only to realize that adequate testing coverage cannot be completely achieved using this tool. In situations like this, the team will need to consider building an automated testing tool that enhances the capture/playback tool’s reach. Other times, it may be discovered that the application doesn’t lend itself to automation using a capture/playback tool, either because there is no compatible tool on the market or because the task on hand is too complex for the capture/playback tool to handle, thus requiring a custom solution.
My book “Effective Software Testing,” (Addison Wesley, Winter 2002) describes additional drawbacks and tool issues in addition to what’s involved when having to “consider to build a tool instead of buying one.”
Author (with Rashka, Paul)of book "Automated Software Testing", July ‘99
Author (with Rashka, McDiarmid) of book "Quality Web Systems: Performance, Security & Usability", August ‘01
Author of book "Effective Software Testing: 50 Specific Ways to Improve Software Testing" (Addison Wesley, Fall/Winter 2002)
Re: Automated Testing and Cost Savings--Article
I just happen to be looking through the forums and this looked like an interesting question. I definitely agree with Elfriede that not everything can be automated and that having human being going through the web site or software is often required because the applications may be to dynamic for the tool or may take longer to write the scripts than to just have someone go through the site by hand. Also the test tools I have worked with don't distinguish if there is a graphic that is a bit out of place, catch spelling errors on a web site or handle security testing as a person go through the site.
But I have seen some advantages to using the automated testing suites. Like say for instance, running a stablity test on a web site to ensure that will handle load over long period of time instead of bring down the web site every half hour. This can cost you hundred of thousands of dollars or even millions. (I don't have exact numbers since the managers handle the monetary number crunching, I just tell them what the stats mean...Even if I did have the numbers I don't think I could give them out) E-commerce sites can have millions of people go through the site per day and many companies expect their sites to be up because that is how they can bring in revenue. Also we use the automated test suites for evaluating hardware or various pieces of third party software to find out what performance differences there are between them, which you may be able to save fair amount of money depending on how you setup your network architecture and tune your servers.
I would like to add one other thing though the automated testing tools are only as good as the people who use them because if you don't have someone that can interpret the data then you lose the effectiveness and any saving that the automated tools can offer.
Have a great day!