| || |
Risk of starting Test Automation before Manual Execution
I have added few points. Many more are welcome....
1. It is not advisable to automate test scripts in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases.
2. If user interface is changing extensively, cost associated with script maintenance will be very high.
3. Lot of defects would be addressed during Automation Script development, which might affect the development of test scripts and automation deadline.
4. Data mining issues will be the major concern and can hamper the productivity.
5. Functional execution result will not be available to Automation resources to validate the failure cases.
I can think of positive reasons of using automation early on. No more for reasons not to do it.
You are over simplifying and you seem to be assuming that all automation is GUI testing which is not the case. For example when scripting tests for web services if you have a well defined interface you don't even need to need to wait for development to start. You can write your automated tests before the dev team have written a single line of code. Another example, GUI this time, with Seibel if the the dev team can provide you with the object model and the GUI design again you don't need to wait for the dev team to start configuring Siebel.
Yeah if anything that is becoming more and more normal as agile practices become increasingly popular.
Writing tests out in something like Cucumber for behavior driven development before any development coding is started is becoming a best practice.
Once few years back, we worked on a automation for about 6 months.
Then a mail came from USA, project scrapped , we were given great thanks for (?).
The company was Tier-1 product based IT company with revenue of many billion dollars at that time.
If you want to consider this also as a one point in your ol.
Your post about the project getting cancelled after spending 6 months preparing made me thinkg of some antidotal stories. It was more chit chat than QTP related, so I put it in the chit chat area and named it "Procrastination".
Here is the link
I come from a point of view that Automation is not just QA, but also Engineering. As part of a BDD/TDD workflow, writing test first, even if it's incomplete is very useful. If automation is thought of as an after thought, and Devs have no involvement from the get go, then it'll be a failed exercise. For example, when functionality if finished, will you ask devs to take time off from working on the next feature to allow QA to catch up on automation? As much as people talk about ways of accounting for technical debt, it never happens. If Automation is not part of Dev, then it'll fail.
Sure the UI, id's, and even some pieces of the work flow are not established. That's a given. But with what you do know, you can scaffold out a test that is basically comments with a fail statement and pseudo code, which the developers can later fill in as the functionality is implemented.
What that buys you is the following:
1. Developers have clearly defined requirements, since they are testable requirements. It's very likely in the process of writing even scaffold automated tests, you'll realize, "Hey, this work flow does not make sense.". And it's even more likely, you'll find that the requirements in the story might be too vague to begin with if you have a hard time writng Pseudo code outline of the test.
2. As the developers flush out the functionality, he'll be the one implementing the body of the test. So both the code and test will be designed in a way that makes sense. For example, its easier for the developer to write both the test, and test hooks at time of implementation because it reduces refactoring if a Automation engineer comes back and requests a test hook (since the automation engineer has no clue of the internals).
3. QA is freed from getting incomplete deliverables. If the tests don't pass (because they are initially set to fail, then pass as devs fill in the details), then it's not worth the QA's time. They can be working on other activities where there is no shortage on.
Last edited by dlai; 07-26-2013 at 08:25 AM.