We got a problem in our testing process. We auto test software that is under constant improvement. I am sure some senior QA Engineers out there have some good ideas how to solve this issue.
The problem is similar to this; our software package consists of a word spreadsheet tool that reads and parses documents in a proprietary binary format. This has a concept of a project. A project contains 6-10 documents. Testing is on WIN2K/XP and UNIX flavors.
This word spreadsheet parser is daily being updated and the test definitions, being documents, read by aforementioned parser, are used for automatic function tests.
Unfortunately, these test needing constant maintenance in order to keep up with the changes in the parsing module. DEV changes in parsing module are often largest and most frequent in the beginning of the product cycles but we still ought to test early and automatic. We got 500 test definitions and increasing.
Any hints on how to direct the development dept. to minimize QA dept having to rewrite/patch the test definitions each day? Possible solutions below please comment and give recommendations:
1. Should we modularize the test definitions somehow? I just donít see how it can be done?
2. Could one solution be to persuade DEV into somehow add a module to their tool that reads in ASCII defined documents ( with the least common denominator ) that are converted by the parser into binary format on which we thereafter auto tests on?
Maybe they could be persuaded into making large parsing changes only ones a month? Or we only test on monthly builds?
3. QA could refrain from develop any automatic testing until very late in the product cycle when no changes will occur, with the drawback of not being able to test all functionallity.
4. Any other ideas?
ISEB Foundation in Software Testing Certified