| || |
Calculating Expected Results - how much to calculate in QTP ?
Curious for people's opinions on this. I'm on a project where testers would like QTP to calculate the expected results.
1. place an order
2. modify the order
set price = 5.3
3. verify the modified order
My experience has been that it's hard enought to reliably code the reliable getting and setting of data from various problem GUI objects, and the first crack at a new project should have testers explicitly specifying all inputs and outputs.
But in the example above, you could save the record in step 1, change the modified value, and the change step 3 to have the tester specify only the changed value, and let QTP verify all 3 fields...very easy for this example, but now imaging an order requiring 20 or 30 fields, where the combination of fields 4 and 5 create a 'tailored description', and it gets more and more complicated with other field combinations and the order of Actions entered. Maybe 2 actions in a row cause one outcome, but a data difference in the 2nd step causes a different outcome. And of course the testers would like QTP to calculate that for them also. However, if you keep accepting the requests from your testers to have QTP calculate more and more expected results, your coding more and more of the business logic into QTP, and the testers are having less ownership and understanding of the test results. Sure the QTP script is more sophisticated - and so is the risk that you're going to introduce coding bugs which will further confuse testers as to the source of error when it occurs - application or QTP script.
Anyone share a similar opinion ? How do you explain - and do you even agree - that maybe the QTP should primarily get and set at least most test data explicitly specified by the testers, rather than have QTP script doing more and more expected results calculation.
After all, time spent coding calculated results is time NOT spent creating new scenarios that involve less of this 'fancy, elegant' type of coding.