BPT Manual tests - using Iterations
I was having a chat with a colleague at work about the use of "iterations" when running a BPT component test manually.
Say, for example, I have a test that is composed of a number of BPT components, and that all of the components are grouped together into 10 iterations. And that for each of those 10 iterations, the tests data is different in some way such that each iteration is unique.
There were 2 points that were raised as to why this set-up is a bad idea:
1. You cannot report on iterations. This means the above test would look like 1 test instead of 10 in the reports that you can generate out of QC.
2. Let's pretend that iterations 1 - 9 passed, and iteration 10 failed. When the defect fix comes back, what do you do? Do you run all 10 iterations again, or do you just run iteration 10? If you only run iteration 10, the other 9 will be "Not Completed", so overall the test will be "Not Completed".
My colleagues preference was to split each of the 10 iterations out into 10 unique BPT component tests, so that they can be run independently of each other.
Anyone have any suggestions / remarks on this?
Re: BPT Manual tests - using Iterations
You should check out configurations and see if they meet your needs better than iterations. You can link requirements to configurations and when you add a test with configurations to test lab it will create for you once instance of the test per iteration so you can report on them independently.
Re the points above:
1. You can report on iterations, it just takes more effort. Use excel reports.
2. You should be able to just run that iteration without impacting the others at least that's the way I remember it with automated tests. Has been over a year though. You could also potentially choose to continue the test and pass the failed test. If you are collecting metrics like attempts to pass though this could cause issues.
When I've used iterations, I've only used them for data creation where I don't really care about reporting all that much. Where I've needed to report and the reporting needed to be simple as someone else was doing it I've created separate test instances (not tests) for what could have been iterations of a single test instance. I had a user defined field that basically said what requirement was being met by the particular data used for that instance.
I just ran into this same issue. Outside of creating a copy of the original BPT with only the iteration I want, I can't figure out how to do just 1 iteration.
In theory reporting shouldn't be too hard as you can just report percentages based on number of steps per BPT.
You don't need to copy the test, just pull it into lab multiple times. Same as you would with any parameterised test. Caveat as above around traceability to requirements.
Reporting based percentages of steps is almost always a bad idea. Steps are almost never equal.