How do most of you organize test sets? I prefer to create test sets by functionality to mirror my test plan tree, and then just continue to add new test cases accordingly. Does anyone see the benefits of creating a new test set for each new product version you test so that you can pick what tests to include? I am not sure why you wouldn't want your test sets to be somewhat consistent and preserve test run history. Also, I would rather just not run some test cases in a set than keep creating new test sets. Currently I am proposing to change the method of organization from a new test set for each version, so any opinions as to pros/cons would be of interest.
When we first started we did a functional decomposition to break out the function points and arranged the test plan tree accordingly. All tests were identified by number as each folder and subfolder was named with a 3 or 4 digit number prefix and the prefix numbers combined identified the tests:
(001_002_0001) Test name
We had a new project for each SW Revision. That was OK for a few years but there was always a question as to which tests to carry to the next project so we decided to keep one project and add in the revisions. We started adding a prefix letter to denote the REV. to the test and the folders so the above test for REV 2 would be (A001_002_0001) and our test sets are identified with the same overall number sequence. That way we know that if some of the tests in the test set have changed, we add the prefix to a new test set and include the old test that still work as well as the new tests. Hey it works for us.
I hope this is not too confusing, but it works well for us and permits the TD history to be continuous allowing for graphs, charts, and reports to cover several years indicating our progress.
Success is the ability to go from one failure to another with no loss of enthusiasm.
~ Winston Churchill ~