This may be a dumb question, or may be in the wrong place. Suggestions and advice welcomed.
I'm testing a large and complex java application with 5 core components, and a whole mess of APIs and supporting admin utilities. Oh, and did I mention that it runs distributed across multiple platforms in both Windows and *nix space? Oh, and there's a couple of database backends that we support (*cough* JDBC) as well.
So in planning the next round of testing, I find that I'm coming up with Avogadro's Number of test cases, each of which falls into several test suites. I need a system which will allow me to
<LI>Write up test cases quickly and "on the fly" (that is, I may have a set of test cases and, as I'm executing them, write down other test cases that occur to me).
<LI>Gather together test cases in various natural and un-natural groupings -- these being the overall test suites. Note that each test case may appear in more than one test suite.
<LI>Tie test cases to a limited set of initial test configurations and/or initial test data sets
<LI>Need the ability for multiple people to work on test cases simultaneously (so can't just keep one big file under source control).
<LI>As tests become automated, I need to tie the test case description (written form) to a test script which may be a Java class, a JUnit class, a bash script, a .BAT file or some higher level function)
<LI>Ideally I need to be able to track the results of running the tests on a particular build in a database; however, this can easily be done with a couple of lines of SQL :-) .
<LI>Did I mention that we're a startup, so this tool should be cheap![/list]
So as I start to do my test planning, it struck me that, while this is a much more complex area than I am usually testing, someone must have solved this problem in the past. Or maybe I just need to buy a book on Data Mining and a bug pot of coffee...
Suggestions welcomed. I've had a couple of goes with Search, both here and on Google, but I'm a little fuzzy in my thinking right now.
I don't want your question to go unanswered.
I think you've posted in a good forum, but the people on the Functional might have the answer so I'm going to move this to Software Process Improvement to see if you can get an idea or two.