| || |
Does anyone have experience testing a Mathematica developer application? I'm looking for help and examples of how to create tests for this platform either using their developer testing tools or anything else you have used that worked.
In section 1.12.5 of The Mathematica Book, you can see how they tested the product themselves (roughly). Or check out their on-line version. That might give you some idea of how to test your application since it must use the Mathematica kernel or make calls to it.
Basically, they use an automated system (with some manual checkstops). You could, of course, attempt to replicate something like that yourself in Mathematica with the Developer's Kit. Beyond that, I have tested programs written for Mathematica, some that I wrote myself and others that other people have written, like tensor packages (GRTensor). With that you can write Mathematica notebooks that check your application out by running tests against your application. Alternatively, you can hook into your application just like you would Mathematica via another language, like Java for example. (You can use Wolfram's J/Link for that.) Then you can build your test suites via the language and have it hit the application, which then calls the kernel.
Presumably the application you are referring to does formulaic calculation and, as such, it is pretty easy to deduce what needs to be done in relation to that. Write a mini "test engine" that simply feeds your formulas a variety of values. You can also write little stubs that do various things. Check out these ideas on language helpers. That is not testing, per se, but it can give you some ideas.
You can also test timing information (i.e., performance related parameters) relative to your application. For example, I like to calculate with known Mathematica values and then apply to those to what I am using via a performance ratio. For example, if I use Sin[x] in my calculations, I know that Mathematica is rated at 0.13 seconds for this. Which means my program should be equivalent to that - with due consideration being made for anything else being done in relation to that calculation, of course.
I also recommend looking at section 2.12 (page 630) in The Mathematica Book. That is not about testing, per se, but it does talk about linking external programs to Mathematica and making the appropriate calls.
Basically, however, you just hook into the developer application you have (such as an add-in, or whatever you are doing) in the same fashion that you would hook into Mathematica and then execute the formulas of your application. You can also do this directly in Mathematica by simply writing notebooks that call functions that do the same thing. So it really depends on how you want to do it. For example, you could use the NotebookCreate method or you can use NotebookPut and then feed the notebook information you want it to have, forcing it to act as a type of test stub for you.
If you ever do graphics and need to check those, that can get a little trickier and is where the manual aspects come in. But you can do things like use the SurfaceOfRevolution function and the RevolutionAxis methods together and then manually check out what it shows you. For example, if you are checking surfaces and you want to make it clear where the flat surface is actually coming from (thus validating your surface rendering) you might do something like:
Of course if you are not worrying about graphics, then this hardly matters. I am just trying to give you a wide understanding of things you can do. This does not even begin to go into making sure your equations are optimized or making sure that your styles are showing up correctly. (The former can be automated to a large extent, the latter is going to be manual.)
Just some thoughts. Hope they help.