Solvency Testing / Validation
I have just started as the test manager on a project testing a companies future solvency (i.e. assets continue to exceed / liabilities. Which is focussed on predicting future events / risks that might impact the solvency of that company.
The problem is that standard testing where you know your expected results would not work, as the key deliverable of the project is to deliver a complex actuarial formula / model that is suitably robus. The simplest example I can give is how do you test what the weather will be for each day in June 2011.
The aproach the project is taking is known as formula fitting, whereby they start with an inital formula, then run scenarios through the formula, validating the results are as expected and then continue to evolve the formula until such time as it is believed the prediction on future solvency is as accurate as possible. For anyone who may have worked on Basel II projects they may have already had to resolve such a challenge.
I have some ideas on what could be done from a validation and testing perspective, but rather than starting from scratch I was hoping somebody may have already had experience and can share this with me
Re: Solvency Testing / Validation
The validity of a complex model is usually performed by a senior modeling analyst.
From a QA perspective, there is little you can do apart from take a baseline of results and monitor for volatility.