We got an platform (runtime environment) to test.
A decision was made to test the platform mainly thru an application. Some unittesting is done on platform too.
The application is the only thing that will be running on the platform for a long time.
In order to create an application a Development Tool is used.
Unfortunately the Dev Tool (in function similar to a compiler) is not yet producing an executable application to test the runtime with. All software (Platform and Tool) is developed by ourselves.
Question is; how do you persuade Dev Leads to start focus on completing the Tool (at least partly) so at least some functionality in the platform could be tested?
QA Dept have raised these concerns to Product Management that its impossible to test efficiently before at least some parts of the tool is finished.
Question; is it fair to raise the concern further to VP or do we have to rethink how to test the platform and application?
I guess Microsoft had a similar interesting issues when they had to test .Net platform using Visual Studio .Net ? How did they do it?
There's a chain of command for a reason. It's best to operate within that chain, at least if you like your job and need the money. I would ask the QA Manager if he/she would mind if you brought this issue to the attention of the VP. If you circumvent authority, authority will remember at the most inopportune times.
It looks like the QA Manager has things in hand, honor his/her dedication to a quality product and see if you can start writing test cases to run on it now.
Hi, sounds like you have an interesting task there. Rich makes some good points but by the sound of it, you're being paid just to turn up & not actually doing much? Have you already created your test suite based on the known requirements? That's what I'd be doing while I waited for the SUT to reach a testable state.
Perhaps it would help to know what your Development model is. If they're still wallowing in waterfall then I guess you've got a hard road.
Can you make your own test-based tool in the runtime to reduce your reliance on Development? If there is no way to harness the runtime without the development test app then you're really pushing the proverbial up a steep incline...
I would keep the pressure on to develop at least a very basic app for the runtime - once you discover the key faults, the test app could be enhanced accordingly...
I would do both. I would escalate the issues by producing a risk assessment document and putting it through "normal" escalation channels and at the same time, I would rethink the testing strategy based on what was available to test with. I think it is key to document everything so that your area is not held to blame if significant error occurs in production. You might, in cooperation with development, want to develop a phased approach to your testing, testing to a specific "milestone" based on what was available to test, adding test cases as new pieces of functionality became available. If you sit down with development and come up with a plan, your testing effort will actually be more organized, and everyone will understand what will (what can) be tested and when. If you go this route, I think it would help you set expectations more reasonably.