We have a customer contractor relationship for the provision of maintenance support for a number of software packages.

The way things are at present is that a maintenance contractor sets the tests that should be applied, and the customer, due to a lack of knowledge, simply approves their recommendation and performs just those tests to prove the system is ok (for that maintenance release). These tests are assumed to be the best to perform a check of those faults which were fixed. Is this common practice? How can it be any different without the customer needing to develop internal knowledge of the software being maintained, in order to be able to truely agree or disagree with the tests the maintenance contractor selects on the basis of fault fixes? To apply all acceptance tests for the fixes performed would be unnecessarily time consuming and expensive.

The other point, is that if the customer were to select tests for themselves, would it be correct for the contractor to get fore-knowledge of what those tests are, in order to perform a pre-delivery run through to ensure acceptance success? Shouldn't these pre-delivery (pre Factory Acceptance Tests) be selected and run through by the contractor themselves on the basis of their confidence of the system to be delivered? And then these tests possibly suggested to the customer as those most appropriate to validate the fixes introduced? If this approach were taken the customer may then decide for themselves to witness the performance of these tests, trust the contractors own results, or just apply their own selection of acceptance tests.

It seems a waste of effort, to me, to define a process which perform tests twice (contractor only, and then contractor & customer). And also gives the wrong message to the contractor that so long as the acceptance tests pass, all will be considered ok?!

Is this true? Am I being too aggressive in my judgement of contractors and the maintenance acceptance relationship?

How do others of you go about maintenance acceptance test selection and operate the customer contractor relationship?

Thanks for any comments.