We have around 50-60 web based application and all of are covered by manual testing. The plan to start some kind of automated testing with SilkTest. We want to hire some company which will create some kind of test harnesses for most of apps and then QA on site will support created SilkTest test harnesses. The issue I see that app may be change dramatically in the new release and there would be no use for already created silktest scripts and we have to rewrite all of them. In this case may be we need to spend money on developing script with third party and start to write them from beginning by ourself?
1. Hiring an outside firm to develop a test harness for you will tie you to that firm until you develop internal resources to support the harness code. I'm reading into your post somewhat here, but it sounds like the outside firm will be the technical resource and your onsite QA's will be the business/application experts (but not necessarily technical).
2. Strategically plan the harness itself to minimize maintenance due to typical application changes.
3. Strategically plan what to automate to maximize utility of the automation while minimizing the known throwaway.
4. Don't try to automate everything from the get-go.
5. That you have manual test coverage alrady is a huge benefit.
6. Don't wear "automation is only for regression" blinders.
[ QUOTE ]
we have some resources on site, but my main concern it is hard to support code written by somebody else.
[/ QUOTE ]
Totally agree, especially if your on-sites normally focus on manual testing, and perhaps aren't programming savvies.
You might want to identify some core resources (perhaps only 1) who are responsible for technical upkeep of the harness/framework. They should also be responsible for disseminating sufficient info about the harness to your QA folks so they can efficiently provide either appropriately structured cases for automation or be able to automate themselves using the provided harness.
[ QUOTE ]
what is wrong with "automation is only for regression"?
[/ QUOTE ]
Nothing wrong with regression-only automation. But keep your eyes open for opportunities to leverage automation for other purposes. As an example: we've had great success using automation to drive large data permutation sets through the UI. It's used both for regression purposes and whenever backend changes are made impacting the UI response.
I want to share some results. We didn't get any good results.
1) Not enough time was given to on site QA to support automation efforts
2) Off site silktest developer wanted to have detailed manual test cases for automation, but after receiving them didn't ask any question during test case development. As result developed script were not very useful.
3) Bad on site management of automation projects.
Basically the approach: we give you specification, you return working code does not work. We need someone to work on site and someone who has a domain knowledge of our applications