QuikTestPro vs UTP vs Certify
Our accounting systems are tested by a large user community. We are webfront end to client/server environment with one mainframe legacy to interface with.
We are leaning to Mercury --- honestly because of their position in the market. Seems safest. We've invested in other products & tools from smaller companies and have been left holding the bag when they downsize.
Is there really a big difference in setting up tests with the non techies (our user community)with these three vendors?
The sales pitch from UTP and Certfy seems compelling (a bit aggressive and annoying sometimes---sales people....) but we can't seem to find any wealth of users that would sway us from Mercury.
Before we sign the dotted line, anything from this experienced group would be appreciated.
Re: QuikTestPro vs UTP vs Certify
Realize something here, with any tool you will need people who can understand it and implement it (i.e., have a techical mindset). With any automation tool, and I mean any of them regardless of what the Sales Wonk is saying, will require some technical knowledge of testing & programming to implement the automation.
You cannot just record and playback anymore (even back in the old days you had to do some programming, i.e., logic in the script) and have to look at the whole process as a real project and have dedicated resources to it.
I highly suggest you go read the articles by Brett Pettichord and James Bach on automation. Also, Elfriede Dustin is a very good read. Read about the success and failures of automation. Then find the evaluation paper on this site (can't remember specifically where it is) and use it, or use it as a guide for yours.
Do a proof of concept with the tools (spend about a week to two weeks doing this) and see which one best fits your needs. Automation is expensive (initial investment and long term money, and time) so make sure you have the right tool to do the job you need. Once you pick a tool start off small with a Pilot project, and then expand from there.
Now talking about the 3 tools. QTP is an automation tool and does not have a canned library of code to do the testing. You have to build your scripts. UTP & Certify are automation frameworks that use the Keyword driven approach. This is pre-canned code for testing. But with both of these you still need to do script development to do the actual testing. I know with Certify (I have not used UTP yet) that you will still build scripts in the tool itself and will need to maintain them. Certify has a "friendlier" interface than QTP (or other test tool that is just a scripting language). So you get some insulation from doing the real hardcore script programming, and can have some non-technical people work with it. But again you will need someone to be the technical guru on the tool.
What these tools (UTP & Certify) allow you to do is seperate your technical test people from your business subject matter expert test people. You may only need a couple (2) of techical test people instead of a group of them (5-6) to do the actual development. This way you could have more SME's (4-5) working with the tool building test data and test scripts to do the actual testing. In some respects this may save some money over the long term.
I could go on for pages.... Just remember something. It is Automation, not Automagic. No matter what the sales person says, these tools are not silver bullets. They are tools, and need to be treated as such. A fool with a tool, is still a fool.
Good luck to you.