| || |
Is TestManager that thick?
My company has asked for an Enterprise wide Test Management tool. They have many testing vendors scattered across the globe conducting functional testing on their behalf. Visibility of testing activity is frustrating as they rely on reports being generated, usually late and with little or no reference to any traceability. Also vendors are not retaining tests and building upon them with subsequent releases, hence testing costs the same over and over again.
Analysis earlier this year of all available products identified the following products as potentials:
1) Mercury – TestDirector;
2) Plan IT - T Plan (Now owned by Segue and called SilkPlan Pro) &
3) IBM Rational - Test Manager.
4) Compuware - QADirector.
Mercury – TestDirector (TD) was identified as the recommended product based on its breadth of functionality & ease of integration supporting our Test Management process. However negotiations with Mercury have gone sour and we have now been directed that we are cannot have this product.
All other tools have thick clients. With our constraints (see below), and my experience with thick clients, there is huge concern having a repository on one side of the world and someone on the other side writing a query for a report that could flood the network. Some of the alternatives have thin clients but they are either reduced in functionality, extremely bug ridden or resides on IIS (non approved by the company).
Against the Constraints, Issues and CtQs below, I am after opinions from anyone who has many Vendors across many countries / timelines conducting testing on behalf of your company and has provided a NON Mercury TestDirector solution.
· Has anyone used any alternative across many countries / timelines? What were the main issues
· Is flooding the network a risk? Or do these application page the results and hence reduce the traffic to only what is presented, a page at a time?
· Are the thin clients really that reduced in functionality compared to the thick clients and are they that buggy? Please dot point the issues
· If cornered into making a decision and TD was NOT an option, what would be your order of choice against the alternatives?
· Compared to TD, which tool has an integrated solution for Test Management?
. What would be the cost difference between each of the alternatives against TD, % more or less?
· Testing is expensive - Outsourcing Test Vendors (OTV) do not reuse previous testing; all estimates / quotes / invoices state same cost as if it was a new product / application
· Testing is expensive - OTVs do not structure their detailed test reports as per agreed test plans / scenarios / cases creating an Apple to Oranges syndrome. Too much time taken trying to unravel the results back to the plans / scenarios / cases.
· Severe defects are raised often too late – Traceability is not maintained
· Testing of critical functionality is sometimes overlooked – Traceability is not maintained
· This is a large organization that outsources testing to many companies
· Peaking with about 200 concurrent testers, OTV can be located anywhere in the world
· The network is secure and each of the OTV are on the secure network
· Network bandwidth has limitations
· No solution can over utilise network bandwidth
· No IIS installations
Critical to Quality Requirements
· Test Scripts are this company’s intellectual property, OTC must leverage previous Testing scripts even if developed by another OTC
· Test Reports must be scoped as per test plans and make reference to test scenarios / cases
· Traceability is paramount to ensure we have coverage, quality & accountability