UFT -number of test scripts 1 tester does/day
Well we have a new manager who doesn’t know much about automation test and wants the automation team to produce test scripts at a ridiculous pace. The number of script per day from an auto engineer depends on so many factors including framework, application, platform, testing resources, scrip maintenance, etc, etc,. However, this manger dictates blanket 10 scripts per person per day! He couldn't say where he got his matrix from. I have a distinct feeling that he is feeling his way out.
Our velocity has never been 10 scripts per man per day based of various factors above.
Question: What are your suggestions? What is your experience on situations where you faced this kind of mandate?
I've only been using QTP/UFT for maybe 8 years.. but I don't think I could write 10 tests per day.. but it depends....
fundamentally: what's a test script?
i.e. one test is not equivalent to any other test..
I could write an 'End-to-End' test that verifies a whole bunch of things as part of a business process.
I could write a stack of smaller tests that test very specific things - whether they are categories of tests for a form (mandatory fields, field lengths, labels, accessibility, web standards) or very small distinct parts of a business process (probably with a few data variations..).
In short, tests can be very very short, or very very long. They simply aren't equivalent in time, effort or efficacy.
further: If I write a script I'm probably going to data-drive it.
If I feed it with a 50-row spreadsheet, then while there may be 1 QTP script that took a week to write, it is equivalent to 50 explicit manual tests, and I've met the 10/day requirement.
...but it only looks like 1 test.
Last edited by a.irvine; 08-10-2015 at 05:45 PM.
... just another Tester ...
Alex its a generic answer which I explained. But he insisted that its 10 individual stand alone, each script executed start to end, that test 1 business function.
I am not sure what he means by 1 business function. Like you said, it could be a click or a number a paths along the application.
Let me show him tomorrow what we do and perhaps he may have other comments.
would like any other suggestions
Helen sounds like you've got a right clown there and its very difficult to take people like this seriously, they need educating (I really mean firing but hey ho).
I would start by asking them what their definition of a script is (I doubt they will have one but got to start somewhere).
Depending on their answer you could then set their expectations by outlining what is involved in developing and proving 1 script.
I have faced kind of problem and I solved this by feeding the basics of automation design and how it works.
Go with set of questions and answers; and explain him.
Automation productivity factors:
1. What is the technology used to develop the AUT?
2. What is level of fields design quality i.e. How the dev handled the UI controls naming conventions and other properties which really impacts your object identification time
3. Steps to be executed for a test case?
4. No of Fields/controls should be touched to perform a navigation?
5. No of screens should be navigated to complete a test case?
6. Level of flexibility they expect? Hard coded data or parameterization for almost all the controls?
7. Test data availability for dry run?
8. Validation level? UI validation, UI to DB validation, UI to webserices, etc.
9. Integration with test management tools if any
10. No of reusable funcs to be designed for a test case?
Better go with some proper template for estimation and explain him.
Thank/Like to help others if my input helped you !!!
I am speaking for myself, not for my employer nor any one.
Since others have already said it, I'll abstain from stating the obvious...
I think my first step would be to ask for a proper framework:
- That manager said a script should test a business function - what does s/he mean by business function? A business requirement, a functional requirement? A mix of both? Something else?
- What does s/he mean by script? (1)
- What should be automated? What shouldn't?
- How robust/reusable does s/he want the tests to be? (I can churn out r&p scripts a lot faster than writing them from the ground up, but they won't be as effective and will require more maintenance later on)
(1) I could write 10 scripts to verify a single textbox (min length -1, min length, max length -1, max length -1, max length, max length +1, alpha only, numeric only, alphanum, etc...) and still have not proven much; I'd rather have created a single script and fed it a series of data and expected results, but hey, if the manager is an ***, I can be one too...
So I'd sit down, preferably with a concerned third party along the ride (someone who can act as a kind of independent 'backup' to provide you with unbiased support) (for example a project manager) and act for clarifications.
I would handle this by giving him my 2 weeks' notice and updating my resume. This is not going to be a pleasant work experience. Even if you manage to talk some sense into him, it's not going to be last dumb thing he does. He's clearly got no experience running an automation team and too much ego to learn the job before thinking he can do it. Run! Run fast!
Might want to try and find out which book on Test Automation they read so you can second-guess his responses to your questions.
oh my god -unbelievable! that sucker opened HP mercury NewTours demo web page and pointing to each icons and links saying that even he could write automation scripts at a pace of 10/day! Very arrogant person. What a joker! On top of that he said he was going retrain us to write scripts faster lol! I had wanted to say lots of thing in-line of what you guys saying above but I just hold my calm. After 10 years as an QA analyst, and 7+ years as an automation engineer, I did nothing today except drinking record amounts of coffee! I can't wait for his re-training sessions. Anyway friends, from time to time we do meet people like this. Just be calm and but firm. Never lower your standards. With time they will see the wisdom in what you are doing. I am just sharing one of my worse experiences.
To be fair that is how HP sells it.