The online community for software testing & quality assurance professionals
Calendar   Today's Topics

Lost Password?


Quality Engineering >> Software Process Improvement

Pages: 1

Reged: 05/25/08
Posts: 4
Loc: Karnataka, India
How to measure productivity of QA Engineer
      #504926 - 08/01/08 02:34 AM

What I know:
For assessing a QA Engineer, we generally look for the count of defect logged by him/her. This gives idea about the effort of QA and also the technical expertise of the engineer.
Few more criteria for rating a QA Engineer:
1. Count of test cases written
2. Count of defects logged
3. Debugging skills
4. Automation usage/scripts creation
5. Input in innovation or idea to improve process or stability of product. (Also includes enhancement defects logged)

Issue I am facing:

I have 4Yrs exp and now as I am looking for a lead position I am asked to come up with metrics that shows productivity or quality work of my team.
My project is in a phase where even if there is a new feature coming in, we rarely get defects. Like 4 defects from one person in 2 months. We rarely get new features to test. In this situation the 1st two criteria to rate a QA engineer eliminates automatically.

There are very few ideas that my team has, I have to push really hard to get ideas from them. As we all know innovation cannot be forced on someone. I am left with the Automation usage and script creation to judge my people.


Can you suggest me some parameters on which I can come up with some productivity metrics and rate my team. I need some criteria (metrics) to judge them and find areas they can improve in.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere

Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: How to measure productivity of QA Engineer [Re: diehard28]
      #504967 - 08/01/08 04:21 AM


My project is in a phase where even if there is a new feature coming in, we rarely get defects. Like 4 defects from one person in 2 months. We rarely get new features to test.

What do you and your team do all day?

- Joe
Visit to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 09/17/01
Posts: 3979
Loc: Yankee Land
Re: How to measure productivity of QA Engineer [Re: diehard28]
      #505006 - 08/01/08 05:24 AM

Personally I find that measuring productivity by bug count is a losing proposition, and a lazy way to judge, basically you are counting on your development team to suck so QA looks good for opening lots of bugs. If you are really looking for a way to measure your team then use your other criteria, check for the test cases (how effective are they?) Are projects done on time? Are they done in a way you can look at your team and be proud, or are the releases embarassing because defects were missed?

Automation and Innovation are good criteria, but maybe you really shouldn't be asking us for specifics. Go to your team, set some goals for them and see how they do to meet them. If they are not forthcoming with goals, check out their interests and see what they want to do in the future and set tasks for them. Now if they are not forthcoming with ideas, is it the team or are you not leading them in the way you expect? There are cultural differences, but in the US I could approach my team and have them be honest with me, so if I sucked in something they could tell me without reprisal, I am not sure its that way in India. Fit the idea into your own team though, question them, prod them, then set tasks for them and see how they achieve things.

Otherwise you can check certain metrics against test cases as well, how many are automated, has the number improved over time. Are you executing X% by the end of a project, or at certain milestones? Those are pure numbers you can look at and see where things stand as well.

- M

Nothing learns better than experience.

"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown

Now wasting blog space at QAForums Blogs - The Lookout

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 03/09/07
Posts: 3755
Loc: Waterloo, Ontario, Canada
Re: How to measure productivity of QA Engineer [Re: michaeljf]
      #505028 - 08/01/08 06:02 AM

I agree with Michael. I would never measure the performance of my employees by the number of bugs they reported. I wouldn't suggest judging productivity by any measure that you can apply a metrc to. See, using a metric to measure something like productivity won't necessarily make them work hard, it will make them try to find ways to defeat your system (like breaking one test into 20 tests) or reporting really simple errors instead of digging for the big errors.

It also sounds like you have some morale issues. You should look into some team building exercises as well. You know one thing that we did? When we initially implemented our exploratory testing we also implemented daily QA meetings for 45 minutes to an hour. This time was to pick our sessions for the next day, but we called it a "brag session". So we basically talked about the bugs we had found that day, or what we'd been up to. Basically, we'd brag about something cool or tough that we did that day. We keep the meetings light-hearted and joke around or talk about world events, if there is any time left after we've finished up.

These meetings can do a few things. They can build team morale, since there is a lot of joking and fun happening, they will give you a great idea of what people are doing with their time during the day (so if someone hasn't reported on much for the last couple days then talk to them and see what they're up to), (HUGE) it promotes innovation because people will often use ideas or methods presented here in their own daily application and might help to expose other issues in the application. There is just so much you can get out of this type of an open meeting. I never thought there would be, but it's really helped.

Here's my take. Are you a QA Measurer? No, you're a QA MANAGER! There is nothing in this title that implies measurement. If you are trying to measure someone in QA to determine their productivity then I think you've probably already failed. You will fail whoever you're trying to provide this metric to, because it simply can't be accurate, and you'll fail your employees, who may be working hard at uncovering serious issues in the software, but you simply never asked what they were up to. To effectively manage, you need to be informed about what your team is doing and what difficulties that function presents. If you don't know, then you won't be productive yourself, because there is no way to measure something that you can't even appreciate or understand.

9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 12/19/00
Posts: 15290
Loc: St. Louis - Year 2025
Re: How to measure productivity of QA Engineer [Re: brentpaine]
      #505058 - 08/01/08 07:21 AM

How not to:

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 05/25/08
Posts: 4
Loc: Karnataka, India
Re: How to measure productivity of QA Engineer [Re: JakeBrake]
      #505750 - 08/05/08 01:50 AM


We spend most of the time in verifying defects logged by us or customer or client.
These fixed come in form of patches, builds and different types of patches. Can't give more details on this as it will be too specific to my company.

Becomes quite monotonous and not much to actually track other than deliverables.


I did go to my team to check for criteria on which i can start some measurment.
You are right about the culture differance, they started thinking who am I to measure their effifiency. But they spoke up later after i told them the objective of this.
The parameters i mentioned is actually disucssed with my team. One of them said to check for techinical expertise, but he could not say how to measure. Last thing he said was to conduct written test.

I like the idea of Automation, even I came up with one. But it has other problem. When I joined this company 4 months back, I was told that my module has 100% automation. And when I started working on it I found that it has not even 50% automation.
Till now there was no one minutely tracking the activities, so things seem so messed up.
I came up with this idea of audit for automation. If person A writes a script, person B will run and check if it workes. This was not at all accepted by the team as they thought that it will be like re working on same scripts.

I have some ideas but it is difficult to be implemented. I saw the testcase has not been updated from last 1 release, even if some one updated it it is in their local hard disk and not in commin ripository.

Thanks for the information. I would like to implement "brag session" as my team needs some positivity towards things and little bonding too.

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 04/02/03
Posts: 4546
Loc: Wisconsin, USA
Re: How to measure productivity of QA Engineer [Re: diehard28]
      #505822 - 08/05/08 05:04 AM

I'll echo what others have said here, but put it more succinctly.

If you measure performance based on defects found, you are setting someone up to lose, no matter what.

If you have a particularly good developer (I have worked with a few), there are few bugs to find and the tester looks bad.

If you have a tester who finds tons of bugs, then the developer looks bad.

Look for the thread that talked about paying testers a bounty for bugs found and see what happened in that scenario.

You are setting up a lose-lose situation here. You need to look for win-win.

Post Extras: Print Post   Remind Me!   Notify Moderator  

Reged: 01/22/07
Posts: 145
Loc: UK
Re: How to measure productivity of QA Engineer [Re: DSquared]
      #505926 - 08/05/08 08:42 AM

It's one of the classic metrics/measurement issues: "Is what I end up with really a good indicator of what I want to measure?" I whole heartedly agree with other posters that defects found is no measure of good testing.

You could discuss with your team how as a team you could show the effectiveness of your team, but you need to look at not only what happens before but also the test phases. In the end what most development managers or customers care about is that once the product goes into production there are few major issues. Good testing contributes to this, but is not the only reason for it.

one useful question is "What would be the impact if the QA team was not here?" It may give some clues as to something that can be measured.

Also consider looking at the other side of coin - how would we know if we were doing a bad job of testing?

If you want to look at an individual level then some form of scoring may be the way to go (although I know a lot of people do not like doing this). Think about the attributes you wish to promote and score the team. Subjective certainly, but often the basis for many staff development schemes.

The story so far:
In the beginning the Universe was created.
This has made a lot of people very angry and been widely regarded as a bad move.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1

Extra information
0 registered and 4 anonymous users are browsing this forum.

Moderator:  blueinatl, AJ, Daniel_S, michaeljf, swt88 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Topic views: 6075

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5