monki
Member
Reged: 02/23/08
Posts: 54
Loc: Hong Kong
|
|
Hello, I am a project manager for a small software house. Our development team used to consist of 10 developers and haven't had much QA practice in place. We recently learned the importance of software testing and hired a couple of QA analysts, each with a few yrs of previous testing experience.
My question is, at year end how should I be evaluating the performance of these testers? Should I be evaluating based on the number of bugs they found? or the amount of documentation they wrote? If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
Thanks.
--------------------
IBM Certified RUP Specialist, ClearCase Administrator, WebSphere Deployment Professional, DB2 Administrator
|
philk10
Advanced Member
Reged: 01/20/05
Posts: 578
Loc: UK
|
|
Any metric based evaluation can be gamed by an astute person - they could either report a lot of trivial bugs, report numerous cases of a bug rather than 1, report bugs that werent bugs...
same would apply to documentation - use big words, double spacing, lots of references and appendices
and why wait to year end ? that used to be a pet peeve of mine, we used to have yearly reviews. Why wait 12 months to tell someone they're not doing well ? cant you evaluate and coach them more often ?
and now a positive suggestion, read this Sticky Minds article from Elfriede Dustin
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=3281
--------------------
 and My Blog
|
Pragathi V
Member
Reged: 12/08/05
Posts: 142
|
|
Monki,
Everything is covered in the article given by Philk above. To share my company's way of evaluation of testers....They will evaluate the testers after every quarter based on some key factors like Test Case Design, Test case execution, defect logging, knowledge on test tools, company's test processes, project technology, domain knowledge, functional knowledge and other soft skills. The tester will be rated against these areas and thus the evaluation process will start.
Hope this helps to some extent.
Regards,
Pragathi.
|
michaeljf
Veteran
Reged: 09/17/01
Posts: 3526
Loc: Yankee Land
|
|
How about adding things in like goals? Communication among peers? Raising issues, or improving processes? Finding new tools or methods to improve efficiency?
There is a lot more to a testers job than just bugs to base performance on.
- M
--------------------
- M
Nothing learns better than experience.
"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown
Now wasting blog space at QAForums Blogs - The Lookout
|
Joe Strazzere
Moderator
Reged: 05/15/00
Posts: 10092
Loc: Massachusetts, USA
|
|
Quote:
My question is, at year end how should I be evaluating the performance of these testers? Should I be evaluating based on the number of bugs they found? or the amount of documentation they wrote? If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
How do you evaluate developers? By the amount of code they wrote? By the amount of documentation they wrote?
How are you evaluated? By the number of meetings you attend? By the amount of documentation you write?
If so, then I guess you could stay consistent and evaluate testers the same way.
I would never evaluate anyone - not project managers, not developers, not testers - mostly by the amount of anything.
Instead, I evaluate people based on their overall work - are they doing the things the company needs them to do? Are they doing these things well?
This might be harder than just adding up lines of code, or number of bug reports, but is the only reasonable approach, in my opinion.
--------------------
- Joe
I speak only for me. I do not speak for my employer, or for anyone else.
Visit my new blog All Things Quality
|
taroli
Member
Reged: 11/26/06
Posts: 61
|
|
HI,
I don't think testers performance would be done based on the number of bugs they have reported. I feel its all gambling.
While doing performance review they dodnt consider the number of bugs found by you or something like that. They mainly consider your knowledge on the domain and products, number of successfull releases you made, the documentation you maintain, your attitude, knowledge transfer, ability to innovate new things, your way of handling things during high work pressures & so on.
Its better you maintain a excel sheet where you note down your daily activities so that you can have a proof to read out during performance review.
In most of the companies performance review would be on yearly basis. You cant judge a performance of an individual in quarters.
I dont know about other companies but in our company of 30 people we do like this and finally they say testers do nothing, so performance is poor. Good Conclusion nah:)
--------------------
If i cant then who can?
|
rtehve
Active Member
Reged: 08/21/00
Posts: 1186
Loc: Warriewood, NSW, Australia
|
|
monki
The truth about evaluating anyone in any profession is often very very subjective. Often a new employee gets off on the wrong foot and their manager is just looking for a reason to get rid of them.
I have seen many feeble attempts by companies to make the process of employee review objective, but every effort I have seen is just window dressing for what is a subjective measure.
If a tester works on a well run project, they will be less areas of contention for the tester so they may look like they are doing nothing.
If a tester works on an abortion of a project, their performance will be buried under many layers of issues.
The only measure I have seen that has some objectivity about it is the number of defects raised that turn into fixes as a ratio. But again if a tester works on a well run project, this does not mean much either.
I was working in a UAT environment when the product I was working on showed 2 major defects that were basically an incorrect interpretation of an interface spec. One caused billing amounts to be out by factor of 100 (off by two decimal places). When I broached the issue with the SIT manager he was very embarrassed and very apologetic. I had very little confidence in this guys work and I started finding a lot of other minor issues.
About a month later at a monthly meeting he got an award for the good work he was doing. I am not a back stabbing person (I could have done a right job on this guy) but it did get up my nose a little. No idea how they came to the conclusion he was doing a good job, all the issues I found (that he should have) were all documented in the bug tracking system.
--------------------
Robert Tehve
rtehve@bigpond.com
|
yagsy
Advanced Member
Reged: 11/26/01
Posts: 560
Loc: San Diego, CA
|
|
Joe wrote:
Quote:
Instead, I evaluate people based on their overall work - are they doing the things the company needs them to do? Are they doing these things well?
To expand on this, I would like my supervisors to evaluate me where if I am not meeting what the company needs me to do and not performing well, that I would investigate why and try to offer assistance, perhaps in further training. Or, with closer examination of the QA process itself might be the reason why QA team members are not meeting the needs of the project and company. Invariably I find it's not that people are performing poorly but instead the process is failing somewhere. Usually this falls into the project planning.
Have any QA members been involved in the process management at all?
--------------------
Going out of your comfort zone requires failure. True genius is measured by your recovery.
...Jean Ann
|
rtehve
Active Member
Reged: 08/21/00
Posts: 1186
Loc: Warriewood, NSW, Australia
|
|
yagsy
"To expand on this, I would like my supervisors to evaluate me where if I am not meeting what the company needs me to do..."
I kind of agree with that, but even at that level there will be a lot of subjectivity. To use the word "measure" means an objective process that comes to the same conclusion no matter who does it, employee performance reviews are definitely not such !
"the QA process itself might be the reason why QA team members are not meeting the needs of the project and company."
100% agree with that, as a company evolves eventually someone comes along and goes over the top with many processes in trying to rectify an issue at the time. Often the resulting change in QA processes are counter productive to everything else and often don't address the issue that sparked the change anyway.
Also what happens is that if a person dares to challenge the process they are met with force and treated as trouble makers. I find it easier to deal with QA greenfields rather than established formal processes that are past their use by date !
"Have any QA members been involved in the process management at all? "
I would actually answer the opposite to what this question suggests, very rarely have I been in a job where I didn't either have to invent the QA processes or fix major flaws in existing ones. Many organisations in Australia use contractors for their QA/Testing process which means they don't treat such as intellectual property and the wheel gets continually re-invented.
--------------------
Robert Tehve
rtehve@bigpond.com
|
monki
Member
Reged: 02/23/08
Posts: 54
Loc: Hong Kong
|
|
Have been a very interesting year since I started this post, I am still with the same company.
Other than myself, we have used 5 different QAs (1 sr tester, 2 jr borrowed from another team in the company, 2 contractors whom cannot speak english)
very interesting how things turned out... i am at work now.. so maybe i will type the story when i got back home 
--------------------
IBM Certified RUP Specialist, ClearCase Administrator, WebSphere Deployment Professional, DB2 Administrator
|
SanthoshTuppad
Newbie
Reged: 04/16/09
Posts: 23
Loc: Karnataka, India
|
|
Quote:
If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
Thanks.
Is there anything called Bug Free? When a Product is tested and the problems are reported and they are fixed by the developers it doesn't mean all the problems have been fixed.
--------------------
TestersBlog.com | RookieTester | BugRepository | WeekendTesting - Test . Learn . Contribute...
|
SanthoshTuppad
Newbie
Reged: 04/16/09
Posts: 23
Loc: Karnataka, India
|
|
Quote:
Quote:
My question is, at year end how should I be evaluating the performance of these testers? Should I be evaluating based on the number of bugs they found? or the amount of documentation they wrote? If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
How do you evaluate developers? By the amount of code they wrote? By the amount of documentation they wrote?
How are you evaluated? By the number of meetings you attend? By the amount of documentation you write?
If so, then I guess you could stay consistent and evaluate testers the same way.
I would never evaluate anyone - not project managers, not developers, not testers - mostly by the amount of anything.
Instead, I evaluate people based on their overall work - are they doing the things the company needs them to do? Are they doing these things well?
This might be harder than just adding up lines of code, or number of bug reports, but is the only reasonable approach, in my opinion.
True. Testers Performance can't be evaluated by the amount of anything he/she does. If a person writes a lot of documentation and if a person writes the same documentation in precise and concise way then you can't evaluate both of these. So evaluating performance on the amount of anything is a bad idea as mentioned by Joe Strazzere.
--------------------
TestersBlog.com | RookieTester | BugRepository | WeekendTesting - Test . Learn . Contribute...
|
Peter Ruscoe
Veteran
Reged: 03/18/02
Posts: 6902
Loc: Tampa Bay
|
|
Quote:
Quote:
If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
Thanks.
Is there anything called Bug Free? When a Product is tested and the problems are reported and they are fixed by the developers it doesn't mean all the problems have been fixed.
It says reasonably bug free. That word makes a lot of difference (though "reasonably" is clearly subjective and product - and customer - related.)
|
GoGreen
Active Member
Reged: 09/01/08
Posts: 1000
Loc: Neverland
|
|
Quote:
How about adding things in like goals? Communication among peers? Raising issues, or improving processes? Finding new tools or methods to improve efficiency?
There is a lot more to a testers job than just bugs to base performance on.
- M
Exactly!!! this is what i wanted to add. Evaluation doesnt only include his testing, test case design ability, test planning, No. of bugs detected etc etc... also need to check how good his knowledge is in Process , what are his Value adds to the company? like has he provided any training/mentored juniors???
Also what is his behavior? his attitude towards Team members? evrything counts. well these points are beside the Technical stuffs.
--------------------
Regards,
Tinker
"My own business always bores me to death. I prefer other people's".
Cecil Graham, Act III by Oscar Wilde.
|
brentpaine
Veteran
Reged: 03/09/07
Posts: 3754
Loc: Waterloo, Ontario, Canada
|
|
Quote:
Quote:
Quote:
If our developer are strong and developed a reasonably bugs-free software, then how are we going to evaluate these testers?
Thanks.
Is there anything called Bug Free? When a Product is tested and the problems are reported and they are fixed by the developers it doesn't mean all the problems have been fixed.
It says reasonably bug free. That word makes a lot of difference (though "reasonably" is clearly subjective and product - and customer - related.)
I work in an environment where we handle very specialized applications. We have about 5 products in the stable along with mini apps that are sometimes written to suit specific client needs. I just did a review of our customer service/support logs. We've had 36 reports of software failure (field issues) over the last 6 months. I consider this reasonably bug free.
That's not to say we didn't find anything in QA. There are thousands of issues reported against the products we produce. It has nothing to do with a developer not being "strong". I have complete confidence in our development staff. That doesn't mean that I don't have my team test anything. Sometimes QA just serves as an intermediary between development and the customer to say, "Yes, this looks good." Maybe sometimes it seems foolish to have QA when a dev team is churning out such amazing code. However, I have never come across a dev team that didn't appreciate QA at some point for stopping a stupid error.
--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------
What is Holistic Testing?
|
Previous
Index
Next
Threaded
Edit
Reply
Quote
