The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Miscellaneous Forums >> General Discussion

Pages: 1 | 2 | >> (show all)
PalsPP
Newbie


Reged: 11/09/06
Posts: 8
Appraisal for a Test Engineer
      #340267 - 11/09/06 03:35 AM

Hi Guys,
I recently joined as a Test Manager where I have a team of 4-5 people and going forward will have around 10-15 team members. I have a prior experience at INTEL Tech as Validation Architect. I have to do appraisal for my team members, and hence I need help from you people as to what all parameters should be considered while doing an appraisal for a Manual/Automated Test Enggineer. Basically I want to comeup with a standard set of parameters & figures against them to calculate the efficiency/Performance of a test engineer.if somebody already has standard template, it will be of great help

Pals.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter Ruscoe
Veteran


Reged: 03/18/02
Posts: 7686
Loc: Tampa Bay
Re: Appraisal for a Test Engineer [Re: PalsPP]
      #340303 - 11/09/06 05:55 AM

Oh boy. If you are trying to come up with a mathematical way to justify performance appraisals, then you are headed down a VERY dangerous path.

I run a dept of 8 in-house testers and a number of off-site contractors. I was new to the job 15 months ago, and had to perform appraisals with only 7 months under my belt.

It took me about 5 days flat to figure out who was pulling their weight and who wasn't. No one was bad enough to let go, but the differences were pretty clear.

I think you would be better addressing this question to the HR Director of your comapny, to see what methodology (if any) is used elsewhere in your company.

With all that said, for all I know, the use of "magical formulae" to assess people might be common in India.

("standard template" - groan)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: Appraisal for a Test Engineer [Re: Peter Ruscoe]
      #340330 - 11/09/06 06:48 AM

Pals,

When you were a Validation Architect, what parameters and figures were considered durng your appraisal by your manager?
Do they apply to your team as well?

Do you have a Human Resources group that could help you in understanding if your company has a standard appraisal process?

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Rich W.Moderator
Veteran


Reged: 03/05/04
Posts: 5815
Loc: West Coast of the East Coast!
Re: Appraisal for a Test Engineer [Re: Joe Strazzere]
      #340353 - 11/09/06 07:42 AM

We use the following:

Managing Employees:
Coaches less experienced teammates
Creates documentation to aid in others' learning
If applicable, provides supervision and ensures quality and timeliness of others' deliverables

Managing Work:
Understands scope, priority and timelines for assigned work
Manages workload to deliver quality deliverables that meet requirement
Communicates status of assignments to supervisor
Notifies supervisor of issues that impact productivity or ability to meet expectations
Adheres to standard processes in completing assignments
Follows standards/uses standard toolsets
Assists Manager in tracking progress, prioritizing work, developing time estimates and work plans and assisting less experienced team members
Resolves issues or escalates to Manager, if necessary
Identifies opportunities to improve existing conditions and processes & implements solutions
Creative in problem solving
Does not miss important items
Does not need to be told the same thing multiple times
Checks work in to central repositories

Additional Responsibilities:

Customer Focus:
Maintains strong relationships with customers
Understands and strives to meet customer needs

Team Orientation:
Works well with others
Assists less experienced team members

Leadership:
A. Sets a good example for less experienced team members
B. Provides direction
C. Proposes and implements change
D. Is held accountable for all actions/work
E. Displays initiative
F. Demonstrates persistence until goals are met
Applies problem solving skills to overcome obstacles
Attempts to solve problems, yet asks for help before productivity is impacted
Adapts to given situations
Understands and works effectively with ambiguity

Communication Skills:
G. Excellent verbal and written communication skills
H. Strong listening skills
I. Translates technical details into descriptions the customer can understand
J. Illustrates concepts clearly
K. Adjusts content of written/verbal communication to the audience
Written communications are succinct, clear and complete

Personal Qualities:
Displays professionalism, honesty, integrity, self confidence, maturity, respect and conviction
Maintains a positive attitude

Self Development:
Accepts and addresses constructive feedback
Pursues practices to improve knowledge and skills
Keeps skills updated and asks for coaching or formal training, if necessary.


Guidelines for Project Commitments:
15%- On time delivery (IT Project manager/owner rating)
70%- Personal contribution (IT Project manager/owner rating)
15%- Customer satisfaction (project sponsor rating)

Then we list all projects and evaluate on each.

--------------------
Personal Comment

Success is the ability to go from one failure to another with no loss of enthusiasm.
~ Winston Churchill ~


...Rich Wagner


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter Ruscoe
Veteran


Reged: 03/18/02
Posts: 7686
Loc: Tampa Bay
Re: Appraisal for a Test Engineer [Re: Rich W.]
      #340359 - 11/09/06 07:57 AM

I'm surprised that isn't proprietary, Rich. It's nice to have though

Post Extras: Print Post   Remind Me!   Notify Moderator  
Sherilyn
Member


Reged: 05/06/03
Posts: 99
Loc: New Zealand
Re: Appraisal for a Test Engineer [Re: Peter Ruscoe]
      #340455 - 11/09/06 03:16 PM

these guys must have a job description rolling around somewhere...start with that as a base, talk to the people they work with, look at their testcases, bugs raised...bugs they missed etc etc...

Post Extras: Print Post   Remind Me!   Notify Moderator  
lynneM - RIP
Moderator


Reged: 02/02/03
Posts: 3101
Loc: FL, USA
Re: Appraisal for a Test Engineer [Re: Sherilyn]
      #340457 - 11/09/06 03:29 PM

In most companies I have worked in I have had to keep within guidelines provided by HR for performance reviews and job descriptions. Also we ensure that way in advance of performance reviews that the staff know what is expected of them so that they can work to that. You may want to see what the previous manager had in place, and when you change it ensure that the staff are aware that the "rules" have changed.

--------------------
Lynne

I have not failed. I've just found 10,000 ways that won't work" --Thomas Edison


Post Extras: Print Post   Remind Me!   Notify Moderator  
PalsPP
Newbie


Reged: 11/09/06
Posts: 8
Re: Appraisal for a Test Engineer [Re: lynneM - RIP]
      #340481 - 11/09/06 09:12 PM

Hey thanks buddies,
That was really good input, let them keep coming, don't stop. I would have applied Intel's parameter here, but this company is little different.My HR doesnt have any rules, infact QA being a new team, she has left it to me to decide.

M waiting for more replies :-)


Post Extras: Print Post   Remind Me!   Notify Moderator  
watkis4
stranger


Reged: 08/16/06
Posts: 100
Loc: England
Re: Appraisal for a Test Engineer [Re: PalsPP]
      #340508 - 11/10/06 01:19 AM

Our appraisals are based on 12 key competencies that apply to everyone in the company, things like Reliability, Business Knowledge, Self Development, etc. Its the company's way of trying to baseline everyone. However this can be a very subjective, so if you get on with your line manager, you may get better results than if you don't. The latest thing we have been asked to look at is if there is a more objective way of measuring our team. Things like if tester A raises more defects than tester B, does he/she get a higher rating? Its a bit of a minefield as we know that this isn't the case, as our projects can vary so much. We are still in debate about how to use this type of metrics/measurements.

Susan


Post Extras: Print Post   Remind Me!   Notify Moderator  
PalsPP
Newbie


Reged: 11/09/06
Posts: 8
Re: Appraisal for a Test Engineer [Re: watkis4]
      #340517 - 11/10/06 02:20 AM

Thanks Susan.I have comeup with few parameters, not all are applicable to all members, it goes like this

Writing Good Testcases
Programming/Automation Knowledge
Defects raised/Discovered
Project architechture knowledge
rootcause capability
Think beyond assigned task
Assume responsibilty
Good team player
Communication Skills

Comments???


Post Extras: Print Post   Remind Me!   Notify Moderator  
Rich W.Moderator
Veteran


Reged: 03/05/04
Posts: 5815
Loc: West Coast of the East Coast!
Re: Appraisal for a Test Engineer [Re: PalsPP]
      #340526 - 11/10/06 03:39 AM

Quote:

Writing Good Testcases
Programming/Automation Knowledge
Defects raised/Discovered
Project architechture knowledge
rootcause capability
Think beyond assigned task
Assume responsibilty
Good team player
Communication Skills




Test case writing is very subjective and unless you establish a standard then how will you evaluate what is good & bad?

Knowledge is also difficult to evaluate, but applied skills can be subjectively evaluated if you spend a lot of time observing, micro-managing that is.

Never grade on number of defects, this is a no-no. You can do a search and see where your topic has been discussed on this forum a number of times.

Is it the job of QA, QC to do the root cause analysis?

Thinking outside the box, as we call it, can be measured to some extent, but be sure everyone knows they are being observed for it. Many people have that ability but do not use it because they are not properly challenged.

The rest are usable. Just use them with understanding.

--------------------
Personal Comment

Success is the ability to go from one failure to another with no loss of enthusiasm.
~ Winston Churchill ~


...Rich Wagner


Post Extras: Print Post   Remind Me!   Notify Moderator  
Myrtle
Awsome Member


Reged: 06/12/06
Posts: 662
Loc: Spain
Re: Appraisal for a Test Engineer [Re: Rich W.]
      #340532 - 11/10/06 04:10 AM

It's a good idea to have a public list of expectations so that your staff know what they need to achieve to progress. For example
Test Automation skills
Level 1 - able to record and playback
Level 2 - able to amend scripts that already exist using scripting language
Level 3 - using scripting language, able to write simple scripts which don't access any external files from scratch
Level 4 - able to read data from external files for input to scripts and to write results to external files
Level 5 - able to manipulate and interrogate database values within a script
Level 6 - able to write complex scripts from scratch and to interpret results

These are just suggestions so that there is a progression path in place.


Post Extras: Print Post   Remind Me!   Notify Moderator  
PalsPP
Newbie


Reged: 11/09/06
Posts: 8
Re: Appraisal for a Test Engineer [Re: Myrtle]
      #340614 - 11/10/06 09:57 AM

This is a real exciting forum and I am loving it!!!I never thought it will work this way. Thanks alot guys for your responses. If anyone of you have done appraisals, please share your experiences here, may be end of this discussion would lead to a standard template :-)

Post Extras: Print Post   Remind Me!   Notify Moderator  
prtester
Active Member


Reged: 05/09/06
Posts: 1059
Loc: Denver, CO
Re: Appraisal for a Test Engineer [Re: Myrtle]
      #340617 - 11/10/06 10:06 AM

I feel that good appraisals come from good management.

With a new team in the past, I've tried to get the big picture first. I looked at all defects closed as not a defect for the entire team. I got to see mathematically just who was responsible for the most of these, then drilled down to see what the misunderstanding was in each case.

I then compared this to the number of defects logged by each team member, comparing the team as a whole and comparing that to the modules that produced the defects and the people who worked on those modules. Some parts of the development are buggier than others, and finding more defects is only part of the picture.

Then, of course, I looked at severity. Who on the team is sniping the big birds, and who is spending all their energy picking nits? Is nit picking a bad thing, or is it necessary work that some people avoid because it's less juicy.

I looked at it as a manager, as the one ultimately responsible for the quality of the output and the ongoing quality of the team. I looked for who needs training, who needs encouragement and who deserves recognition (and can stand as an example to follow). I also looked for the deadweight. Are they salvageable, or does the team need trimming?

When I was done, I had my evaluation, and the metrics to back it up. I didn't just consider the defects, although mathematically that is the low hanging fruit. I also looked at the other output, test cases and requirements reviews. But I took the attitude that I take to an AUT. I need to give a fair, clear and honest evaluation and I need to be able to back my conclusions.

One of the best things that you can do is lay the groundwork for next year with a clear communication on how the team is evaluated, then let them do their best.

tester, sometimes a manager.

--------------------
see my blog: www.sqablogs.com/rainbowtesting


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter Ruscoe
Veteran


Reged: 03/18/02
Posts: 7686
Loc: Tampa Bay
Re: Appraisal for a Test Engineer [Re: prtester]
      #340620 - 11/10/06 10:30 AM

Quote:

may be end of this discussion would lead to a standard template



Heaven forbid.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Yury
Advanced Member


Reged: 12/31/99
Posts: 649
Loc: Toronto
Re: Appraisal for a Test Engineer [Re: Myrtle]
      #340667 - 11/10/06 08:22 PM

Quote:

For example Test Automation skills
Level 1 - able to record and playback
Level 2 - able to amend scripts that already exist using scripting language
Level 3 - using scripting language, able to write simple scripts which don't access any external files from scratch
Level 4 - able to read data from external files for input to scripts and to write results to external files
Level 5 - able to manipulate and interrogate database values within a script
Level 6 - able to write complex scripts from scratch and to interpret results


Myrtle, why do you want to have test automation specialists with skills levels 1 through 6?
What would you do with them?

I hope you are not planning to get them involved in test automation.


Post Extras: Print Post   Remind Me!   Notify Moderator  
0fer
Member


Reged: 09/23/06
Posts: 162
Re: Appraisal for a Test Engineer [Re: PalsPP]
      #340669 - 11/10/06 10:10 PM

Quote:

Test Automation skills
Level 1 - able to record and playback
Level 2 - able to amend scripts that already exist using scripting language
Level 3 - using scripting language, able to write simple scripts which don't access any external files from scratch
Level 4 - able to read data from external files for input to scripts and to write results to external files
Level 5 - able to manipulate and interrogate database values within a script
Level 6 - able to write complex scripts from scratch and to interpret results




Here is a good example of the problems that you are facing by trying to create a template for evaluation of people. My way of evaluating an automation engineer is totally different. I dont measure the number or complexity of commands in a scripting language she/he knows by heart for instance (although I agree that it can be a calculable indicator, but evaluation is not only about counting). Just as an example I will mention few of the things that I will look for to evaluate an automation engineer (without any order, and I dont cover all my important things here) like: reuse of code, modularization, generate libraries or engines creation, code readable and use of comments and documentation, no hardcoding of names/values/coordinates, smart choosing of automation targets (cost, state of the feature, etc ), coverage, data verification, robustness (run smoothly every night after a successful build without supervision), logging, and so on. How are you going to decide which way of evaluation is better for your organization? Maybe you should first find an automation lead and let her/him evaluate the automation engineers in your team. And again, you will have to evaluate the lead as well, and I really dont think that you can (unless you are a guru scripter) evaluate your automation lead according to the scripting ladder that was suggested above and I dont think that it is desirable even if you can. But again, I present here only one possible way of appraisal.

Ofer


Post Extras: Print Post   Remind Me!   Notify Moderator  
PalsPP
Newbie


Reged: 11/09/06
Posts: 8
Re: Appraisal for a Test Engineer [Re: 0fer]
      #340751 - 11/12/06 08:09 PM

Thanks alot to all of you, I guess I got some way to head. I will keep these points in mind while doing appraisal.

Thanks Again


Post Extras: Print Post   Remind Me!   Notify Moderator  
Myrtle
Awsome Member


Reged: 06/12/06
Posts: 662
Loc: Spain
Re: Appraisal for a Test Engineer [Re: Yury]
      #340787 - 11/13/06 12:56 AM

hi yuri,
I wouldn't expect to have someone on each level of this scale but, if someone is new to automation and can manage Level 1, they would know that career progression can be achieved when (as) they achieve greater expertise as specified in the other levels. This makes it easy to see what to aim for. This was just an illustration.


Post Extras: Print Post   Remind Me!   Notify Moderator  
ljeanwilkin
Moderator


Reged: 01/13/03
Posts: 1182
Re: Appraisal for a Test Engineer [Re: Myrtle]
      #340968 - 11/13/06 12:47 PM

I'm was kind of horrified by your list, Myrtle, but feel better now that you said it was just an illustration.

If a candidate told my automation staff they were able to do record and playback, I don't believe I could trust them not to snicker...

- Linda


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | >> (show all)



Extra information
0 registered and 24 anonymous users are browsing this forum.

Moderator:  Rich W., AJ, blueinatl 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 46401

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5