Measuring QA productivity
I hope I don't ask a dumbo question here; but I'm new to the management part of QA things...
Basically, I'm a one (wo)man dedicated QA team but some other folks are supposed to be testing as well and I'd have to monitor and coach those folks, next to myself [img]/images/graemlins/smile.gif[/img] ...
Anyhow... I would like to improve the QA services and document this process accordingly...
I already got some crossreferences when reading the topic about measuring the developer productivity - but well, that doesn't really suit here in all the cases...
My ideas would be to check:
- bugs found rate (and in that aspect, dig down to the areas to see where most bugs were found)
- after go-live issue rate (on things that were tested and were not)
- customer satisfaction (done via measurable lessons learned activities and surveys)
- time spent on testing (whereas that is pretty tough to judge as it includes "time wasted on waiting for fix"
- quality of deliverables and documentation (i.e. enough information in QA strategy / plan / scripts - if ever present...)
Do you get any other ideas in mind?
Thanks a lot,
Re: Measuring QA productivity
Do a search on Metrics & KPIs as I know we've talked about this before and that may give you some info.
From my standpoint I would not measure bugs found as some higher ups may take this as "QA isn't doing their job" if the bugs found rate is low... it could just be that you have exceptionally clean code.
One of the things I measured that you don't have here is number of new test cases added per period of time. What we wanted to show was that QA has tasks outside of testing such as increasing our test cases (manual and/or automated) and we wanted to show that we were increasing our documentation (some folks had a false perception that if we weren't testing we must be sitting on are butts doing nothing so we also started documenting where we had been in requirements reviews, design reviews, code reviews, prototype sessions, etc.).
Sorry to ramble. My only caution is be careful what may be portrayed by what you measure and be very clear in what you are presenting and to whom. When I did the metrics I published what the metric was/what it was measuring/what the target rate was, etc so as to be very clear.
Re: Measuring QA productivity
Thanks a lot for your feedback, blueinatl
"KPI" was the thing I didn't search for and boom that thing gave me quite some ideas - sorry about that...
Agreed that I need to be careful on how I present results - not least because it's mainly going to end up on my lonesome shoulders. I want to improve myself as well as I know there is some space to improve, but not with a self-slap into my face...
In any case this is most likely a long term task and I have some time to prepare, think about it and find the best way to present whatever the outcome might be.
I guess this year it's going to be more about defining KPIs then about measuring QA productivity anyhow; considering the fact it was not yet done before in our organization on QA dept. level.
I guess I fight my way through, as usual... [img]/images/graemlins/wink.gif[/img]
Thanks again for your help and hints,