| || |
Developers role in system testing
I've been lurking on this forum for a while now and would like to throw up a few topics for discussion. I'm currently in charge of system testing for our product, which is a multiple component "system" rather than just a single piece of software. As well as developing a test plan and performing manual testing, I'm also coordinating the development of an automated framework that we can use to stress our system and will also be used to give us a degree of confidence in the system before we start manual testing. We have a development team of around 20 people and there are four of us specifically earmarked for the system test team.
The first issue I'm interested in is that of developers writing their own tests. I did a search and found a thread about this from around December last year. The general consensus seemed to be that this wasn't a good idea, however the management here has decreed that feature development or bug fixing of the system can't be marked as complete until the developer has written a test for their work (they should already be writing unit tests). Ideally we would incorporate their test into our test plan, but I see it more as a starting point for the system test team to write a proper test. I have provided a template for this, written detailed instructions and put aside a place in the source repository for these tests to be put. However, despite my efforts, the uptake of this regime has been somewhat slow. Has anyone else been involved in this kind of thing? Any thoughts about how to best inforce this kind of regime?
The second issue is that we are trying to institute a policy whereby all developers are rotated through the system test world. The reasoning behind this is that it will help the developers get to see the system functioning as a whole (as opposed to just seeing their component in isolation) and will hopefully spur them along to producing better quality software. From my point of view, this seems to be a big headache as I will need to train up each person in how to install and configure the system, and then have some confidence that they're actually capable of executing the tests properly. Hopefully they will also be able to contribute something to system testing, but I sometimes get the feeling that this is seen to be something of a side-effect :-) I need to come up with a plan of how to best structure this rotation process. Some of the questsions I need to answer are:
1) How long should they stay in my team? Too short and the training overhead is too significant. Too long and I think they'll get bored and be counterproductive.
2) What will they do while they're in my team? Will they write test specs and help develop automated tests, or will I use them as lab monkeys to manually run through my checklists?
Once again, has anyone else been involved in a situation where the development team is rotated through system testing? What are people's thoughts on the matter? Any ideas or experiences would be most welcome.
Re: Developers role in system testing
I have been in a situation (some years ago) when the company I was in determined that rotation of developers into test was a good thing. As a new and inexperienced QA Manager, I agreed. Here started some of the worst months of my working life.
- Developers don't want to be System testers, they are forced into it, do not see it as career enhancing, suffer it for the period of their rotation and heave a big sigh of relief when it is over.
- My experienced testers and I had to spend time teaching them what to do, so that diluted our efforts and effectiveness
- The developers were pretty hopeless at testing, in general, and we had to redo some of their work
- It did not improve the developer's quality. When they went back to developing they were just glad it was all over and they could get back to hacking, err.. coding.
- Because the developers were not interested, they knew they were only with us for a short time, they did not put the effort in - when weekend working or late working was on the agenda they were often "washing their hair"
- The best developers were not put on the rotation, as they were "too valuable" to development, so this caused a two class system within development - those who were too good to go to test and those who were not.
After 6 months, the whole system was abandoned. No-one was getting any benefit
Re: Developers role in system testing
Asking developers to tear apart their own or a collegue's work is counter-productive. Let them do what they are good at (and what management is paying them really good money to do). Forcing them to do a stinit in testing will only cause them to develop attitude problems and you'll probably lose some of the better ones to other companies.
I also think it is counter-productive to require the developers to produce system test plans. First, as stated above, they aren't good at testing and won't think of the really good test cases. Second, even if you just use these as a starting point, they may allow your test plan designers to slack off, take the easy way out, and also not think of the really good test cases.
It is *great* that you require developers to document their unit test plans. But even those have no benefit for your system test team and they shouldn't be published outside the development team. Why discourage that practice by opening them up for criticism by people who are better at test design?