| || |
JIT Testing - Part Three
As part of the on-going saga of implementing Just-In-Time testing techniques, I'd like to pass on the results of our first two "real" sessions. We chose two "small" last-minute projects. We invited key developers, the PM, DBAs, and the BA.
We found the sessions consistently drive out roughly 40 test requirements in an hour. Using Darrel's suggestion for marking each requirement with a priority sticker worked like a charm. One team was totally into it; it was really impressive. The BAs have asked us if we're willing to do this "earlier in the process" - I practically wept openly at the invitation. In one meeting, we had the 3rd party vendor present, and they've asked for additional information, as they have NO QA and they would like to implement it in their shop. I'm going to drag out Rob Sabourin's information and pass it on.
Part of my concern was doing a good enough job to do justice to the techniques; I would hate to have something with potential benefit fail due to my own inadequacies; I am not as Dynamic as Rob (although I love to teach) and did not play Twister as I was proving a point (well, for one thing, I was wearing a skirt...).
I'm going to assume at this point that the training went well, as we ended up generating 140 test requirements, where we previously had 0. We have major large projects that have requested we try this technique for individual pieces of functionality to provide the initial test base. The developers like it because they have input and understand what will/needs to be tested up front, although they understand we will add to the test base.
Some group members had more difficulty with the open flexibility and contributory nature of the exercise than others. We have some staff that have difficulty working outside a very structured, methodical process. It was quite interesting - it had nothing to do with age. Some our younger members (20s) had more problems than the the oldest members (50s). It all had to do with outlook. By the end of the meeting, however, everyone had contributed something and several were real surprises. I'm ready to adopt several developers, a DBA, and a BA into the QA group - their tests were that good (and creative). Those people that liked it were in the majority and several loved it and are bringing the others into it with their enthusiasm.
We also drove out several major holes in the functionality through this exercise; we had developers looking at each other and saying "I don't know what will happen with that....". In one case, the developer shamefacedly admitted that the system crashes "if you try that".
We took the pile of requirements and made a hierarchical list of what needed to be tested according to priority and copied the entire group. They can add to or delete from the list. We are entering the list into TestDirector (requirements tab) to provide traceability and meet SOX requirements. Overall, the process took a total of 3 hours; that's meetings and copying the group on the list and entering them into TD. Normally, we'd spend several weeks on requirements and tracking people down to ask questions.
Our next step is the actual testing, and my last blurb on this will be how the testing actually worked out, what problems were missed in production - in other words, was coverage adequate?
Due to my own cautious nature, I'd say we have a really good start and I'm very hopeful about the outcome...
Re: JIT Testing - Part Three
Just for the rest of us that are coming in late [img]images/icons/smile.gif[/img]
Thanks for the continued sharing! I've worked at an employer where we tried a close representation of this, failed ultimately and totally due to the lack of flexibility in the pool of resources involved.
Waiting in anticipation of the final chapter [img]images/icons/wink.gif[/img]
Insanity: doing the same thing over and over again and expecting different results
Re: JIT Testing - Part Three
I'm glad I could contribute in a small way to the success so far. When I was involved in the "dot" excercise, I saw exactly what you did. There was a lot of give/take and interaction among the players. Some were essentially lobbying for their item to be ranked #1 and did a good job of explaining why. The interaction and byplay were the keys. Like your session, it ended up really being fun, which is always a good benefit when you are at work.