xuel
Member
Reged: 09/08/02
Posts: 56
Loc: Shanghai,Shanghai,China
|
|
Our company just moved to Scrum Agile process this year. The development managers ask test team to stop using defect tracking system. They hope testers to report the defect to DEV face to face. I know Scrum Agile process is lightweight-process. But is there any necessary measurement we should keep in Scrum Agile?
|
KishoreQATestEngineer
Advanced Member
Reged: 10/08/06
Posts: 613
|
|
Basic intention of the Scrum model need remove the more doucment and deliver the product as early as possible instead of other model. so there is no question of again maintain metics here...
Regards,
Kishore
|
Walen
Active Member
Reged: 05/09/01
Posts: 1064
|
|
Not tracking defect progress in ANY methodology is a certainty to ensure disaster.
Not tracking measurement may mean they have no questions about the state of the product.
--------------------
P. Walen
|
michaeljf
Veteran
Reged: 09/17/01
Posts: 3527
Loc: Yankee Land
|
|
Well, I'd be ok with reporting bugs as tasks in the backlog, otherwise they'd be lost and this way you know when the work is done and by whom. Not reporting them at all is more of a danger sign, you want them somewhere, even if its just the backlog.
--------------------
- M
Nothing learns better than experience.
"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown
Now wasting blog space at QAForums Blogs - The Lookout
|
brentpaine
Veteran
Reged: 03/09/07
Posts: 3754
Loc: Waterloo, Ontario, Canada
|
|
I agree with Walen and Michael, there are so many more reasons to use bug tracking systems then using them as a metric. There is making sure that duplicate issues are not logged. You could wind up with 3 developers working on the same issue.
Also, I don't know about anyone else, but there are times when we're finding 10 or 20 bugs a day at the beginning of a project. Even if these issues were fairly trivial, there is no way the developers could fix them all in a day. So where do they end up? They end up on the backburner and only the most pressing issues are handled. Then, development completely forgets about them and they are never looked at again. I don't like where that's going.
I'd be less worried about Metrics and more worried about this project tanking. If you can't hear the death march, then let me help you out, "You're in a death march!"
--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------
What is Holistic Testing?
|
michaeljf
Veteran
Reged: 09/17/01
Posts: 3527
Loc: Yankee Land
|
|
Quote:
Even if these issues were fairly trivial, there is no way the developers could fix them all in a day.
That's right, even if they are bugs they are still Work that needs to be done in the iteration, someone needs to be assinged as a resource. Not tracking means you are now adding hidden work, and your estimates for the entire iteration are now more than useless.
Good luck with it, and Happy Dragon Boat Festival!
--------------------
- M
Nothing learns better than experience.
"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown
Now wasting blog space at QAForums Blogs - The Lookout
|
GoGreen
Active Member
Reged: 09/01/08
Posts: 1002
Loc: Neverland
|
|
Quote:
Basic intention of the Scrum model need remove the more doucment and deliver the product as early as possible instead of other model. so there is no question of again maintain metics here...
Regards,
Kishore
I dont agree.
--------------------
Regards,
Tinker
"My own business always bores me to death. I prefer other people's".
Cecil Graham, Act III by Oscar Wilde.
|
xuel
Member
Reged: 09/08/02
Posts: 56
Loc: Shanghai,Shanghai,China
|
|
Thanks all for your help! And I think we have got enough reasons for keeping using the defect tracking system. Let's go back to the question of the metrics in Agile. Based on your experience, is there any good or necessary metrics used in Scrum Agile? Thanks.
|
brentpaine
Veteran
Reged: 03/09/07
Posts: 3754
Loc: Waterloo, Ontario, Canada
|
|
Well, as far as metrics go, you are probably already using some method of estimating project work (whether it be hours or story points). If so, then you can use this as a basis for any other metrics that you'd like to see.
However, I'm not a huge fan on metrics, overall. Metrics have been used for decades now to measure every aspect of a software project, but in my time I have yet to see one which does lie and I have yet to see one which actually helps deliver a project on time.
At any rate, I find that the best metrics I have worked with are based on historical data. Remember the quote, Quote:
Those who cannot learn from history are doomed to repeat it.
This quote is so true. As you become more and more familiar with a particular project you gain valuable insight into how to prevent future instances of the same problems.
I don't know why we choose to believe numbers when we have knowledge experts right in front of us. I don't know how many times I've heard SQAs say "I know! I've been saying that for months!" Or "I could have told you it was not going to meet schedule." Metrics cloud what's right in front of our eyes!!! Unfortunately some management doesn't recognize this, and they're living in a dream world with pixie dust farts and unicorn terds because when a project tanks, there are plenty of SQAs who can tell you why and can tell you why you should have just listened to them instead of searching for numbers.
--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------
What is Holistic Testing?
|
Walen
Active Member
Reged: 05/09/01
Posts: 1064
|
|
Generally, I agree with Brent on this. I have no use for metrics in general.
I'd suggest starting with some fairly fundamental questions. What is it that the management team will want to know about the project or the state of the project - not "metrics", not the "number of bugs found per test cases executed", but questions. "What is the state of project Z? Is it on schedule? Will we be able to implement/deploy/roll-out/ship the product on time?"
The answers to these questions may lead to other questions. They may require some consideration on the part of the test lead/manager. That consideration, indeed, the process of obtaining answers, usually to the second set of questions in my experience, will give you the first level measurements you generally need to start with.
Other questions may also be asked by the development management. For example, "What is the state of the team? Are they confident of delivering the product on time? How confident are they that the project will be successfull?" These questions may lead to other measurements.
So, before answering the question about metrics, since I have found that no set-piece collection works for every project, what are the questions that may be of interest to your management team?
--------------------
P. Walen
|
GoGreen
Active Member
Reged: 09/01/08
Posts: 1002
Loc: Neverland
|
|
Quote:
Thanks all for your help! And I think we have got enough reasons for keeping using the defect tracking system. Let's go back to the question of the metrics in Agile. Based on your experience, is there any good or necessary metrics used in Scrum Agile? Thanks.
Yes . metrics is needed.metrics is needed. metrics is needed. metrics is needed. metrics is needed.!!!
You will never know your performance or anything for that matter without measuring. be it agile or waterfall,
one good metric for scrum agile is Burndown Charts. you can google it.
--------------------
Regards,
Tinker
"My own business always bores me to death. I prefer other people's".
Cecil Graham, Act III by Oscar Wilde.
|
brentpaine
Veteran
Reged: 03/09/07
Posts: 3754
Loc: Waterloo, Ontario, Canada
|
|
Burndown Charts are great tools, but it's still not a very good metric, it's more an estimation tool, and it's inherently inaccurate. Again, since it's a tool that estimates your performance based on estimates that you plug into the system it is prone to error.
It's certainly not the worst thing you could be using but, again, it's something that will only improve over time. You'll likely find that until you get good with your story creation and estimation and your team is honest about their story estimates, your burn down charts are going to be completely misleading and inaccurate.
So, again, it comes back to the knowledge experts. With each subsequent scrum and for each release of that product, the people who are managing the scrums will evolve to carry a vast amount of knowledge that will help them to create very accurate, informational, and useful reports. HOWEVER! When they leave, it's not simply hiring someone on to replace them, it's a massive learning curve and knowledge gap. Regardless of how good the last report that was generated was, I guarantee that when that person leaves, the next set of reports will be in left field.
It's not about report quality, it's about people quality, period.
--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------
What is Holistic Testing?
|
Walen
Active Member
Reged: 05/09/01
Posts: 1064
|
|
Quote:
Yes . metrics is needed.metrics is needed. metrics is needed. metrics is needed. metrics is needed.!!!
You will never know your performance or anything for that matter without measuring. be it agile or waterfall,
And having a metric because you need a metric is a waste of time and energy. Without knowing what it is that management wants to know about, any "prepackaged" metrics may well skew their perception.
I'm curious if there are metrics that QA/Testing can measure that can be applied differently because of the methodology in use. There are some non-QA/testing measure I can think of but no testing measures.
The first post said they were going TO Scrum-Agile, and talked about measurements and tracking defects, etc. I'd start by asking if the metrics in place currently provide adequate information to the management team, do they need more? Do they want insight on other aspects of the project?
If management wants to know if severe defects are are being corrected at an "acceptable" rate, You may want to see the average time for Severe defects to be fixed/closed may be. You may also want to track the average of instances of the appearance of new defects in a module/function following a fix to a severe defect.
Ah well.
--------------------
P. Walen
|
brentpaine
Veteran
Reged: 03/09/07
Posts: 3754
Loc: Waterloo, Ontario, Canada
|
|
Quote:
You may also want to track the average of instances of the appearance of new defects in a module/function following a fix to a severe defect.
And depending how finite we wanted to get, we might also want to track this based on the day of the week. I'm sure we'd find more errors occurring on certain days if we let te data run long enough.
--------------------
Brent
--------------------
9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
--------------------
What is Holistic Testing?
|
Previous
Index
Next
Threaded
Edit
Reply
Quote
