SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 15
  1. #1
    Points for Confirmed Friends
    Guest

    Single Fix Rate for Bugs?

    My manager has asked me for the "Single Fix Rate" on the bugs being fixed and regressed.

    What does this mean? I think it means the percentage of the time that a bug was fixed on the first pass of regression testing.

    Am I right?

    ------------------

  2. #2
    Senior Member
    Join Date
    Feb 2001
    Location
    Colorado Springs, CO, USA
    Posts
    864
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    I think you're right - it does sound like any defect that never goes to "Retest failed" or "Reopened" status.

    When I first saw the subject I thought this was going to be a question about a client who wants a single rate for each bug fix - like $20 per bug. Yikes!
    Thanks,
    Tim Van Tongeren

  3. #3
    Senior Member
    Join Date
    Dec 1999
    Location
    Chicago,Illinois,USA
    Posts
    2,537
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    Just to be sure, however, I would simply ask your manager what he meant. "Single Fix Rate" is certainly not a standard term in defect metric circles. So if he meant something different, I would find that out.

    ------------------

  4. #4
    Points for Confirmed Friends
    Guest

    Re: Single Fix Rate for Bugs?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by JeffNyman:
    Just to be sure, however, I would simply ask your manager what he meant. "Single Fix Rate" is certainly not a standard term in defect metric circles. So if he meant something different, I would find that out.
    <HR></BLOCKQUOTE>

    I did just that. He insists it is a standard term, although I know I have never heard of "Single Fix Rate". Anyway, it's exactly what I thought. The percentage of the time that a bug fix from Engineering passes QA's initial regression. He says that this is a very important metric to examine for QA's "Release Planning". He says that another important piece of information for "Release Planning" is the percentage of test cases in QA's test suite that have not yet been executed due to missing functionality in subsequent builds.

    I'm at a stage of a project where all functionality has been delivered from Engineering minus one important feature. So my manager is asking how many QA test cycles are needed to test the system with this last feature added. I've informed him that I need to know more details about the new feature (e.g. what new files are being added, which pre-released files are being changed, what pre-existing functionality would be most affected by the addition of this feature). He didn't understand exactly why I needed this information, but did supply it to me.

    Frankly, I think that this information is more important for my "Release Planning" at this stage of the project than examining "Single Fix Rate", but maybe both are important?



    ------------------

  5. #5
    Senior Member
    Join Date
    Dec 1999
    Location
    Chicago,Illinois,USA
    Posts
    2,537
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by stevew:
    He insists it is a standard term, although I know I have never heard of "Single Fix Rate". Anyway, it's exactly what I thought. The percentage of the time that a bug fix from Engineering passes QA's initial regression. He says that this is a very important metric to examine for QA's "Release Planning".<HR></BLOCKQUOTE>

    Usually this is the reopen rate for a given set of defects. In other words, if an existing defect was in the system and marked as fixed by the developers then, if QA has to reopen the defect, it is marked as a staged metric. In other words, you can measure the ratio of open to re-open defects as well as the ratio of re-open to "Fixed and Closed."

    So I agree with him that it is a metric - just not by the name he gives it. But that is fine: as long as he makes it clear what he wants the name hardly matters.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>He says that another important piece of information for "Release Planning" is the percentage of test cases in QA's test suite that have not yet been executed due to missing functionality in subsequent builds.<HR></BLOCKQUOTE>

    Yep. That can give you an idea of how much testing is not possible at the current stage. But the reason it is not possible is because it is not available. That can become a crucial distinction later on after a release has gone out, potentially with problems. This metric is often associated with the time for test because if those missing features were not added until much later (say, right at the end of the test cycle) and there are problems with those features, then it pays to have metrics that state, at least, partially what some of the circumstances were.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Frankly, I think that this information is more important for my "Release Planning" at this stage of the project than examining "Single Fix Rate", but maybe both are important?<HR></BLOCKQUOTE>

    Yes, they are both important. The "Single Fix Rate" is more to get a handle on how much is going back to development even after it was "fixed." It sounds like your manager is being proactive and making sure that if problems come up, he will be able to cover his posterior by being able to show if a lot of functionality was ping-ponging between QA and development. Also, as you keep using that metric you can start to build up historical data for the next project. Then you can determine, at that time, if the reopen rate is much higher or much lower than the last time. If the defect tickets list the area of functionality, then you can also track metrics for what areas of functionality are reopened (i.e., not fixed correctly the first time around) the most often. This might give you indications of particular trouble areas in the programming or particular trouble areas with the programmers.

    Also, in the case of your overall statements about the new functionality, what your manager wants from you (the test cycles) and what you want from him (the detailed feature spec) are really the same thing. The details how how something is setup will determine, to a certain degree, how it is tested. For example, if the feature has a slew of permutations or combinations, then your test cycles will be much greater. On the other hand, if the missing feature is pretty straightforward, the test cycles might be shorter. (Not I am not talking about "easier" or "harder" here - just shorter or longer.)

    [This message has been edited by JeffNyman (edited 12-22-2001).]

  6. #6
    Member
    Join Date
    Sep 2003
    Location
    Dublin
    Posts
    31
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    I am currently tracking the fix rate on our project. I started doing it cause I felt a lot of time was spend regressing bugs that weren't fixed correctly or properly unit tested. The results of which surprised me somewhat - a mean of 61.3% of bugs regressed pass leaving 38.7% of all bugs coming back to us failing regression.
    Some builds were even down as low as 52% pass rate.

    I was expecting a pass rate of around 70% but feel this is even too low.
    Does anyone have any thoughts on what would be a good figure to reference against should the need arise when the time lines have slipped and the finger is pointing as test, as it generally does. Does 80% - 85% sound about right?

  7. #7
    Senior Member
    Join Date
    Sep 2003
    Posts
    140
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    As far as picking a heuristic value I'd imagine a SFR of less than 85% would be a sign of a project in a little trouble one way or another. That 61% rate seems pretty bad just based on experience. Depending on how many defects there are it will take a few cycles if the project is allowing 1/3 of defects to persist through each cycle. Hmm, quick example, 100 defects:
    1st cycle = 100 left
    2nd cycle = 33
    3rd cycle = 11
    4th cycle = 4
    5th cycle = 1
    This is assuming no new defects are being created, I'd guess projects with low SFR's may also have higher rates of new defect creation?

  8. #8
    Senior Member
    Join Date
    Feb 2003
    Location
    FL, USA
    Posts
    3,646
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    I actually track the Failure Rate of defects so that we know the % of defect fixes that fail retesting. From the Metrics I am tracking most projects are 18% or under is considered OK - the average is around 15% - we have had however failure rates as high as 52% but these were for a short time during a project although the final rate was about 30%.

    This rate can be used with other information to estimate the time it will take to complete testing of a product.
    Lynne

    I have not failed. I've just found 10,000 ways that won't work" --Thomas Edison

  9. #9
    Moderator
    Join Date
    Mar 2004
    Location
    West Coast of the East Coast!
    Posts
    7,756
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    Lynne,
    How do you justify a rate like 52%? Incompetent developers? Testers not fully testing around the bug until it is fixed? Poor error diagnosis by testers? Poor artifact collection or description of defect? All of these could contribute, I just wonder what you have determined with your data collection.

    Thanks.

    Rich
    Personal Comment

    Success is the ability to go from one failure to another with no loss of enthusiasm.
    ~ Winston Churchill ~


    ...Rich Wagner

  10. #10
    SQA Knight
    Join Date
    Aug 2000
    Location
    Elanora Heights, NSW, Australia
    Posts
    3,271
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Single Fix Rate for Bugs?

    Single Fix Rate

    This sounds kind of a useless figure. To complete a typical bug cycle you have:

    Identify
    Analyse
    Assign
    Rectify (re-design ?)
    Unit Test/Module test
    Rergression Test
    Release

    The duration of all these activities will be dependant on the nature of the fault. I have had some bugs take days and weeks to fix, others in hours. The issue I have is that "Single Fix Rate" implies uniformity in bug fixing process, something which I can't agree with.
    Robert Tehve
    rtehve@bigpond.com

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 8.82%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 01:53 AM.

Copyright BetaSoft Inc.