SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 15
  1. #1
    Senior Member
    Join Date
    Jan 2001
    Location
    Pune, Maharastra, India
    Posts
    237
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    A Fault That Could not be reproduced

    Whenever I find a fault, these days, at least I keep a log of how I found it. The actions I performed, events fired etc. Though it has increased my testing time.

    But Sometimes, When I go to developer and bring him to my desk to show him the fault, I am surprised to see that everything is OK.

    Developer laughs and says, I know it is perfectly working , you may have taken a wrong code?

    I feel upset, After having spent so much of effort If I do not a result which I should show at the appropriate moment -- What is the use ...

    How you feel? How do I recover from this?
    Any guidelines?

    Govind/Pune/India
    Govind_b_kul@hotmail.com

    ------------------

  2. #2
    Senior Member
    Join Date
    Mar 2001
    Location
    UK
    Posts
    484
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Great! it works on his machine!

    That's OK if he's going to be the only user.

    In my mind, delivering the 'correct' system (i.e the correct bits of code correctly compiled and correctly installed onto a correctly configured version of the correct sort of machine) into TESTING is a way of practicing delivering it to the customer.

    If he can't correctly deliver a version to you then how can he hope to deliver the correct version to the customer?

    Your situation can lead to many process improvement opportunities!

    What about the build procedure?
    What about configuration management?

    ------------------
    Softly, Softly catchee monkey...
    Softly, Softly catchee monkey...

  3. #3
    Senior Member
    Join Date
    Aug 2001
    Location
    Atlanta, GA
    Posts
    2,693
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Govind,

    When I have these situations, the first thing I determine is if the developer and I are truly accessing the same environment/version of the application. In many cases, developers run their own instances of the application locally, and it can be very different than the instances being tested. Even the development environment will be very different. Be sure that he's accessing the same *exact* environment. In some cases, even running a different version of the JDK can cause differences in such results.

    When that's confirmed, look at environmental issues. The machine configuration, software, etc. can all contribute to defects as well. Try testing on multiple machines before going to the development team. It's never a bad idea to double-check yourself when you find defects. In some places I've been, we each had two computers on our desk for just this purpose. If you have a lab, even better!

    If it happens often, consider emailing your defect to another member of the QA team, and see if *they* can reproduce the defect in the way you've written it up. Perhaps even with writing down steps, you miss something important, like whether you clicked with the mouse or hit <enter> on the keyboard. In this way, you'll be able to isolate things more before going to development, and hopefully help isolate the defect more at the same time.

    ------------------
    Destiny is not a matter of chance, it is a matter of choice; it is not a thing to be waited for, it is a thing to be achieved..."
    Annemarie Martin
    Secretary
    Association for Software Testing

  4. #4
    Junior Member
    Join Date
    Oct 2001
    Location
    UK
    Posts
    12
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    What is it with some developers?

    I've got one here who won't believe a defect occurred unless you can reproduce it, on demand, in front of him. We've spent hours recreating some of the more obscure ones.

    In general we record ALL defects, along with whether (or not) we could reproduce them. But, that could just be the paranoia created by working in e-commerce

    We cetainly wouldn't want to be on the receiving end (of the resultant flak) when a defect is revealed by a customer - specially if it ended up in the press



    ------------------

  5. #5
    Senior Member
    Join Date
    Dec 2000
    Location
    Pune, Maharashtra, India
    Posts
    1,037
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Perhaps if you instrumented the code under test with logging facilities, that might help matters a great deal.

    I mean ::OutputDebugString() calls in the code. Keep DBWin32 or some such utility running on the server/client side that can be saved.

    If I were a developer, I'd demand a fully reproducible bug in the absence of such a log.

    Consider case #1: Bug occurs due to incorrect logic implemented in code
    This bug can be reproduced with some effort - effort will be more if the bug is in some obscure part where execution goes only occasionally.

    Case #2: The problem is not in the code itself but the test data / data being handled by the code at the time of failure.
    The only way this will ever be tracked is if the identical data is used [which can sometimes be a problem] or if comprehensive logging is enabled. Partial logging may not record it.

    My point: Logging and saving log files saves heartburn and increases testability.

    ------------------
    - Suresh
    ["Carpe diem, quam minimum credula postero!" -Horace]
    Suresh Nageswaran, CQA, CSTE
    Lead Consultant
    Kanbay Inc.
    Pune, India.
    sureshnageswaran@yahoo.com

  6. #6
    Member
    Join Date
    Feb 2001
    Posts
    31
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Govind,

    I agree with QAGirl.

    This kind of situation comes several times during our testing process.

    If I am unable to reproduce a particular defect then I've to figure, what steps did I take to make this happen and when I am sure that those are the correct steps, I email them to the developer with the details about the problem and we go from there.

    And as QAGirl mentioned, we should always try to reproduce the defect in different environments.

    Rashmi-

    ------------------
    Rashmi-

  7. #7
    Senior Member
    Join Date
    May 2001
    Location
    München, Germany
    Posts
    171
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    I think it is important to reproduce defects caused by behaviour of the user. It is too easy to fall into the trap that you think you did something different from what you actually did.

    In cases where I can not reproduce it, I provide as many details as possible: log files, environment, screen shots, etc... I put a potential cause of the defect and then move on. I don't expect much effort to be spent on these -- a cost/benefit trade-off usually sets these at a lower priority*.

    The operations of computer systems, while for the most part deterministic, are very complex and unless you go to extreme measures you always have an amount of pseudo-randomness in your environment (especially if the terminal is interactive).

    Unfortunately, many software defects can be attributed to this randomness -- many are preventable, but some others aren't. Reproducible defects are easier to manager because they are easier to isolate and likely attributed to a source code defect. The ones that can't be reproduced are sometimes related to bugs in the operating system, small hardware faults, or other conditions, that even if you could determine, you may not be able to fix -- often workarounds can be put in place, but these are not always trivial.


    ------------------
    &lt;edA-qa@disemia.com&gt;
    Quality Assurance
    --
    edA-qa@disemia.com
    TestPlan - Superior Web Application Automation & Testing

  8. #8
    Member
    Join Date
    Dec 2001
    Location
    Manchester, United Kingdom
    Posts
    35
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Recently I came across the exact same situation, but when I checked his PC he was developing in Internet Explorer 6.0 when we was testing what the clients would use, which was 5.5. Sure enough when he went to IE 5.5 the problem showed up on his machine.

    Annoyingly after fixing the bug, he went back to developing in IE 6. I raised it as a configuration management issue, but no-one seems to care.

    The best example though happened a few years ago on a multi-million pound project where the system just would not work on any other machine apart from the developers. The cause, he had hardcoded it in some strange way so that the PC had to have a /user/temp/Bob directory, otherwise it wouldn't work. It couldn't be fixed at that stage without costing a fortune, so the installation documents were changed to say that the system required a /user/temp/bob directory!

    The moral of this story...developer's PCs and the environment they work in are often different to what the expected user will have.


    ------------------

  9. #9
    Senior Member
    Join Date
    Feb 2001
    Location
    Colorado Springs, CO, USA
    Posts
    864
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by Govind:
    When I go to developer and bring him to my desk to show him the fault, I am surprised to see that everything is OK.
    <HR></BLOCKQUOTE>

    I think this question is specifically about bugs that are not reproducible on your own machine, not about configuration problems at developers' desks. Like many people have posted:

    1) Verify that you can reproduce the bug. If you can, try to isolate the variable or combination of variables that cause the defect. If you can do this, you can ensure those variables are present to demonstrate the defect.

    2) Have other testers check for the defect on their machine to ensure the problem isn't with your client configuration, user profile/authorities, or other variables specific to just you.

    [This message has been edited by testgeek (edited 02-14-2002).]
    Thanks,
    Tim Van Tongeren

  10. #10
    Junior Member
    Join Date
    Nov 2001
    Posts
    26
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: A Fault That Could not be reproduced

    Sometimes bugs can't be reproduced on demand, but show up often enough that it's certain they exist. This weems to be particularly true of large server apps written in java or J2EE. At my last position, it wasn't so much that the developers wouldn't beleive you when a bug couldn't be reproduced on demand, as they dispaired of trying to debug it.

    ------------------

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.40 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 7.50%
vBulletin Optimisation provided by vB Optimise v2.7.1 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.3.0 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 10:52 AM.

Copyright BetaSoft Inc.