SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Page 1 of 2 12 LastLast
Results 1 to 10 of 17
  1. #1
    Senior Member
    Join Date
    Apr 2007
    Posts
    1,005
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    System downtime = defect?

    Good morning, all!

    Here's a little process disagreement we are having here on my project. Let me set the scene:

    The team:
    7 business analysts
    2 QA/testers (a consultant and myself)
    10 or so developers.
    Management on both sides, QA is on the business side.

    On Thursday, my QA consultant cohort got a "stack trace error" (which is the rules engine equivalent of a browser error, from what I've seen) while working in the application, because the application timed out between screens. He wrote up a defect with all the fixin's (screenshot, procedure leading up to it, etc.) and submitted it through our defect tracker (JIRA).

    IT rejected the defect and told us not to submit timeouts as defects because it indicates a resource shortage and not a problem in the code. Since they don't have the ability to fix it, they don't want to see defects on it - I should email the IT manager (who is at her desk about 30 minutes a day) instead, and she will escalate it if the problem is still going on. They were quite angry about it, actually (I'm over the wall from them and I hear them talk).

    I sent an email back to them saying that an error is an error - if there's a problem that stops testing I'm logging it whether they can fix it or not. My position is that I'm the one that's going to fry when they give us 2 weeks for UAT and we're locked out for 8 of the 10 days. Plus, IT already demanded that I not contact them directly about defects and only use JIRA, so basically they just want me out of their hair.

    Now I've been called to a meeting with my own boss to "discuss the situation." I'm going to go in with two key points:

    #1: Irrespective of the root cause of a defect, a defect is a defect and QA is responsible for documenting them all.

    #2: System problems that stop testing most definitely must be logged because at the end of the project I am going to be held responsible for how much testing occurred in a certain time period and I want to have concrete records on when testers could not test.

    What I've tried to communicate to everyone is that a defect is not a work order. I don't care what anyone does with the defects, it's just my responsibility to document them.

    Thoughts on any of this?

  2. #2
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    [ QUOTE ]
    Clint: IT rejected the defect and told us not to submit timeouts as defects because it indicates a resource shortage and not a problem in the code.

    [/ QUOTE ]Then what is it? Is something improperly configured? Is the design flawed? Are there critical timing components that are failing under a certain processing demand level?
    .
    .
    [ QUOTE ]
    Clint: Since they don't have the ability to fix it, they don't want to see defects on it ...

    [/ QUOTE ]Ask them what they do want - a change request, etc.?
    .
    .
    [ QUOTE ]
    Clint: They were quite angry about it, actually (I'm over the wall from them and I hear them talk).

    [/ QUOTE ]<changing response tone to cynical> Ask them if anger is curing the issue.

    <changing response tone to less cynical>
    Revise your estimates accordingly and push for their signoff. This is effective and puts owner nostrils over the fragrance.

    To me, whether a defect or not, it appears to have a schedule impact. I would vote that you team up with them and wrestle the schedule impact issue.

  3. #3
    Senior Member
    Join Date
    Apr 2007
    Posts
    1,005
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    [ QUOTE ]
    [ QUOTE ]
    Clint: IT rejected the defect and told us not to submit timeouts as defects because it indicates a resource shortage and not a problem in the code.

    [/ QUOTE ]Then what is it? Is something improperly configured? Is the design flawed? Are there critical timing components that are failing under a certain processing demand level?

    [/ QUOTE ]


    The problems are stemming from the fact that our test environment and our production environment are running on the same hardware, so at certain points during the day, when production spikes, test goes down because production (understandably) takes priority.
    .
    .
    [ QUOTE ]

    [ QUOTE ]
    Clint: Since they don't have the ability to fix it, they don't want to see defects on it ...

    [/ QUOTE ]Ask them what they do want - a change request, etc.?

    [/ QUOTE ]

    They don't want to see it at all because they feel it's not their area. Part of the breakdown as I see it is the perception by IT of a defect as a work order.
    .
    .
    [ QUOTE ]

    [ QUOTE ]
    Clint: They were quite angry about it, actually (I'm over the wall from them and I hear them talk).

    [/ QUOTE ]<changing response tone to cynical> Ask them if anger is curing the issue.

    <changing response tone to less cynical>
    Revise your estimates accordingly and push for their signoff. This is effective and puts owner nostrils over the fragrance.

    [/ QUOTE ]

    I'm not in charge of the estimates. [img]/images/graemlins/frown.gif[/img]
    They are dictated by the business management, or that is to say, the schedules are and any estimates are ignored.

    But I will take that point to the business management: any schedule they set is going to get missed if they assume that testers will be able to test all day long.

    [ QUOTE ]

    To me, whether a defect or not, it appears to have a schedule impact. I would vote that you team up with them and wrestle the schedule impact issue.

    [/ QUOTE ]

    So, in this case, would you not want to document these types of problems and instead would just partner up with IT to revise the schedule? I personally would feel a lot more comfortable if the reasons for the schedule slipping were documented.

  4. #4
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    Oooops! I didn't say to not document. [img]/images/graemlins/smile.gif[/img] I suggest not pushing the word "defect". Certainly document the schedule impact due to this issue, even if you document it only in email.

    [ QUOTE ]
    Clint:
    The problems are stemming from the fact that our test environment and our production environment are running on the same hardware, so at certain points during the day, when production spikes, test goes down because production (understandably) takes priority.

    [/ QUOTE ]Maybe the outcome will be separate environments?? You actually may have a lever here. What do you mean when you say "test goes down"? If prod "grows" your situation can only go one direction - worse. This is just not a good approach - prod and test on the same server???

    Someone needs to take ownership of the problem!

  5. #5
    Senior Member
    Join Date
    Apr 2007
    Posts
    1,005
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    [ QUOTE ]
    Oooops! I didn't say to not document. [img]/images/graemlins/smile.gif[/img] I suggest not pushing the word "defect". Certainly document the schedule impact due to this issue, even if you document it only in email.

    [/ QUOTE ]

    The reason why I like the idea of logging it in the defect tracker is that it creates an audit trail. Plus, if I had my druthers, the workflow would be:
    -tester #1 observes test has gone down
    -tester #1 logs downtime as defect
    -tester #2 observes test has gone down
    -tester #2 goes to log defect, but sees that it's already been logged, and watches the prior defect.
    -test comes back up
    -everybody that was watching the downtime defect gets notified.

    So it reduces work duplication and it lets testers know when they can get back in.
    [ QUOTE ]

    [ QUOTE ]
    Clint:
    The problems are stemming from the fact that our test environment and our production environment are running on the same hardware, so at certain points during the day, when production spikes, test goes down because production (understandably) takes priority.

    [/ QUOTE ]Maybe the outcome will be separate environments?? You actually may have a lever here. What do you mean when you say "test goes down"? If prod "grows" your situation can only go one direction - worse. This is just not a good approach - prod and test on the same server???

    Someone needs to take ownership of the problem!

    [/ QUOTE ]

    "Test goes down" - users in test start to see very long response times, eventually leading to timeouts and integration errors. Eventually you can't even log in to test. This goes on for at least a few minutes every day, sometimes all morning or all afternoon.

    The recommendation for separate hardware for the test environment has been made many times. I don't know all the ins and outs of why they haven't done it; I suspect it's mainly because if they're going to spend any money it's going to be on production and not on test.

    As you have intuited, I'm kind of hoping that if we are able to get clear metrics on exactly how painful that situation is, we in the testing areas can use those metrics as a lever to raise awareness of the issue.

  6. #6
    Moderator
    Join Date
    Jan 2005
    Location
    England
    Posts
    765
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    Certainly I agree with your 2 points and from what you've said it sounds like the problem is that IT aren't part of the solution process (JIRA) but ARE part of the problem.

    In my opinion you have to document these issues and a defect management system is the right place to log them, but you need the right people using JIRA.

    You don't know if this is a code issue, a database issue or a network issue. And even if you do, the person who comes along next year might not. So JIRA is not only about defect management, it's also a knowledge base.

    If you logged each of these different issues in a different system (or emails) then you'd lose your ability to report on the problems you've encountered and you'd lose your knowledgebase.

    I agree the problem seems to be a process problem and the perception of what the defect management system is for. We have a similar thing here, issues are raised but it's unknown if it's an issue with code, environment or documentation. If the defect goes to a developer then they instantly see it as a work request and it can get their back up if it's actually a documentation problem.

    So we have development, analysis, database, testing and the business (cutomer) all using JIRA. If the issue is clearly in one of these areas it goes straight there. If there's some question about it then the JIRA initially goes to the development manager and he steers it to the right person.

    I'm not sure this should really be his job, but it seems to work.

    It seems to me that you need to have everyone on-board with using JIRA. You can then assign them straight to IT, or devlopment can assign it to IT if they discover it's not a code issue.

    Good luck,
    Steve
    Everywhere's within walking distance if you have enough time.

  7. #7
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    To me, if defect = hot button, then one must find a cool button. A defect report that continuously gets rejected and upsets people is not likely to have any more "love" attached to it than the previous attempts.

    Still, it needs to be documented and given visibility. That is a give. What is the most sensible medium? Email leaves a trail as well and can be saved off into critical documents.

    The biggest thing I see at this point is computing cycles competition between test and prod. Again as prod grows and requires more computing cycles, the more crippled your efforts in test become. Capture and record the time of each outage.

    What is even riskier here is that your own efforts in test may be impacting prod?

  8. #8
    Moderator
    Join Date
    Mar 2000
    Location
    Orange County, CA
    Posts
    3,187
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    In reading your point of view AOQA, I don't get the feeling you have logged the issue with the information respecting the end user's point of view. And please forgive me if this is not the case in reality and it was just not conveyed here.

    From your and your testing perspective, it is an impact on time and a problem, but to development it is something that is out of their control to handle it. But instead of logging information of this type in a matter-of-fact manner, what about changing the perspective to that of the end user. If I was using the application, to me it isn't an issue of whether the system timed out or not, but how I was informed it timed out.

    It seems to me changing this one issue from a stack error/timing defect to one in the style of "user is not informed in a graceful manner when time outs occur", at least then they might address the issue if they are going to look into notifying the user of the time out issue (as you can bet they will happen from time to time in production), and time has to be spent on it anyway. most developers when assigned the task would much prefer to not have to put a user notification in a program, they'd rather fix the issue.
    Insanity: doing the same thing over and over again and expecting different results

  9. #9
    Senior Member
    Join Date
    Apr 2007
    Posts
    1,005
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    Well, I've returned from my meeting and the ruling is that IT's discomfort with referring to resource shortages as a defect takes priority over my discomfort with letting it go undocumented. So, we will be relying on an email to IT managers when the system goes down.

    I guess I will start saving those emails into offline documents so when somebody inevitably comes to me and says that I never said that there were system problems, I can produce proof that I have.


    [ QUOTE ]

    It seems to me changing this one issue from a stack error/timing defect to one in the style of "user is not informed in a graceful manner when time outs occur",

    [/ QUOTE ]

    That's a good point and something that also needs to get addressed. There's a whole category of errors of that type that haven't been looked at yet. I've been trying to determine which iteration is going to contain error handling of that type, apparently it's not this one.

  10. #10
    Senior Member
    Join Date
    Mar 2007
    Location
    Waterloo, Ontario, Canada
    Posts
    3,628
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: System downtime = defect?

    Ouch, sounds like you have a rough situation here. I would make sure you get signoff that they realize the problem. That's a definite. If they acknowledge that the issue exists and they acknowledge that this is affecting your ability to complete your work in a timely manner, then the accountability for this issue has been removed from you. Just get it in writing.

    I had a situation at one point where I didn't get something in writing. In the end, I was giving congrats to my staff for hitting our scheduled release date, and I was getting crapped on in a meeting by our dev manager, who I later found out promised it a week earlier to the CEO. This type of cruddy communocation is what ultimately led me to leave a job which I loved.
    Brent
    --------------------
    9 out of 10 people I prove wrong agree that I'm right. The other person is my wife.
    --------------------

 

 
Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.00%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 02:42 AM.

Copyright BetaSoft Inc.