How to get the necessary documentation from dev for testing
Usually any code branch that is submitted to QA originates with product management, which makes it easy for QA to test, because the development work was driven by user stories and product requirements that can also drive testing. I work in an agile environment so I'm used to asking questions and having a back and forth with dev when I don't have enough information.
However sometimes dev will run away with a project and submit a code branch with little documentation and no product management review. A bug fix will, according to dev, necessitate rebuilding a code framework that affects the whole product, "All this other work was necessary to do it right." Dev will create a new feature that hasn't been asked for or reviewed by product management to fix the bug, "It isn't a new feature so don't test it against any existing product requirements, just make sure that the bug is fixed." Unhinged by any external product requirements, expectations are driven by what dev did and chose not to do, "We didn't code it to work that way, so that's not a bug." Dev tries to accomplish a laundry list of their own for the project beyond the simple bug fix but doesn't document them in the ticket, "That's not customer-facing so you shouldn't be testing that." What documentation dev does provide is a short, jargony explanation of what they did that is hard for testers to interpret and fails to describe much of the work done, "We documented everything you should need, it's all right there. (It isn't.)"
At a certain point, the agile process breaks down. When QA asks questions spends two weeks of testing asking questions and asking follow-ups and still learns of new dev goals and expectations for the first time in the clear-to-ship meeting, the process has failed. That is an unreasonable burden on QA to obtain this information on their own.
By the way, all of the above happened with one code branch. In hindsight, it probably should have been rejected by QA in the first place for insufficient documentation.
How do I avoid this problem in the future? Is there a standard of documentation that QA can reasonably demand before a code branch will be tested and which QA can hold dev accountable to? What is that standard and how can I explain it to dev so that it can be enforced? Thanks!
It's not unreasonable to demand something you need to do your job right?
I think you need to present the case. If they provide you with XYZ, then here are the possibilities of things that can happen, and here is where the company will be if it those things keep happening. Then outline a vision of how things can be if XYZ were provided and how much the company is better off having XYZ.
I think you want to present it in a way to make it look like your primary job is to help devs. Say it makes you both look bad when bugs ship to production, and you're there to help the devs to produce the very best code they're capable of.
If that doesn't get the devs on your side, I'm sure the CTO and CEO will take heed.
I don't know how to identify and communicate what XYZ should be. This is what I need help with.
Originally Posted by dlai
Typical information you can expect
1) What changed, and what are the acceptance criteria. - in agile it's usually in a form of a story. To justify any work or change done to code, there needs to be a business reason or a technical reason. Those should have a method to validate in some way. In modern era of change control, if you find any undocumented changes, it's general practice in most shops to automatically back it out for safety sake. For example, say you get a rogue commit that's not associated with a story. How do you know it's not a virus that submitted things into the source?
2) Information Architecture - At any point, you should be able to swap out the teams entirely. So any information architecture needs to be documented. How can anyone identify risks without formally communicating changes in information architecture?
3) A statement outlining the risks. Usually this comes in form of either additional acceptance criteria and considerations in a story in Agile, or a formal TPS (test procedure specification) report in formal Test Plan.
4) Chain of responsibility. Who to go to for what. It's too much of a waste of time for devs and QA to hunt down the people you need, and you may not have all the things you need up front. For example, you may need to talk to BA's to clarify requirements, DB Admin to setup enviornments, etc...