QA of a Product vs. a Network?
In a recent discussion with my manager regarding QA's approach to testing my current company's "product" (a LAN installed a our customer's sites with a satellite connection for software updates), he stated, "QA's test approach of a networked system like ours should be different from testing an actual "product".
I'm not entirely sure what he means by this, but I'm assuming that he's referring to the ability we have to update our system software remotely versus a "product" like say a Videogame. I think he thinks that the release planning for our system's software can be more relaxed than with a finished product. He may be correct, but being the Director of QA, I tend to always place Quality at the top of the list and don't believe that our test approach should be any different. Goal as always is to catch as many serious defects prior to release, as possible. With our system, bugs will be readily apparent to the user/customer. We do have a "phased" release process. The software is released and installed at a certain number of customer sites and then steadily ramped up in number of sites over a period of time. So there is a tendency for management to say, "Well, we can go ahead and ship with some serious defects, come up with a fix and then go back and rework an X number of customer sites".
Just curious if others agree with my manager's statement?
Re: QA of a Product vs. a Network?
If anything, I'd say you'd need to have more rigorous testing cycles. And the focus should be on:
1. Working with as much "production" data as possible
2. Simulating the deployment configuration as closely as possible
Perhaps management needs to understand the cost of all the rework, the loss in reputation and the resulting "business you dont get". If bugs are going to be even more readily apparent, this cost could multiply easily.
Maybe you could modify the release plan this way:
1. Identify release criteria and different levels of (sets of ) stability criteria.
2. No deployment until some critical release criteria are met.
3. First phase of ramp up begins after stability criteria set #1 is met.
4. Next phase of ramp up begins after stability criteria set #2 is met.