We frequently role out software version upgrades with the software we use. We have no established guidelines for doing this and as a result we frequently end up in trouble. We have missed several large bugs in the version releases in the past and have cost the company whole days of downtime nation wide. Are there any references or resources for establishing best practices for end user software functionality testing?
I don't have an official resource to offer, just some advice. Do you have in house technical support? If you do, then you have an excellent resource for end user testing. Those folks know every silly thing the end user will do with the software and then some. If you don't do your own tech support then you need to find about 5 POWER users that are willing to take the time to use the software for a couple of weeks before mass rollout. We roll out in our state first so we can send someone quickly if needed. We let this go for about 2 weeks and if the calls to technical support are only for upgrade questions and not issues, we roll out everywhere else.
If you roll out often because of issues found, you're going too soon. Perhaps it is not the end user testing you need to watch but re-evaluate your process as a whole. Sounds like your coverage may be a little slim.
- you should evaluate the nature of the bugs you've skiped and improve the testing coverage for those bugs. That could be possible only if you find a pattern in thoese bugs. For example, are these mostly caused by poor negative or border testing, scenario testing or any other testing type.
I mentioned these types of tests, because I saw that most part of skiped bugs, happened in our company, for these kind of tests.
What we have done, was to involve some end users ( or people from customer support, which have more a user perspective in testing - UAT) and get theirs help in defining the testing scenario for the UAT.
... Not even me.