Negative tests in Regression Test Suite
Are we to include negative tests in Regression Test Suite. If so, to what extend we need to do that? All the negative tests or part of it?
Appreciate your suggestions
In my opinion, it depends on the percentage of your total test suite that is made up of negative tests. If you have just one or two happy flows and tens of negative test cases, then it would make sense to include some of the negative tests.
I'd advise to always focus on the positive test cases first, though. The main task of a regression test is to determine whether a change in code or configuration does not break any existing functionality, and that is most easily and most accurately determined by executing positive regression test cases.
Absolutely. Would you think it a good idea for an online store to not test credit card rejections in regression? As with all sorts of tests though they must be prioritized and have an execution plan put together for each release based on risk and impact of change.
Thanks for your responses. @NoUse4aName, Does that mean we need to update our regression suite based on the risk and impact analysis, each time before the release?
So then there could be possibilities to remove tests from existing regression suites rather than adding every time.
Further are we suppose to do regression testing in component level or Acceptance level (End to End)?
Last edited by shabar; 06-14-2014 at 04:51 PM.
Yeah, otherwise your regression suite will eventually grow to such a size that you can not execute it in a timely manner. Have to make judgement calls for example a test that has never failed and you have not touched any related code this release. Can retire it to some sort of regression archive that would only be brought back into the regression suite when a related code change takes place or there is some major system overhaul that requires a full functionality check.
Well it depends on the project perspective. We don't consider negative cases in regression pack but we done scripting for negative scenarios and execute them based on the priority set by functional testing team. It's wise to have a discussion with Functional testing team before do scripting.
What do you mean by functional testing team? Generally we have one test team, may be with different area like web, mobile device, back-end etc. So in this type of situation as a team we all have to decide whether to include negative tests or not.
Originally Posted by venu.adhikesevan
What general rules we should apply for this?
Shabar, Usually Manual Testing Team(Functional testing) will hold the ownership of the testing and they will give the requirement for automation to Automation team. Since the Testing team is going to help us to prioritize the scenarios its wise to have a discussion with them.
Regarding the rules we haven't followed any rule maybe you can consider previous posts from NoUseForName which is clearly mentioned the risk factors to be consider.