Results 1 to 5 of 5
  1. #1

    Test Matrix, Scenarios and Cases

    Hi All,

    Our QA team is following the below methodology when creating Test Matrixes. Yet management would like to remove ‘Test Scenarios’ from our plan. Please read through this and provide your feedback – opinions and arguments for or against what we have set up are needed.

    Test Matrix: What we are testing in the GUI
    Test Scenarios: How we are testing the Matrix (steps and various ways)
    Test Cases: A combination of test scenarios covering the end to end process in various ways.

    Example of Test Matrix:
    System Password Field

    Example of Test Scenario1:
    Narrative: Set Password = ABC!@#
    Expected results: Validate the password is excepted by the system. Validate a successful log in with this password.

    Test Scenario2:
    Narrative: Set Password = ASDFGHJ
    Expected Results: Validate error message is displayed “Password must include Alpha characters with numbers or characters, please re-enter password”

    Test Scenarios continue to cover the various requirements of passwords.

    Example of Test Case
    Test Case 1 = Test Scenarios 2, 1, 37, 42, etc.
    (note: number 37 and 42 are symbolic of another area of functionality within an application)
    The purpose of breaking everything out into scenarios is
    Ease of combination with other functionality
    Focus in on one area and repeat a single scenario when a defect is found.
    Avoid Ad-hoc testing with unrepeatable results.

    If we remove the scenarios, in my opinion we lose 75% of our test coverage. Or should the scenarios become the Matrixes? Or anything else? Please provide your opinion on this set up

    "We may encounter many defeats but we must not be defeated."
    -Maya Angelou

  2. #2
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    St. Louis - Year 2025

    Re: Test Matrix, Scenarios and Cases

    Ask management to sign a risk statement summarizing what they want. they will either sign or allow you to keep your current coverage.

  3. #3

    Re: Test Matrix, Scenarios and Cases

    It may also be helpful to have a common understand of what a test scenario is, and what a test case is. It seems that the terms are used in more than one way in your original description -- perhaps this is another cause for confusion. Check http://www.testinginstitute.com/terminology.php for a definition of the terms.

    Aside from that, it is almost impossible to conduct testing (especially GUI testing) without scenarios (which are the vantage points, or roles, a person has while using the application) unless the application is so basic that there are not that many options.
    Jay Ruuska

  4. #4
    Senior Member
    Join Date
    Jun 2000
    Charlotte, NC, USA

    Re: Test Matrix, Scenarios and Cases

    Seems to me a matter of semantics. What you describe as 'test scenarios' could just as easily be re-worded to 'verification points' within a particular test case.

    In looking at how you've parsed this out, it seems to me also that simply saying a test case is equivalent to multiple test scenarios is probably not quite true, unless the scenarios are structured in such a way that they flow seamlessly from one to another.

    Either of the approaches (include test scenarios and test cases; or drop the test scenarios) may provide the coverage documentation your boss is asking for.

    To their point ...
    What is your 'test matrix' referring back to?

    If the requirements are specifically: 'A password of "ABC!@#" should be accepted', and 'A password of "ASDFGHJ" should be rejected with the error message ... blah, blah, blah ...', then I can see having the specific 'test scenarios' listed in the matrix.

    If the requirements simply state: 'Passwords must include at least one numeric and one special character', then your 'test scenarios' are simply variations / permutations of the allowable / disallowed input.

    Final point, be clear that the test / traceability matrix is either referring to the actual business and/or functional requirements or it is referring to test requirements. These are not necessarily the same.
    Resistance is futile.


  5. #5
    Join Date
    Mar 2004
    West Coast of the East Coast!

    Re: Test Matrix, Scenarios and Cases

    I would suggest that you ask management if they take their automobiles in for repair to mechanics who can not afford the right tools for the jobs. If a mechanic has only a few basic tools, he may be able to fix the auto, but is it really fixed to factory specs, or even to customer satisfaction? Probable not. These scenarios are no more than tools for the testing department and the better the tools the better the results and the more reliable is the fix. If they take away the tools then the risk of defects in the field go up. However, if the test cases fulfill the tools requirement and cover the same transactions, then perhaps you are doing the same testing twice. We do not use scenarios, but we use test cases which are complete and can be run in a stand-alone fashion, so in essence they are really scenarios and thus the easy confusion between scenarios and TC's, not that it matters a bit what you call them. So to satisfy management, if I were in your position, I would call all tests Test Cases, and drop the word scenario from your vocabulary. But only for this one employer.
    Personal Comment

    Success is the ability to go from one failure to another with no loss of enthusiasm.
    ~ Winston Churchill ~

    ...Rich Wagner



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
BetaSoft Inc.
All times are GMT -8. The time now is 07:36 PM.

Copyright BetaSoft Inc.