The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Quality Engineering >> Requirements and Design

Pages: 1
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
100% Requirements coverage
      #655980 - 01/13/11 02:50 AM

Hello

I'm attempting to write a test policy document for my company and have come across a stumbling block when it comes to defining one of my suggested quality targets, namely for testing to achieve 100% Requirements Coverage.

To someone who isn't clear on the definition of this, their first instinct implies that we are therefore testing absolutely everything! This isn't true however as we are only testing the application from the point of view that the user can do what the are required to do in the application. The target does not involve all the other functionality and all combinations and variants of data in the application.

Has anyone any suggestions as to how I might explain this a little better?

G


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: 100% Requirements coverage [Re: Graeme Z]
      #655990 - 01/13/11 03:42 AM

I guess I'm unclear on your definition as well.

Perhaps you could explain it as:
If it's in the Requirements, we must test it.
If it's not in the Requirements, we might not test it.

Does that capture the point you are trying to make in your test policy?

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
Re: 100% Requirements coverage [Re: Joe Strazzere]
      #655993 - 01/13/11 03:59 AM

That sounds a little better than my wooly explanation

If it was up to some the target would be "make sure it doesn't crash". I don't think anyone could be 100% sure of that...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: 100% Requirements coverage [Re: Graeme Z]
      #656004 - 01/13/11 04:25 AM

I'm curious.

In your Test Policy for your company, what are you hoping to achieve by having "100% Requirements Coverage" as a quality target?

When I read it, my first thought was that he's trying to motivate people to give him extremely voluminous Requirements.

But upon re-reading it, I wonder if you are trying to say "Here's how you can measure QA. If we test all the Requirements, we are doing a good job. If a bug reaches production because something wasn't in the Requirements, that's not our problem."

I've never seen "100% Requirements Coverage" specifically mentioned as a quality target before.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
Re: 100% Requirements coverage [Re: Joe Strazzere]
      #656010 - 01/13/11 05:15 AM

Well I'm the only test person in the company, so I'm looking put some context into what I do and provide something measurable that will tell people that the application we are developing is what we intended it to be.

I can try and provide a number of levels of test coverage to ensure this. Requirements coverage, specification coverage, path coverage, state transition coverage, etc, all of which I'm looking to achieve some degree of in the testing cycle. By attempting 100% requirements coverage, we can at least have some confidence that the customers get the very least that they are looking for in the application and that the software we create will do what we intended it to do.

By setting the target as achieving 100% requirements coverage it also means that we concentrate our testing efforts on on the important things, for example "does user name field accept the right number of characters" rather than "does the dialog have a windows close button". We want to capture both obviously, but we want to focus our efforts more on the former.

I think the focus of the testing is the most important factor in this. As I mentioned before, if you ask a developer what I should focus on, they'll say "make sure it doesn't crash". Does that kind of make sense?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: 100% Requirements coverage [Re: Graeme Z]
      #656012 - 01/13/11 05:29 AM

I think I understand.

When you say "100% requirements coverage", it means you'll focus on written requirements, to the exclusion of unwritten requirements?

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
Re: 100% Requirements coverage [Re: Joe Strazzere]
      #656013 - 01/13/11 05:41 AM

Essential yes.

We have regulations to work under so we must be able to prove that we built what we said we'd build, and for me this is the simplest way of identifying this. We will also therefore need to show coverage against specifications as well.

It's also something we already do but I need to ensure that everyone else understands what the statement implies.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: 100% Requirements coverage [Re: Graeme Z]
      #656017 - 01/13/11 05:59 AM

Quote:

I need to ensure that everyone else understands what the statement implies.




Yup, that's the tricky part. People read their own implications into whatever someone writes.

What I've found most helpful is to state as explicitly as possible what you won't do. Often that helps in communicating.

For example:
- We won't be looking for look-and-feel issues, unless they are individually stated in the Requirements.
- We won't be testing performance beyond what is stated in the Requirements
- etc, etc.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
SurendraMohan_J
Newbie


Reged: 12/15/10
Posts: 7
Re: 100% Requirements coverage [Re: Graeme Z]
      #656129 - 01/13/11 10:31 PM

Hi,

See if this helps..

Define high level scope & Out of scope.
Ex:
Scope:
1.All the navigations
2.Functionality of xx component etc.

Out of scope:
1. Performance of the application
2. Security etc..,

Then explain the requirements covered as part of scope in detail.

Methodology what you are going to follow.

Use the tracebiliity of what you are covering.

You can define metrics accordingly.

Regards

Surendra Mohan


Post Extras: Print Post   Remind Me!   Notify Moderator  
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
Re: 100% Requirements coverage [Re: SurendraMohan_J]
      #656147 - 01/14/11 01:40 AM

We had a little discussion on this yesterday and it opened up the point about what things should go into the requirements. Should we be including things that we can't test? If we can't test a requirement then maybe there's no point listing them as a requirements as we will never be able to prove that they have been met.

It also became apparent that we needed to understand that 100% requirements coverage is test metric and not a development metric. So we are not saying that we will implement 100% of the requirements stated, but that we will be testing 100% of the requirements that have been implemented into the application. This I think was quite a key point and people seemed a little more relaxed about that idea.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: 100% Requirements coverage [Re: Graeme Z]
      #656165 - 01/14/11 03:53 AM

Quote:

It also became apparent that we needed to understand that 100% requirements coverage is test metric and not a development metric. So we are not saying that we will implement 100% of the requirements stated, but that we will be testing 100% of the requirements that have been implemented into the application.




So you have Requirements (ie, things that are required) that won't be implemented?

Seems odd.

In my experience, if something is actually required, then it must be implemented. Otherwise, it would be dropped from the requirements.

And when we test, when we encounter a requirement that hasn't been implemented, we write a bug report.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
SurendraMohan_J
Newbie


Reged: 12/15/10
Posts: 7
Re: 100% Requirements coverage [Re: Joe Strazzere]
      #656371 - 01/16/11 11:20 PM

Hi,

Agree with Joe.

Quality persons focus should alwats be at the requriements not at the development.

You can keep the metrics as 100% developted requirements for your monitoring.

Regards

Surendra Mohan


Post Extras: Print Post   Remind Me!   Notify Moderator  
Graeme Z
Junior Member


Reged: 02/20/04
Posts: 24
Loc: England
Re: 100% Requirements coverage [Re: SurendraMohan_J]
      #657650 - 01/25/11 03:44 AM

We managed to settle on "Tests to provide 100% requirements coverage of the implemented requirements."

So this makes a distinction between requirements that were thought of but were dropped because of resources and time scales and keeps the definition test based. If you simply say "provide 100% requirements coverage", it can be seen that we would want to develop all requirements which as stated above we may not do.

Test Strategies and Policies are not easy things to implement!


Post Extras: Print Post   Remind Me!   Notify Moderator  
AnandTambey
Advanced Member


Reged: 11/11/07
Posts: 647
Loc: India
Re: 100% Requirements coverage [Re: Graeme Z]
      #659326 - 02/04/11 04:52 PM

Agreed with Joe and other gentlemen.

Usually when you say 100% coverage of requirements, you essentially be building a traceability matrix to every phase like design,code ,system test and UAT.

Thus if a requirement dropped all the dependent phases and impact can be found easily. Hence finally you will be covering 100% coverage of implemented requirements with confidence.

--------------------
Kind regards,
Anand Tambey

RSS Feed : Break To Make it Better
A Lazy person could be the best automation professional, if he is not lazy in implementing his ideas to reduce his work. ~Anand Tambey


Post Extras: Print Post   Remind Me!   Notify Moderator  
GauravPandey
Member


Reged: 03/29/10
Posts: 83
Loc: India
Re: 100% Requirements coverage [Re: AnandTambey]
      #660642 - 02/15/11 07:12 AM

Hi

I am not sure if we CAN test all the requirements.

If we look at it - we have three types of requirements -

- Functional Requirements
- Quality Requirements
- Constraint (or conditional requirements)

We may "attempt" to cover the Functional Requirements, however I am not sure on how we would be able to cater to Quality Requirements or Constraint Requirements though!

Regards
Gaurav Pandey

--------------------
Regards
Gaurav Pandey
http://www.gauravpandey.co.in/


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alby_George
Newbie


Reged: 04/13/12
Posts: 3
Re: 100% Requirements coverage [Re: Graeme Z]
      #704174 - 04/13/12 05:39 AM

Hi

There are two scenarios. 1) Requirements doesnt change once approved and baselined 2) Requirements change or evlolve as the project is progressing. The second case is seen in projects since the customer at first may not have a clear idea of the requirements. They arrive on the final requirements after running many iterations of prototype building.

In scenario 1) 100 % Requirements can be achieved by maintaining a testing traceability matrix in which the BSG team ( Business Analysts Group )list down the each individual feature and the testing team updates the test case id corresponding to that feature. An approved NFRs (Non Functional Requirements ) should be available with the project to test the NFRs.

In scenario 2) 100% test coverage is rarely possible due changing requirements and ambiguities on implicit and explicit requirements.

Additionally if you have to test interfaces in your project you have to get their connectivity in staging environment which again is not under your control

IF you plan to explain it using numbers. Take two drop down boxes with values as "Yes " and "No" .You will have to test following combinations. 1) "Yes" , "Yes" 2) "Yes" , "No" , 3)"No","Yes", 4) "No","No" . For two simple boxes combination is 2! so imagine if you have some 300 screens to test . The required effort will be more than the total effort allocated for the project.

Regards
Alby


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 2 anonymous users are browsing this forum.

Moderator:  blueinatl, swt88, AJ, Daniel_S 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 12305

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5