The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Quality Engineering >> Quality Methodologies

Pages: 1
darkstar
Member


Reged: 07/04/05
Posts: 34
Test documentation when time is really tight...
      #299498 - 07/09/05 09:39 PM

hi everyone

I'm not sure to what extent this question might have been discussed elsewhere, but anyway...

I often find myself on small-ish projects, where time alloted for testing is tight (primarily, I do manual functional testing, and we have Test Director).

When I have the time, my preferred approach is to enter all the requirements in Test Director, link those to test scripts in the Test Plan area of TD. And link those scripts to test sets in the Test Lab area, and run the tests there. I find this helps me to get clear on what I need to cover, what scenarios I should run, and acts as a good checklist and what I've done and haven't done, etc. The trouble is, doing all this can be somewhat time-consuming and there are times when I just need to get straight to testing the critical stuff. So, in these situations, I'll typically dispense with TD and do a rough and ready test plan, just using MS Word.

I suppose my key question is: if you're really short of time to test, what do people think is the minimum amount of documentation you need to get reasonable confidence that the software is not going to have bugs that the customer will turn around and say: how could they have not found an obvious bug like that?? (Well, I'd like to do my best to ensure this doesn't happen and would like to hear some advice from some of the testing gurus here...)

I guess if you're really clear-headed about the testing priorities and have a really good memory, then you could just go ahead and test without any documented plan or scripts. My brain, however, doesn't work that well...

Like to hear what people have to say. I hope all this makes sense...


Post Extras: Print Post   Remind Me!   Notify Moderator  
ljeanwilkin
Moderator


Reged: 01/13/03
Posts: 1182
Re: Test documentation when time is really tight...
      #299499 - 07/10/05 04:48 AM

It makes perfect sense. I've run into the same problem many times.

At an absolute minimum, I put a test outline (could be what you refer to as "test requirements"?) into the requirements tab in TD. I make sure there's enough detail in the outline to allow me to test from it. Generally, the outline takes me a very short amount of time to develop; the test cases take considerably longer.

In some cases, we create test cases with no detail (just the title and test case #), linked to the test requirements in the outline. This allows us to use TD to manually "run" the scripts, marking each one as pass or fail. This allows us to pull stats, etc. from TD, even without test case detail.

We then play "catch up" after the testing to flesh out the test cases, when possible. When not possible, we keep the test outline updated to ensure we can regression test.

Our tests are still documented, results are still documented, reports are still generated, and we would pass an audit. The most dangerous part of the process is that the goal always has to be to get the test cases written; otherwise, you end up with a huge bank of test outlines, which are difficult to pass off to other staff to test (limits your flexibility).

Hope this helps!

- Linda


Post Extras: Print Post   Remind Me!   Notify Moderator  
darkstar
Member


Reged: 07/04/05
Posts: 34
Re: Test documentation when time is really tight...
      #299500 - 07/11/05 06:46 AM

hi Linda

thanks for your help. I hope you can bear with me while I get clear on what youve said.

I think here is a good place to start using examples here goes:

- youve built a website that takes orders for widgets
- step 1: enter name & address
- 2: enter credit card details
- 3: enter number of widgets required
- 4: submit

(simple I know, but should help illustrate what Im attempting to say)

so, in the case of what youve described, my test outline might say (I assume):

- test name & address fields (maximum character limits, special characters/numbers not allowed where appropriate, etc)
- test credit card details: invalid/valid details, declined & accepted cards, etc
- test number of widgets field (maximum and minimum number limits, etc)
- test Submit button (all data written away correctly, credit card accept/decline works correctly, etc)

the question here is: how much detail do you think is necessary in the test outline? I mean, is it basically just a list of the names of the functions youre going to test, or is there more to it? Perhaps just a couple of notes next to each function to jog your memory about particularly important things you need to do? Is there an effective way to mark the level of priority of the tests in TD?

Next, you say that you link your test cases to the test requirements in the outline, and that sometimes theres no detail in the test cases. So it sounds like the test cases might simply be a duplicate of the list of test requirements in the outline. And that this is done just for the purpose of tracking what youve tested and whats passed/failed? If the test cases dont have any detail in them, how do you, for example, record what data inputs youve tested with particular scenarios? You must note this kind of stuff somewhere, surely, otherwise how could you go back and flesh out the test cases? And Im not sure what you mean by we keep the test outline updated to ensure we can regression test.

and, if I understand your last bit correctly, youre saying that if you continually test using this method, youve got to be careful you dont end up with a large number of test outlines, and no detailed test cases. And if this happens its difficult for other staff to re-use your test documentation at some later time if the same functionality needs to be tested (for some reason), because theres no detail of what and how to test. That right?

thanks again for your help [Smile]


Post Extras: Print Post   Remind Me!   Notify Moderator  
ljeanwilkin
Moderator


Reged: 01/13/03
Posts: 1182
Re: Test documentation when time is really tight...
      #299501 - 07/11/05 10:52 AM

My test outline is a bit more detailed, as I write it as if I'm going to test from it (whether I do or not). It might looks like this

Name and address
Name
Verify first name (a/n, 20 chars)
Verify last name (a/n,40 chars,-, ', space)
No first name (error message)
No last name (error message)
Both first and last blank (error message)
Address

...and so on. I agree with you that it should be just as detailed as required to jog your memory about particularly important things you need to do.

You can easily add a field to the requirements tab for "priority".

And yes, the test cases might simply be a dup to allow for tracking. It could be bypassed and a field put into the requirements tab, but TD has a lot of nice test management features that hinge on the test cases. If the test cases don't have any detail in them, but we have other documents that specify data inputs, we attach it to the test case.

Keeping the test outline updated just means that if we don't have time to go back and write test cases, we do make sure that any additional tests we performed (maybe based on changed functionality or fixes) are incorporated into the outline.

And you got the last paragraph right; the main pitfall of testing from outline is never getting around to writing test cases. Sooner or later, it comes back and bites you. Either the original testers leave or get reassigned, or you need to hand off your scripts to another group...

- Linda


Post Extras: Print Post   Remind Me!   Notify Moderator  
Rebecca1
Member


Reged: 01/07/04
Posts: 259
Loc: FL
Re: Test documentation when time is really tight...
      #299502 - 07/11/05 11:03 AM

Would the outline suffice for Sarbanes Oxley?

Post Extras: Print Post   Remind Me!   Notify Moderator  
Walen
Super Member


Reged: 05/09/01
Posts: 1254
Re: Test documentation when time is really tight...
      #299503 - 07/11/05 01:36 PM

Sometimes it is reasuring to know that you're not the only one in that boat, eh? I've used a similar approach to Linda's, with a couple of minor differences.

When we get down to a project like that, we'll develop a high level "path". We tend to use developers to exercise areas they did not develop, but who know the business processes well enough to be business analysts. The "path" begins as a combination guide and checklist to jog their memory and help them to work the process, rather than the application, for the first pass. Then, turn around and hammer limits, exceptions, etc.

During this time, the QA Lead will oversee what people are up to, and capture the process they are using, populate the "path" so it becomes a plan - or at least a record of what was done. If we need to repeat it, we know what we did the first time and can regenerate the process. If we add steps, they are captured and added to the plan.

This allows us to make sure the basic bones work - you can get from one screen to another doing "normal" activity, and when that is reasonable stable, detail people to start hammering item codes for widgets and component rules for them - focusing on the most vital first.

One great drawback is that you have a Lead who becomes a scribe instead of an active participant in the testing itself. The flip-side is that I know I always have a reserve that can be thrown in if needed, who is disciplined enough to capture the process used for a given suite while executing it.

It isn't exactly Hoyle, but the times I've needed it, it has worked pretty well.


Post Extras: Print Post   Remind Me!   Notify Moderator  
ljeanwilkin
Moderator


Reged: 01/13/03
Posts: 1182
Re: Test documentation when time is really tight...
      #299504 - 07/11/05 02:34 PM

Rebecca,

Yes. Every bug, enhancement, or new project that comes through has all test outlines (requirements) and resulting defects fully traceable back to the original documents and all testing can be recreated.

Also, *most* of our outlines have been expanded into test cases within a release or two (we have monthly releases). Using the outline to test is our absolute minimum and we only use that method when the time constraints make any other option impossible. It's just another tool in our toolbox. It's not our preferred method, but it gets the job done in a crunch, and (what is most important to us), there is no degradation in either the amount or type of testing performed.

- Linda


Post Extras: Print Post   Remind Me!   Notify Moderator  
rishi_m
Member


Reged: 05/25/05
Posts: 15
Loc: Mumbai
Re: Test documentation when time is really tight...
      #299505 - 07/28/05 04:22 AM

there isn't enough time for thorough testing

Which functionality is most important to the project's intended purpose?
Which functionality is most visible to the user?
Which functionality has the largest safety impact?
Which functionality has the largest financial impact on users?
Which aspects of the application are most important to the customer?
Which aspects of the application can be tested early in the development cycle?
Which parts of the code are most complex, and thus most subject to errors?
Which parts of the application were developed in rush or panic mode?
Which aspects of similar/related previous projects caused problems?
Which aspects of similar/related previous projects had large maintenance expenses?
Which parts of the requirements and design are unclear or poorly thought out?
What do the developers think are the highest-risk aspects of the application?
What kinds of problems would cause the worst publicity?
What kinds of problems would cause the most customer service complaints?
What kinds of tests could easily cover multiple functionalities?
Which tests will have the best high-risk-coverage to time-required ratio?


Post Extras: Print Post   Remind Me!   Notify Moderator  
rishi_m
Member


Reged: 05/25/05
Posts: 15
Loc: Mumbai
Re: Test documentation when time is really tight...
      #299506 - 07/28/05 04:24 AM

My simple opinion to your question is Read Requirement or Design Document word by word & Sentence By sentence and check in the Program that everything is done as client has mentioned.

Regards,
Rishi


Post Extras: Print Post   Remind Me!   Notify Moderator  
TestingClass
Newbie


Reged: 08/28/12
Posts: 7
Re: Test documentation when time is really tight... [Re: rishi_m]
      #719302 - 11/05/12 08:53 AM

Test functionality which has largest safety impact
Test functionality which has largest financial impact on users
Test most important functionality which projects intended purpose
Test most visible functionality to the user
Test most visible functionality which has Complex functions
Test most important feature in terms of customers view
Test functionality which has lot of bug fixes or updates
Test most important feature that are developed with new tools
Test most important feature that are most important to the stakeholders
Test functionality which is implemented in rush & panic mode, which may leads to more errors in the code.
Test functionality which has developed under extreme time pressure.
Test complex functionality which may leads to more errors in the code.

--------------------
-Software Testing Class
www.softwaretestingclass.com


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 4 anonymous users are browsing this forum.

Moderator:  blueinatl, AJ, michaeljf, swt88 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 4692

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5