The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Software Testing >> Exploratory Testing

Pages: 1
AUF_Testing
Newbie


Reged: 02/26/11
Posts: 18
Exploratory Testing and Record Keeping
      #698903 - 02/17/12 04:01 PM

What are folks' best experiences of maintaining long term records of the testing that has been carried out under exploratory testing?

As a bit of background, I've a small team of experienced testers (in testing, the systems being tested and the business itself) and we've all recognised the fact that more issues are uncovered through exploratory testing than through traditional scripted testing (not least because the developers can see what we've scripted and know to verify it themselves; which I still value) - the company is very risk adverse, is a stickler for documentation and has a tendency to only accept strictly formalised processes as opposed to more open processes.

The risk side I'm trying to handle through as much automated testing as feasible; however the other two points are proving stumbling blocks, so any experiences that can be shared would be appreciated.

Cheers


Post Extras: Print Post   Remind Me!   Notify Moderator  
MansoorShaikh
Member


Reged: 07/18/07
Posts: 193
Loc: Mumbai
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #698914 - 02/18/12 01:46 AM

You will never have enough time to do a full exploratory testing. I would recommend to go for Orthogonal Array Testing apart from other usual testing. This will help you identify test cases which have high coverage with minimum test cases.

Also, dont try to automate everything that can be automated. First identify the test cases which will add value once automated, eg, test cases which needs to be executed often, critical scenarios,etc.

Thanks,
Mansoor


Post Extras: Print Post   Remind Me!   Notify Moderator  
Walen
Super Member


Reged: 05/09/01
Posts: 1254
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #698979 - 02/20/12 05:41 AM

Quote:

What are folks' best experiences of maintaining long term records of the testing that has been carried out under exploratory testing?




Two questions that are related to each other -

How do /you/ define "Exploratory Testing?"
What "long term records" of testing are you concerned about keeping/maintaining? Why is ET different in your mind?

--------------------
P. Walen

My Blog: http://rhythmoftesting.blogspot.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
AUF_Testing
Newbie


Reged: 02/26/11
Posts: 18
Re: Exploratory Testing and Record Keeping [Re: Walen]
      #699135 - 02/21/12 01:05 PM

In my mind, exploratory testing is the process of 'preferably' experienced testers going beyond just what's scripted in front of them and testing the application through knowledge of the product and doing their analysis on-the-fly.

Now the company has historically expected every requirement formally documented (not really an issue for me, up until I need a requirement changed mid testing cycle) and every test script detailed in full with it linked to one or more requirements that it correlates to. Some of these test scripts are taking the testers hours to write, which could have been spent actually testing with some high level notes being taken so that the tests can be repeated.

What I'm concerned about is ultimately external auditing. Although, through writing this I'm coming round to the idea that external auditors don't care about the format of testing records, as long as there are some records - it would still be interesting to hear other people's views on the matter.


Post Extras: Print Post   Remind Me!   Notify Moderator  
MrBungle
Member


Reged: 05/11/09
Posts: 85
Loc: Canada
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #699285 - 02/22/12 04:32 PM

Why don't you try reducing the details in the scripts? Do you have steps in there?
If you do, then you could try putting test conditions only.

I'm facing a similar problem in my company. We're used to write test scripts so that anybody in the test team can execute them. I think it's a loss of time in most cases.

I'd rather save this time.
Later, if someone else needs to execute the tests, he may have to learn the functionality, and I admit this can be time consumming. But that won't always happen, and when it does, you get someone who knows what he's doing rather than someone executing steps in unknown territory.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Walen
Super Member


Reged: 05/09/01
Posts: 1254
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #700926 - 03/09/12 06:39 AM

I aplogize for the long delay in responding.

It sounds a bit like you have addressed part of the question in your own response. When you said "auditors don't care about the format of testing records, as long as there are some records" I find this generally true.

Griffon Jones has done some interesting work on the same topic for medical device and application testing. I am currently working on some research for financial services - and finding much the same results as Griffon has.

In short, keep records that would, in court, serve to convict you have having rigorously tested the application/system. There are tools available, more now than a few years ago, that can track what you actually do while testing an app that I find more usefull than written records - partly because they do not "forget" small steps - they record everything.

Additionally, paired-testing can also serve a similar purpose - where two people are working together to exercise a system, one is acting as a scribe recording what is done, and what the results appear to be - both are conversing as to what the next steps are.

These records have held up under audit where I've worked (when the auditor has a clue about either the environment or software development methodologies).

Remember the Function should drive the FORM - What you need should be more important than making it pretty.

--------------------
P. Walen

My Blog: http://rhythmoftesting.blogspot.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
ShmuelGershon
Newbie


Reged: 03/10/12
Posts: 1
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #701001 - 03/10/12 11:15 AM

In my experience, this is an area where text-based notes separated by testing sessions can help.
You can tie session notes to a test-execution in your test-management tool, or to a test-area in whatever artifact you use to track them.
The fact that they are text-based make them very easy to search in bulk (you can even let Windows Indexing do that for you), and make them very easy to handle for audition. If the notes format makes sense, it will be even simple for an auditor to review.

Exploratory Testing should be a great method for audited environments - auditors may prefer to see a documentation of what was actually done than a documentation based on what was planned to be done.

I use (my own) tool Rapid Reporter for that as it creates suitable organized files automatically, but any other way that will fit your way of working can fit - even Notepad if you can come up with a good way to organize the notes within the doc.
Note: My work context is not an audited environment, but I have learned from people who worked in such contexts.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Walen
Super Member


Reged: 05/09/01
Posts: 1254
Re: Exploratory Testing and Record Keeping [Re: ShmuelGershon]
      #701037 - 03/11/12 08:40 AM

Ummm - Actually Shmuel, I intended to mention Rapid Reporter and got side tracked.

--------------------
P. Walen

My Blog: http://rhythmoftesting.blogspot.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
dlai
Junior Member


Reged: 05/02/06
Posts: 1041
Loc: CA, USA
Re: Exploratory Testing and Record Keeping [Re: Walen]
      #705559 - 04/30/12 08:28 AM

You might want to look into James Bach's articles on Exploratory testing. http://satisfice.com/articles.shtml He did a lot of work evangelizing and formalizing the process.

You'll see stuff on test charters, outlining the areas you plan on exploring and what sort of things you are testing for. It can be something like, "I am testing the login module for odd behaviors when users try out of the normal inputs." (I'm over simplifing it here) The idea is a high level description of what you are doing so someone else taking your charter will be performing similar types of tests in the same area.

You'll also see alot of Session-based test management. How to time box your session, document your findings, what sort of conclusions you get. This is sort of like your test results.

--------------------
David Lai
Sr. QA / Test Lead
LinkedIn profile


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter RuscoeModerator
Veteran


Reged: 03/18/02
Posts: 7686
Loc: Tampa Bay
Re: Exploratory Testing and Record Keeping [Re: dlai]
      #705566 - 04/30/12 09:42 AM

Not a bad suggestion, David, but it doesn't address the original question.

Post Extras: Print Post   Remind Me!   Notify Moderator  
tonybruce
Advanced Member


Reged: 06/22/07
Posts: 532
Re: Exploratory Testing and Record Keeping [Re: Peter Ruscoe]
      #711856 - 07/10/12 06:53 AM

Both James Bach and Michael Bolton have written a lot about keeping records. Be a good idea to start there, if you haven't already.

Also, specifically some good info here, it's used for medical devices, I'm pretty sure risk would be a factor there. http://www.satisfice.com/blog/archives/602

James Lyndsay also has some great stuff http://www.workroom-productions.com/papers.html

In my own experience my ET is documented as I'd have a session report.

I however have never been audited.

There are plenty of ways to cover off the requirements, record coverage, etc which can work well with ET. Just need to try a few things and find what works for you

--------------------
We live in the dumbest world on the planet and there's no colour the sky can't be at any given time. Remember that!
http://dancedwiththetester.blogspot.co.uk/
http://twitter.com/tonybruce77


Post Extras: Print Post   Remind Me!   Notify Moderator  
AUF_Testing
Newbie


Reged: 02/26/11
Posts: 18
Re: Exploratory Testing and Record Keeping [Re: tonybruce]
      #713558 - 08/01/12 03:10 PM

I would like to say a massive thank you to the responses, and the links from Tony to the workroom-productions stuff have been awesome (now I just need more free time to finish reading them).

Cheers!


Post Extras: Print Post   Remind Me!   Notify Moderator  
tatashine
Newbie


Reged: 02/15/12
Posts: 14
Loc: US
Re: Exploratory Testing and Record Keeping [Re: AUF_Testing]
      #718555 - 10/22/12 07:53 PM

Rapid Reporter could help you out. http://testing.gershon.info/reporter/
It's lightweight, auto-narration your actions, and you could capture any screen as wanted with 1 click.

--------------------
a member of QASymphony. www.qasymphony.com
qTrace - Screen Capture Software with Intelligence
qTest - Cloud-based Test Management


Post Extras: Print Post   Remind Me!   Notify Moderator  
afaberes
Newbie


Reged: 11/21/12
Posts: 3
Re: Exploratory Testing and Record Keeping [Re: tatashine]
      #719913 - 11/21/12 01:45 PM

If you are looking at simple bookkeeping for auditbility, you could look at tools like https://github.com/ebrehault/resurrectio or https://chrome.google.com/webstore/detail/capycorder/niijdolnjmjdjakbanogihdlhcbhfkho that provide some simple recording of actions in a browser.

Of course, this provides recording at a low level (click, enter value, etc.) and you may want to use other annotation tools.

Aurelie
---------------------
http://www.qwushu.com


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 1 anonymous users are browsing this forum.

Moderator:  Peter Ruscoe 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 7203

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5