The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Software Testing >> White Papers and Articles

Pages: 1
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Agile Testing Fact and Fiction
      #519189 - 09/23/08 11:12 AM

A good summary: Agile Testing - Fact and Fiction see http://www.sdtimes.com/content/article.aspx?ArticleID=32884

-- Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
philk10
Advanced Member


Reged: 01/20/05
Posts: 578
Loc: UK
Re: Agile Testing Fact and Fiction [Re: Elfriede Dustin]
      #519492 - 09/24/08 05:04 AM

"Agile projects are an excellent opportunity for QA to take leadership of the [testing] processes,"

Shouldnt QA have leadership of the testing processes anyway, agile or not ? Who has control of the testing processes on waterfall ?

Myth 4
"By automatically capturing the testers process and documenting their keystrokes and mouse clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating " - sounds like record/replay which means the tester will be spending all their time repairing broken scripts when the GUI changes

--------------------
and My Blog


Post Extras: Print Post   Remind Me!   Notify Moderator  
DSquared
Moderator


Reged: 04/02/03
Posts: 4546
Loc: Wisconsin, USA
Re: Agile Testing Fact and Fiction [Re: philk10]
      #519501 - 09/24/08 05:19 AM

"Who else is better placed to bridge the gap between users and developers, understand what is required, how it can be achieved and how it can be assured prior to deployment?"

Ummmm...the business analyst?????? If not, what does a BA do?

His comment about TDD not being sufficient and you need to include white box, black box, regression, etc. Clearly, he doesn't understand the difference between process (TDD) and methodology or implementation (TDD via whitebox, blackbox, etc).

I found the article full of inaccuracies and leaps of logic that are just not supported. I'm not even sure where he came up with his list of myths. I've never heard them expressed in any agile project I have ever been involved with.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Agile Testing Fact and Fiction [Re: DSquared]
      #519598 - 09/24/08 08:53 AM

Quote:

....
I found the article full of inaccuracies and leaps of logic that are just not supported. I'm not even sure where he came up with his list of myths. I've never heard them expressed in any agile project I have ever been involved with.




Please provide him specific examples of the inaccuracies.
Let's give him constructive feedback and avoid high level bashing of the entire article. I for example, have heard about some of those myths. I am sure others have, too.

Thanks
Elfriede


Post Extras: Print Post   Remind Me!   Notify Moderator  
DSquared
Moderator


Reged: 04/02/03
Posts: 4546
Loc: Wisconsin, USA
Re: Agile Testing Fact and Fiction [Re: Elfriede Dustin]
      #519842 - 09/25/08 05:39 AM

The two I provided are very specific.

I would also suggest that he source the myths. You have heard the myths, I have not. If they were sourced, that would answer the question.

Caveat: I'll limit my comments to Agile Scrum, since that is the flavor of Agile that I have used and been involved with. I will further limit my comments to my specific experience with Scrum. Others may have entirely different experiences.

Quote:

September 23, 2008 If your company is practicing agile development, you might have been faced with some of the same challenges as George Wilson. As Business Group Manager at AIG Computer Services, he simultaneously managed IBM mid-range and PC development projects in a TickIT management environment, which requires ISO 9001 QA certification. Now, as co-founder and general manager of test automation tool maker Original Software, he preaches agile practices as the best means to achieve quality.

"Agile projects are an excellent opportunity for QA to take leadership of the [testing] processes," Wilson says. Rather than take a back seat while developers drive the ship, he recommends that testers take the lead. "Who else is better placed to bridge the gap between users and developers, understand what is required, how it can be achieved and how it can be assured prior to deployment?" This requires that the QA team be fluent in agile themselves, so Wilson offers the following facts to debunk some common agile testing myths:



First, QA or developers do not take the lead. The scrum master does. It is up to the scrum master to ride herd on the myriad talents required to bring the project to successful completion. The concept of QA leading or Developers leading is an archaic left-over from older SDLC methodologies. In Scrum, everyone is equal. I've already addressed the QA versus BA question.

Quote:

Myth One: You only need to unit-testTDD testing is sufficient
Test Driven Development is a good start, but for those who think it's all there is, "for the vast majority of commercial developments this simply isnt true. Even strong proponents of agile development recognize the need for their armory to include a range of testing techniques...including white box, black box, regression testing, stress testing and UAT," said Wilson.

Therefore, the most effective agile projects will include investigative (black box) testing efforts that support (rather than compete) with white box testing. "Good investigative testing will reveal problems that the developer missed before they get too far downstream," said Wilson.



As indicated earlier, I have NEVER heard this one. No one has ever told our teams that TDD is sufficient. I've also addressed the blurring of process (TDD) with implementation methodology (black box, white box, etc). Where was it said that TDD is not black box or white box? Or for that matter, where is it said that you have a choice of black box OR white box, but not both? We have used white box, black box, scripted, manual, automation, exploratory, unit, functional, user acceptance, and any other type of testing that you can name, all in the same project. The point is that Wilson is drawing a line where none exists.

Quote:

Myth Two: You can reuse unit tests to build a regression test suite
Has anyone ever told you that conventional downstream testing is unnecessary because every line of application code has a corresponding test case? "Some TDD proponents ... suggest that by reassembling unit tests, everything from User Acceptance Tests to Regression Tests can be performed," said Wilson.

While this may sound feasible, Wilson says it isn't realistic because the granularity and objectives of white box unit tests developed in TDD serve a different purpose than downstream black box testing. "While the overall objective of a unit test is to prove that the code will do what is expected, the aim of regression testing is to ensure that no unexpected effects result from the application code being changed. These two objectives are not synonymous. Checking that an attribute has a valid date format [for example] is not the same as checking that for a given input, the value of the field contains an expected date."



This myth appears to assume that TDD is the basis for development. If that assumption is not true, does this myth still hold true?

Quote:

Myth Three: We no longer need testers or automation tools
Not true. Testers perform a "different and equally valid role from their development colleagues," says Wilson, and should therefore be have an equal place at the project table. "It is [also] widely recognized that traditional test automation tools have failed to live up to the hype of the vendors." Perhaps vendors (no offense, Original) should play down the hype.



Contrary to this, Scrum almost requires automation and skilled testers to build the automation. At least in my experience. We had the teams literally begging us to join the team to automate the testing.

Quote:

Myth Four: Unit tests remove the need for manual testing
Few would disagree that manual testing is repetitive, expensive, boring and error-prone. And while TDD can reduce the amount of manual effort needed for functional testing, it does not remove the need for further manual or automated black box testing. "By automatically capturing the testers process and documenting their keystrokes and mouse clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating. Though manual testing is a time-consuming (and therefore expensive) way to find errors, the costs of not finding them are often much higher," said Wilson.



Automation is based on a good manual test. If you have garbage testing as a basis, you end up with automated garbage. It is also stated that TDD can reduce the amount of manual efforted needed for functional testing. Really? Where is the data to support that statement? He also states that manual testing is a time-consuming (and therefore expensive) way to find errors. This statement implies that automation is less time-consuming and less expensive. That is patently absurd. There are tons of data that show that the initial automation effort is much more time-consuming and expensive than the equivalent manual test. In addition, manual testing (along with exploratory) can find new defects, whereas automation can only find the defects that the test is coded to find.

Quote:

Myth Five: User acceptance testing is no longer necessary
In his experience with agile development, Wilson says that acceptance testing is often positioned as working with the customer to resolve incorrect requirements rather than correcting functionality that does not map to what the user needs. And maybe it's actually both. "When the user initially defines their requirements, they do it based on their initial expectations. When they see the system 'in the flesh,' they will invariably come up with different or additional requirements," he says.

While the agile process might reduce the frequency of this happening, Wilson believes it is unreasonable to expect the approach to resolve them entirely, so there should be no expectation that user acceptance testing will be obviated. "This is especially true when it comes to the business user signing off approval of the user interface, since they may have envisaged something different from the developer."



User acceptance is for BOTH requirements validation and functional validation. To expect that the requirements are solid and pre-validated before development is again a hold-over from older SDLC methodologies. Agile is based on the assumption that there will be changes to the requirements as the project matures.


Post Extras: Print Post   Remind Me!   Notify Moderator  
martinh
Moderator


Reged: 02/14/01
Posts: 1087
Loc: Melbourne
Re: Agile Testing Fact and Fiction [Re: DSquared]
      #574258 - 06/11/09 06:12 PM

I am finding it a little difficult to see where the confusion is coming from but it is definitely there. Either Mr Wilson or the author of the article, (or both?) has a narrow view of testing and agile.

I agree with the earlier comment that the reference to record-playback should have been picked up and dealt with in the article. Why wasn't it? Is it because the author doesn't realise?

I came away from reading this feeling that both the author and Mr Wilson are looking at Agile testing using Waterfall minds. If anything, they are replacing these 'Myths' with myths of their own...

--------------------
"Not every solution was derived to address an obvious problem" - Me (quite recently indeed)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 1 anonymous users are browsing this forum.

Moderator:  AJ, Jeanj 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 8484

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5