The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Software Testing >> White Papers and Articles

Pages: 1
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
What's Your Automation Index?
      #554943 - 03/06/09 04:06 AM

Just because a test is automatable doesnt necessary mean it should be automated, instead calculate your Automation Index, see What's Your Automation Index?

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: What's Your Automation Index? [Re: Elfriede Dustin]
      #554967 - 03/06/09 04:39 AM

I like the checklist, Elfriede. But, I don't think I understand the value of the Automation Index.

Automation Index = Number of test cases feasible to automate / Total number of test cases

To me, everything is feasible, but not always practical. Wouldn't you agree?

This Index applies to the overall project, right? If so, how does it help you decide whether or not to automate any particular test?

I'd worry that some well-meaning but misguided Manager would decide "We need to increase our Automation Index", or even worse "We must achieve a 95% Automation Index by the end of the year", or such.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: What's Your Automation Index? [Re: Joe Strazzere]
      #554992 - 03/06/09 05:57 AM

Quote:

I like the checklist, Elfriede. But, I don't think I understand the value of the Automation Index.

Automation Index = Number of test cases feasible to automate / Total number of test cases

To me, everything is feasible, but not always practical. Wouldn't you agree?

...




Joe, Exactly my point. Hmh, was that not clear reading the entire article? That's really the point of it....after the initial formula...everything is feasible but not always practical.
Hopefully people don't stop reading after seeing the formula.

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: What's Your Automation Index? [Re: Elfriede Dustin]
      #555004 - 03/06/09 06:55 AM

Quote:

Hmh, was that not clear reading the entire article?




Sorry, no - not for me. Perhaps the word "feasible" was the source of my confusion.

Anyway - I liked the checklist part. Still not sure I understand the value of the Index.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: What's Your Automation Index? [Re: Joe Strazzere]
      #555176 - 03/07/09 06:26 AM

Quote:

... Still not sure I understand the value of the Index.




The automation index is to be used as a general indicator that shows whether your tests lendselves to automation and how many, if any at all; it provides an general gauge for test automatability and related ROI.
As part of an outcome of intial project assessment the resulting automation index is useful for us when trying to determine if the tests and what portions of the test effort are automatable.

Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
Turbografx
Super Member


Reged: 10/21/05
Posts: 1756
Loc: London, U.K
Re: What's Your Automation Index? [Re: Elfriede Dustin]
      #557502 - 03/18/09 08:45 AM

Hmmm... I'm with Joe on this. I like the checklist, but not sure what extra the index gives us... Sounds a bit like the bugs found matrix.... open to abuse.

If each item on the checklist was ranked (like TPI Test Maturity Matrix)could produce some pretty graphs


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede Dustin
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: What's Your Automation Index? [Re: Turbografx]
      #566517 - 05/02/09 09:36 AM

Quote:

Hmmm... I'm with Joe on this. I like the checklist, but not sure what extra the index gives us... ...




The index is a general gauge / metric that describes the number of automatable tests as a subset of all planned tests.
Most metrics are open to misuse or misinterpretation, so be specific about how the automation index (AI) was derived, e.g. you could provide with it the associated automation feasibility guidelines and explanation/criteria that helped determine test automatability. This way the AI can be understood given its context.

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
keshavashukla
Member


Reged: 07/15/08
Posts: 153
Loc: Mumbai,India
Re: What's Your Automation Index? [Re: Elfriede Dustin]
      #593590 - 09/29/09 10:38 AM

just a query : can we go for automation , even if the index is Low , how to determine the cutoff below which we should never recommend automation

--------------------
@;-
To err is to CODERS , to TEST is divine

Keshava


Post Extras: Print Post   Remind Me!   Notify Moderator  
QAAutomation
Advanced Member


Reged: 02/12/03
Posts: 542
Re: What's Your Automation Index? [Re: Elfriede Dustin]
      #593645 - 09/29/09 04:49 PM

Quote:

Quote:

Hmmm... I'm with Joe on this. I like the checklist, but not sure what extra the index gives us... ...




The index is a general gauge / metric that describes the number of automatable tests as a subset of all planned tests.
Most metrics are open to misuse or misinterpretation, so be specific about how the automation index (AI) was derived, e.g. you could provide with it the associated automation feasibility guidelines and explanation/criteria that helped determine test automatability. This way the AI can be understood given its context.




I completely agree with above post. Most of the times we keep talking about automation framework and then with very few dedicated automated resource in the team, we fail to make that automation framework visible to the business sponsors or to the management. Why? Because we have no metrics associated to the automation framework and that is where I guess bringing in the concept of AI will complement the framework and also make it accountable @ project & application level.


Post Extras: Print Post   Remind Me!   Notify Moderator  
ViplavAnand
Newbie


Reged: 02/08/11
Posts: 24
Re: What's Your Automation Index? [Re: QAAutomation]
      #678406 - 07/12/11 09:39 PM

Link seems not to be working.

--------------------
Regards,
http://www.articlesontesting.com


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: What's Your Automation Index? [Re: ViplavAnand]
      #678522 - 07/13/11 04:32 AM

If you use Google, you'll find other sources of this article.

Such as:
http://www.softwaretestpro.com/Download/HtmlPublication/208
http://www.testpoint.com.au/attachments/093_What's%20Your%20Automation%20Index.pdf

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 1 anonymous users are browsing this forum.

Moderator:  AJ, Jeanj 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 10937

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5