The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Software Testing >> Automated Testing

Pages: 1
jimhazen
Super Member


Reged: 01/31/02
Posts: 2003
Loc: Highlands Ranch, CO, USA
Automation Myths, Lies & Scriptless/Codeless tools
      #718289 - 10/17/12 09:33 PM

As Dennis Miller used to say: "I don't want to go off on a rant, but..."

I have just got to get this off of my chest. Here we go again. I was in a Webinar for a new Automation tool vendor who was touting their tool as 'Scriptless'. It was a comparison of their tool and 'scripted' test automation tools/practices. How their tool could be used by anyone and that their Recorder technology would allow for 'quick' creation of tests. That and some of the other smoke&mirrors grape kool-aid they were selling was just disingenous and misleading. As you can tell it just set me off.

Earlier this year I was in a Webinar for a vendor who said their tool is 'codeless'. Again complete myths and lies.

My frustration is that every 5-7 years we are dealing with the idiots in Marketing/Sales at these companies that are selling Snake Oil. I've been working with automation tools for 20 years now, and I keep seeing this happen. So here we go again having to do damage control because of these FRACKING morons. And you know what sucks is that the cycle seems to coincide with people in management who don't know better or haven't learned their lesson about these issues.

Everyone wants that Silver Bullet to shoot while riding the magical Unicorn of misconception that they think is Test Automation. You know my favorite phrase for this is "It's Automation, not Automagic!" (I also use a more explicative one, but will get bleeped here).

I would like to list both companies names and tell you all to bombard them with emails calling them out for this crap. But I think most of you will figure it out.

So in the future if you hear about the Scriptless and Codeless tools and their silver bullets please feel free to call B.S. on them. Enough is enough already.

Ah... there, now I feel a bit better.

--------------------
Jim
-------------------------------------------
For all the general stuff to know about QA/Test go here http://www.softwareqatest.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
IanFraser
Super Member


Reged: 07/11/04
Posts: 2112
Loc: Brisbane
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: jimhazen]
      #718293 - 10/17/12 10:31 PM

As someone who worked on one of these tools I would say you are entitled to your views.

Here is the problem Sales driods distort the real real facts here. Scriptless in this case needs a definition.

If you have a test team with lots of manual testers and 1 to not many automation specialist the process of automation can be a slow one.

Add a "Scriptless" process for the manual testers and the time to automate drops drastically.

The key to these tools is an approach that involves the whole team in the creation of automated assets.

The tool I helped to develop for QTP had a "Write Once Principle" This where the tests are developed in the tool and from there they can be deployed to manual test, test management tools or to automated tests. They key is that there is one central place to maintain the test asset.

I was constantly frustrated by how sales driods presented the tool as a magic solution.

I no longer work for this company so I wont name the tool.

A friend and I prototyped this methodology a number of years ago and we saw it working and working very well.

I would be interested to hear who it was by PM.

--------------------
You can buy my Art from: "Post Cards now available"
Ian Fraser Landscape Photography
World Wide Shipping.

http://mowogman.wordpress.com/

Edited by IanFraser (10/17/12 10:31 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
jimhazen
Super Member


Reged: 01/31/02
Posts: 2003
Loc: Highlands Ranch, CO, USA
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: IanFraser]
      #718348 - 10/18/12 07:17 PM

Ian I will not condemn you for working on or with one of those tools. I am in total agreement about the condemnation of the Marketing/Sales people who continue this practice of Smoke & Mirrors.

I actually thought both tools had some nice features in them, and that this was a selling point for them. But they were highlighted in the wrong way in my opinion and experience. It was a disservice to their potential customers, and to those of us that have to go clean up the misperceptions set in place.

I have written some 'front-end' pieces for a couple of tools I've worked with over the years. But I never touted them as the silver bullet.

This is just a pet peeve of mine, and a recurring headache I've had to deal with for 20 years. Just makes me hang my head and shrug in disgust.

--------------------
Jim
-------------------------------------------
For all the general stuff to know about QA/Test go here http://www.softwareqatest.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
IanFraser
Super Member


Reged: 07/11/04
Posts: 2112
Loc: Brisbane
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: jimhazen]
      #718349 - 10/18/12 07:45 PM

Jim

You cant imagine how peeved I am after spending a good deal of time being invested and active in the development of tools and methodologies only to see the whole thing wasted by stupid people.

The goal was to provide a foundation to enable automation and improve productivity by allowing more people to be part of the automation side of the project.

What is interesting is a number of sites with an early excel based version of the tool are still activily using it. They key here is they have no in house automation expert but they are able to continue with the tool and develop and maintain automation.

The sites with "Automation" "Experts" the tool is no longer in use. Now one of my pet hates is automators who seem to want to play with their own frameworks and pretend they are productive.

I now work in the Ranorex world and these tools have just made a transistion to Ranorex. At this time I am not using this methodology in Ranorex.

I departed the circle of development almost a year ago for a number of reasons. I do believe the methodology has a future but only if it is presented honestly and openly... But we both know that will never happen while sales driods are involved.

--------------------
You can buy my Art from: "Post Cards now available"
Ian Fraser Landscape Photography
World Wide Shipping.

http://mowogman.wordpress.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: IanFraser]
      #718439 - 10/20/12 06:05 AM

Jim,
Interesting post, I haven't seen the tools you are talking about. Other tools I've seen advertised as being scriptless are Automation Anywhere or Worksoft but I have never seen them in action.
I would like to invite you to attend a webinar on our tool Automated Test and Re-Test (ATRT) which uses yet another scriptless approach to automated testing. We actually have developed a model driven approach to automated testing, and yes there is no scripting involved.
Please sign up for the webinar at http://idtus.adobeconnect.com/e4ufrv8r9nu/event/event_info.html - I definitely would be interested in getting your feedback to our approach, because I think we've come up with something pretty useful.
As you know I have been in automation for a fairly long time myself and I think we are on to something very user friendly with our ATRT solution.

If you can't make that date, we are recording the webinar and if you sign up we'll simply send you a link so you can watch it later.

Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
jimhazen
Super Member


Reged: 01/31/02
Posts: 2003
Loc: Highlands Ranch, CO, USA
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: Elfriede Dustin]
      #718443 - 10/20/12 03:08 PM

Elfriede,

Hi! Long time since our last chat. I'll check to see if I can make the webinar.

Offhand I would bet your 'scriptless' method would be a bit more real than what some are touting. And I say that because you mention using a 'Model' based approach, which is light years different than what the the hacks I saw promoting. So you I can believe, and also I know from past discussions that you (and probably anyone else working for/with you) will not present your system as a Silver Bullet.

Best to you.

--------------------
Jim
-------------------------------------------
For all the general stuff to know about QA/Test go here http://www.softwareqatest.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: jimhazen]
      #718461 - 10/21/12 06:55 AM

Thanks for the vote of confidence Jim - will send you the recorded webinar link or for now take a look at this

http://idtus.com/wp-content/uploads/2012/09/ATRT-Overview.mp4
(skip to about 1/3 into the video, so you can get straight to the capture portion)

Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
Corey_G
Veteran


Reged: 09/14/01
Posts: 4281
Loc: Boston, MA
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: Elfriede Dustin]
      #718464 - 10/21/12 09:59 AM

why is scriptless considered a feature or "good thing"?

a powerful programming language and API is a distinguishing feature of any tool I use.

--------------------
Corey Goldberg
Homepage: goldb.org
Twitter: twitter.com/cgoldberg
Google+: gplus.to/cgoldberg


Post Extras: Print Post   Remind Me!   Notify Moderator  
IanFraser
Super Member


Reged: 07/11/04
Posts: 2112
Loc: Brisbane
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: Corey_G]
      #718468 - 10/21/12 01:49 PM

Quote:

why is scriptless considered a feature or "good thing"?

a powerful programming language and API is a distinguishing feature of any tool I use.




Because not everyone in a test team can write code.

--------------------
You can buy my Art from: "Post Cards now available"
Ian Fraser Landscape Photography
World Wide Shipping.

http://mowogman.wordpress.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
FredMan
Super Member


Reged: 10/29/07
Posts: 1226
Loc: Lenexa, Kansas
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: IanFraser]
      #718517 - 10/22/12 07:43 AM

Hi Elfriede,

Does this tool test Microsoft's Dynamics AX applications?

--------------------
Onward thru the fog...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: FredMan]
      #718534 - 10/22/12 11:01 AM

Quote:

Hi Elfriede,

Does this tool test Microsoft's Dynamics AX applications?





Yes, the tool is entirely GUI technology independent.

Take a look at the demo or sign up for the webinar if you get the chance to hear more.

Thanks
Elfriede


Post Extras: Print Post   Remind Me!   Notify Moderator  
FredMan
Super Member


Reged: 10/29/07
Posts: 1226
Loc: Lenexa, Kansas
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: Elfriede Dustin]
      #718600 - 10/23/12 06:53 AM

Thanks, I'll do that.

--------------------
Onward thru the fog...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: FredMan]
      #719164 - 11/01/12 05:23 PM

by the way, here is a link to the webinar: https://idtus.adobeconnect.com/_a1091778...p;pbMode=normal

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
IanFraser
Super Member


Reged: 07/11/04
Posts: 2112
Loc: Brisbane
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: Elfriede Dustin]
      #719165 - 11/01/12 05:58 PM

Elfriede

I have looked at the presentation. The methodology and process look very interesting.

Is it limited to image type validations?
Can do actual text or data validations?
Can you apply data from external sources?

--------------------
You can buy my Art from: "Post Cards now available"
Ian Fraser Landscape Photography
World Wide Shipping.

http://mowogman.wordpress.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
broop
Member


Reged: 01/03/01
Posts: 195
Loc: Grandville, MI, USA
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: jimhazen]
      #719199 - 11/02/12 07:48 AM

Ha!

Hey Jim, you know the difference between a used car salesperson and a software salesperson don't you?

The used car salesperson knows when they're lying.


Don't let it get you down too much... unless you work for one of those said companies.

Bo


Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: IanFraser]
      #719223 - 11/02/12 06:02 PM

Hi Ian,


Quote:

Is it limited to image type validations?



No

Quote:

Can it do actual text or data validations?



Yes, the tool allows for data pooling, e.g. reading data from a file you can set up/ excel/.csv, etc.; OCR; etc.

Quote:

Can you apply data from external sources?



Yes, see previous answer; you can also type text into fields, command line, etc.

Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
jimhazen
Super Member


Reged: 01/31/02
Posts: 2003
Loc: Highlands Ranch, CO, USA
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: broop]
      #719232 - 11/03/12 04:48 PM

Broop,

No I do not work for any of the vendors. I have worked for partner consulting firms (HP/Merucry, Segue, IBM/Rational) that use those tools.

Nope I'm just an automation guy who wants to get work and do a decent job for my clients. I want to see automation projects succeed, and to reduce the problems due to misconceptions caused by the smoke & mirrors sales job.

--------------------
Jim
-------------------------------------------
For all the general stuff to know about QA/Test go here http://www.softwareqatest.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
mbrychuk
Newbie


Reged: 10/30/12
Posts: 8
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: jimhazen]
      #719524 - 11/08/12 05:53 AM

Elfriede, nice solution. More importantly, you have managed to meet multiple quite ambiguous requirements. I can offer couple suggestions that may be valuable for your user base:

1. Instead of "function" you can use work "keyword". This is more user oriented. It is also more aligned with the industry. The actual "function" is a piece of code that supports the "keyword" behind the scene.

2. Model driven approach for authoring implies "learning of a new way of doing things", i.e. "change". Granted, with any solution there will be an element of learning. What can make the solution more adoptable by general Testers community is the reduction of required learning (change).

The normal way of working for testers is a test case mode like this:
Step #, Location, Action, Data, Expected Result.

You can make your solution to accommodate this mode, and reduce the element of change by allowing Users to author test cases in this format.

A few benefits are: less change for people who have to adopt the solution, easy automation of existing manual test cases, and reusability of test cases for both automated and manual testing.

You can achieve this in your tool, or add it as an alternative feature. The challenge will be to simplify the authoring process, i.e. scaling back the urge to innovative :-)

3. There is are good reasons why UI and back end testing tools are usually different solutions. I would try to educate the client and swing their requirement on this. The potential danger is they would expect one tool to handle well both testing types. Putting too many eggs in one basket may inflate client expectations, and present a challenge for you down the road.

4. I didn't see the capability of the tool to test System of Systems and Large Data Analysis. I'd be very careful in managing client expectations on both of these fronts. These requirements go well beyond what you can do with image based testing. These are the areas where the cost of automation can skyrocket. This is where the scriptless approach may turn into not so much scriptless.

Generally speaking, very nice tool from what I saw in presentation. Thank you for sharing the webinar link.

We have designed simpler solution. It does meet all DOD requirments stated in your case, but may offer some good ideas: http://sqta.wordpress.com/test-automation-framework/


Post Extras: Print Post   Remind Me!   Notify Moderator  
mbrychuk
Newbie


Reged: 10/30/12
Posts: 8
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: mbrychuk]
      #719525 - 11/08/12 05:56 AM

Ooops, correction to the recent post. The example solution that I provided the link to does NOT meet DOD requirments.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Elfriede DustinModerator
Moderator


Reged: 12/28/99
Posts: 1351
Loc: Washington, DC
Re: Automation Myths, Lies & Scriptless/Codeless tools [Re: mbrychuk]
      #719631 - 11/10/12 10:12 AM

Thanks for your feedback. Good to hear you like ATRT.

Here are responses to some of your points:

re:1 ..Use keyword instead of function. Well, a function in ATRT is a group of test steps that can be reused over and over again in different parts of any test case via reference (by just referencing the function) - a keyword is something different. So I really do mean function.

re: 2 I can see your point, but also I believe testers can adjust to slight changes in terminology, yet we definitely want to remain flexible here.

re: 3 your comment..."there are good reasons why UI and back end testing tools are usually different solutions." It depends, right? There are certain test scenarios where combining both would be useful. ATRT allows users to test the UI, takes a UI action and then verifies the expected result on the backend (if that's what you want your test case to do)

re: 4 wondering about systems of systems and large data analysis... Good point. I didn't demo that in the webinar, but good feedback so can I make this clearer in a follow-on webinar. One of ATRT's strength is system of systems automation and automated data analysis. Stay tuned for the next webinar - Planning one for Jan 23, 2013

In the meantime if you prefer to see for yourself sooner and if you are US citizen you can actually request an eval license here http://idtus.com/products/atrt-test-manager/

Thanks again for your valuable feedback,

Elfriede

--------------------
Elfriede Dustin
My amazon blog
Solving your Automated Software and Security Testing problems
my twitter handle @ElfriedeDustin


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 8 anonymous users are browsing this forum.

Moderator:  Elfriede Dustin, AJ, Joe Strazzere 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 17071

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5