Automation Testing Approach
I am a bit surprised by what I think is the approach taken by the automation testing tools I have seen so far - granted I am fairly new to using them myself. I have spent the last few days beginning a study of how to better design in "Testability" into a future product. From looking at the tools out there - such as QTP, there really doesn't seem to be much support for it.
Current testing packages appear to be limited to Screen Scraping and SendMessage(ing). This method is lacking. I can see that if you are given a completed application and told to find a way to test as much as you can - it will get you pretty far. But you are left doing silly things like spinning in a tight loop, waiting for text to appear in some window etc.
In the QTP case, if that silly application had support for WScript, it could do nice things like register for events on my running application. Then I could directly notify script functions that I just completed some unit of work, it is now time for it to check for this property. Or I could pass the script internal errors for it to log, this would be very powerful.
It may be that there is more support for tighter integration that I am finding but the support staff I have been talking to are not able to lead me to it.
Thanks, just blowing off some steam
Re: Automation Testing Approach
While I haven't used QTP, your statements hold true for the products that I have used. I reckon if you have a scripting language already built into your app, adding a recording component that works at application rather than GUI level would create a stronger long term solution. My feeling is that many of the existing tools focus on black box testing of applications that have been developed without testability in mind. That said, check out AutomatedQA's open application mechanisms in TestComplete. In my own situation, which involves lots of 3d graphics, I think an internal recorder is the way to go. You might find this thread to be interesting in this regard.