The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:
Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Testing Tools >> HP Functional Testing / Mercury QuickTest Pro / QTP

Pages: 1 | 2 | 3 | >> (show all)
BoBoMonkey
Member


Reged: 06/09/05
Posts: 246
QTP Framework VS Custom Framework
      #330527 - 09/18/06 06:20 AM

Hi all,

Today we had a surprise visit from our VC. Our QA head wanted to give him a taste of our Automation Effort. So was asked to run a few test cases. I Started the Demo by telling him that we have a Custom Hybrid Framework For working with QTP. Our Framework reads the test cases that are written in a excel sheet and interprets that at runtime into UI action.

When i showed him the excel sheet containing the test cases he interupted me and asked me why we needed such a Custom framework when QTP Provides an in-built Keyword driven framework.

I expliained by giving the usual argument that excel sheet acts an abstraction between the tool and the manual tester(who doesn't understand code or the tool). Later I brought up reusability, maintainability and many such points.

But he kept on asking me the same question "why build a custom framework when one is provided by QTP?"

To start with I did not know that this gentleman was aware of the QTP keyword view (normally VC's won't know this!!)and secondly I am still not able to find a definitive answer to his question. I could not convince him at all and felt really embaraced.

The question is NOT "Why use a Framework" but the question is "why NOT use a Framework that comes in-built in QTP?"

The QTP keyword view as i can see is not so different from an excel sheet. So why not use it and save the pain of building a custom hybrid framework?

So guy's if you are using a custom framework and have good points in favour-of such a framework (or if you feel otherwise) please share them with me.


Post Extras: Print Post   Remind Me!   Notify Moderator  
jbotash
Guru


Reged: 01/19/00
Posts: 2009
Loc: USA, CA
Re: QTP Framework VS Custom Framework [Re: BoBoMonkey]
      #330533 - 09/18/06 06:59 AM

QTP DOES NOT have built-in framework.
It has implemented functionality to build it by automator.
Period.
Even more: there is no abstract FrameWork in the world that can be used as is for every particular application.
That's why QA automation exists: to implement custom FrameWork, custom automation functionality - using automation tool.

Your post can be used as guidline -
what automation approach SHOULD BE.

Good luck to you.
You are absolutelly correct.

Regards,
Janna


Post Extras: Print Post   Remind Me!   Notify Moderator  
Yury
Advanced Member


Reged: 12/31/99
Posts: 649
Loc: Toronto
Re: QTP Framework VS Custom Framework [Re: BoBoMonkey]
      #330549 - 09/18/06 07:48 AM

Quote:

So guy's if you are using a custom framework and have good points in favour-of such a framework (or if you feel otherwise) please share them with me.


And what were your reasons for implementation of this particular framework?
Could you share them?


Post Extras: Print Post   Remind Me!   Notify Moderator  
tonymro
Advanced Member


Reged: 07/19/99
Posts: 542
Loc: Coast Range Mountains of Orego...
Re: QTP Framework VS Custom Framework [Re: jbotash]
      #330561 - 09/18/06 08:02 AM

QTP does not provide or include a framework, and the "key-word view" should not be misconstrued as a keyword-driven framework. QTP's "key-word view" merely abstracts some of the built-in commands to a dumb-downed keyword style display.

Not all "custom" frameworks implement or support action-word or keyword-driven test scripts. You'll find several threads in the "Automated Testing" forum that discuss the pros and cons of keyword-driven vs. non keyword (function/method library) frameworks.

--------------------
Sr. Software Test Automation Architect


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: tonymro]
      #330580 - 09/18/06 09:17 AM

Whilst were on the topic of the KeyWord View, does anyone actually use it?

To me it's a marketing or sales gimmic, does it add any "real" value to the product.

Only use I made of it was when I quickly flicked to it as a lazy way to syntax check my code but v9.0 has a syntax checker button so I won't need it when v9.0 + fixes is issued.

Mark Smith.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Maestro_007
Member


Reged: 03/08/05
Posts: 27
Re: QTP Framework VS Custom Framework [Re: tonymro]
      #330583 - 09/18/06 09:21 AM

As far as I can tell, the VC was asking about Quality Center, not the Keyword-view. I think either he's mistaken about QuickTest's abilities, or you and he need to agree on the definition of the word "Framework". I mean, I can interpret that word to mean a number of different things.

Say, for example, when VC says "Framework", he means "wrapper around your object model and scripts which allows a test case to be executed against the AUT and results to come back in a simple format", then you are using QTP's built-in framework.

But by "Framework", I take it that you mean "wrapper around the suites of test cases which will execute multiple tests in an arbitrary order, then report all the test results back to a dasbboard-like interface". If you mean that, QTP most definitely does not provide it in situ. For that you either need the multi-test manager, or you need Quality Center.

If he thinks you have licenses for QC and that you're not using them, that may be it. If he truly doesn't understand the capabilities of QTP out of the box, I think it's a simple miscommunication.

Over at the Inquisition, we're slowly but surely developing our way out of needing QC licenses, and we're trying to show everyone else how to implement their own version.

We've implemented what we call a "Framework", which allows non-testers to develop and execute tests and then report results back to email or a 3rd party project management tool. It sounds like a similar thing. I assure you, there would be no way to implement this without writing some serious code yourself or buying QC.

Multi-test manager is pretty cool, though, in case you haven't checked it out. All but the smallest companies would outgrow it in about a week, but it is something...

--------------------
http://www.softwareinquisition.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: Maestro_007]
      #330589 - 09/18/06 09:28 AM

Not sure you can use Multi-Test manager & Framework in the same sentance. Framework & scheduler cannot be confused even by a VC

Mark Smith.


Post Extras: Print Post   Remind Me!   Notify Moderator  
lynneM - RIP
Moderator


Reged: 02/02/03
Posts: 3101
Loc: FL, USA
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #330593 - 09/18/06 09:32 AM

The VC probably got his info from a Mercury Marketing/Sales Rep. The information given is not truly accurate, will have been mis-interpreted by an executive and it will be almost impossible to replace his "knowledge" with reality.

--------------------
Lynne

I have not failed. I've just found 10,000 ways that won't work" --Thomas Edison


Post Extras: Print Post   Remind Me!   Notify Moderator  
prtester
Active Member


Reged: 05/09/06
Posts: 1059
Loc: Denver, CO
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #330605 - 09/18/06 10:26 AM

Quote:

Whilst were on the topic of the KeyWord View, does anyone actually use it?

To me it's a marketing or sales gimmic, does it add any "real" value to the product...

Mark Smith.




Hi Mark,

I agree, it's totally a sales gimmick, and it works. All the time I see QA shops that have purchased this package based on what they believe they can do with the keyword driven, record and play testing...without having trained VBscript coders in house, without paying the premium for the trained automation developers.

Most, but not all of these are part of a larger non-IT business and their "testers," if they are even called that, are pulled up out of other non-IT departments because of their inside knowledge of the business.

These folks, and the wandering corporate execs who are used to making decisions on areas where they have no expert knowledge, are vulnerable to the amount of energy that Mercury has put into this gimmick.

But I do believe that QTP has a great deal of built in power that gets shoved to the side because it isn't understood by the automation experts that come in and just use it as an engine to run their own code.

I suspect if there was more openess to outside QTP manuals and trainers, if it were easier for QTP experts to spread their knowledge, then that powerful middle ground would get more of the exposure and use that it deserves.

tester

--------------------
see my blog: www.sqablogs.com/rainbowtesting


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: prtester]
      #330755 - 09/19/06 04:22 AM

Couldn't agree more.

What the salesmen forget to mention is that you still need an experienced resource to create & maintain the code, functions etc before the non-QTP resources can select these items from the keyword view

I wrote a framework that will parse any XML message in HL7 format to any of our subsystems. All the non-QTP "testers" need to do is fill out a row in the datatable with the XML address, delivery channel, expected results (payload contents, error messages etc) but they can't get that right most of the time.

I remeber a while back a company called Omsphere were marketing a "Framework" for QTP but haven't heard much about it in the last year (probably because people found it easier/cheaper to build their own).

Mark Smith.


Post Extras: Print Post   Remind Me!   Notify Moderator  
ppat7046
Active Member


Reged: 02/01/01
Posts: 785
Loc: USA
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #330803 - 09/19/06 06:57 AM

Hi BoBoMonkey,

Here is my reasons why we should use custom framework.

1)
I created some of my script using QTP built-in Object Repository on my PCs C drive.
Now management decided that everyone should able to log into a network via Citrix server, OPEN QTP there and execute different scripts.
But, could not execute and I got error message Action 1 not found.
The reason all the script I have created into my PC on C drive.
If I did not have used QTP built-in Object Repository and have used custom framework then I could have execute script from network.

2) You can start coding even when your application is not ready
Lets take the Login page as an example. The descriptions of the Login page will be
Code:

Object Type Property Property Value
----------- --------- ---------------
Browser title Scope
Page name Scope
WebEdit name User Id
WebEdit name Pwd
WebButton name Submit


Based on the above descriptions you can create Dynamic Object Repository as below.
B="title:=ABC"
P="name:=XYZ"
Set BP=Browser(B).Page(P)
LO_UserId_WE="name:=User Id"
LO_Pwd_WE="name:=Pwd"
LO_Submit_WB="name:=Submit"

The script could be:
BP.WebEdit(LO_UserId_WE).Set "Prashant"
BP.WebEdit(LO_Pwd_WE).Set "Patel"
BP.WebButton(LO_Submit_WB).Click

3) No need to keep same types of objects
For example: Page 1, Page 2, and Page 3 have Submit, Cancel and Reset buttons.
When QTP learns the objects it captures three objects for each page, totalling 9.
However with Dynamic Object Repository you have to declare only once, totalling 3.

4) You can make changes to Dynamic Object Repository without making a change to the script. For example: If object property changed then script does not need to be changed.

--------------------
Thanks,
Prashant Patel

Edited by ppat7046 (09/19/06 06:58 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Yury
Advanced Member


Reged: 12/31/99
Posts: 649
Loc: Toronto
Re: QTP Framework VS Custom Framework [Re: lynneM - RIP]
      #330806 - 09/19/06 07:03 AM

Quote:

The VC probably got his info from a Mercury Marketing/Sales Rep. The information given is not truly accurate, will have been mis-interpreted by an executive and it will be almost impossible to replace his "knowledge" with reality.


I agree that the biggest problem in GUI test automation is a "marketing fog of war" created by vendors' sales people. It looks like this misinformation completely surrounded higher managers.

On the other hand test automation people often start implementation of a pretty complex framework without clear understanding of its benefits, drawbacks and prerequisites.
I was concerned that this was the case here because BoBoMonkey was not able to articulate his reasons for implementation of a particular framework.

BTW, what is "VC"? Is it Vice Consul?


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: Yury]
      #330810 - 09/19/06 07:12 AM

Probably Vice Chairman (similar to VP).

Mark Smith.


Post Extras: Print Post   Remind Me!   Notify Moderator  
BoBoMonkey
Member


Reged: 06/09/05
Posts: 246
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #330914 - 09/19/06 10:29 PM

yes, It's vice chairman.

Guys I thank you all for participating in the thread.
Although I am a strong proponent of automation framework I am going to be as objective as

possible whn replying in this thread.


before i write my reply I would like to share some details about the custom framework that i

have built.

It's a hybrid framework ( keyword + Bussiness action driven). I have designed it so that it

can act as a layer of abstraction between the manual tester and the tool. Please keep in

mind that the target user will not have any development/coding background.

The manual tester in order to create a automation script would have to type keywords in the

excel sheet which is in the form

Code:

keyword ComponentID Testdata status
entertext username bobomonkey
entertext Password password1
ClickButton Login
CreateMatter TestTitle|trademark|USA
verifyStringonPage TestTitle matter created



The excel sheet get interpretted at runtime and the actions and bussiness actions get

trigerred.
The Status column carries the pass/fail of each step and gets updated at runtime. For a test

cases the pass al the verification steps (not shown need to pass.
The result file is the same excel file contating Pass/fail status of all steps and

testcases.
Also the the hierarchy is
Testsuite-->Testmodule-->Testgroup-->Testcase
I can control what to run.


These are the points that i could gather after reading your replies:
Custom Framework is better than keyword view because:

1) No need to TOOL-Train manual tester.
2) it's easier to type in the excelsheet than work in the keyword view
3)No need for QC
4)Duplicate objects in OR (redundancy)

Adding to this, I thought of some more points:

5) No need to store static script. all logic in XL sheet that gets compiled at runtime.

6)No need to use QTP object Repository. could use dynamic repository (created in runtime)

But again I can't find that argument that could seal the deal in favour of Custom Framework.

All the above points can be explained away and workarounds could be found.

for example point 1 is thoughtworthy, In case of static scripts (point 5) one could argue

that there isn't any major Disadvantage of storing test cases in the form of static QTP

scripts.
Duplicate objects (point 4) could be avoided by editing the OR. Even if Duplicate objects

(save button for different Page) exists in the OR it isn't a huge problem.
Also One could use a separate OR for each module instead of using single OR.
No need for QC (point 3) doesn't help much either, Why not use the Damn thing (QC) when it

comes free with QTP.
(Point 2) is arguable.
(point 6) isn't compelling enough.


So far I haven't heard a compelling argument that tilts the poll towards custom frameworks.

I hope to get more replies before i present my case to the VC.

Edited by BoBoMonkey (09/19/06 10:31 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: BoBoMonkey]
      #330947 - 09/20/06 02:11 AM

Quote:

:) 3)No need for QC





How does your framework negate the need for QC

Mark Smith.


Post Extras: Print Post   Remind Me!   Notify Moderator  
BoBoMonkey
Member


Reged: 06/09/05
Posts: 246
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #330956 - 09/20/06 02:33 AM

well, I have planned to create HTML LOG files for test run , there will also be summary reports at testSuite, testModule level. I am planning to publish them on the intranet where they can be tracked.
offcourse i can't handle all features of QC with my framework

Edited by BoBoMonkey (09/20/06 04:00 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
tonymro
Advanced Member


Reged: 07/19/99
Posts: 542
Loc: Coast Range Mountains of Orego...
Re: QTP Framework VS Custom Framework [Re: BoBoMonkey]
      #331077 - 09/20/06 07:44 AM

Quote:

3)No need for QC

How does your framework negate the need for QC




By QC, are you referring to Quality Center?

--------------------
Sr. Software Test Automation Architect


Post Extras: Print Post   Remind Me!   Notify Moderator  
mwsrossoModerator
Veteran


Reged: 09/30/01
Posts: 4974
Loc: Doncaster, UK
Re: QTP Framework VS Custom Framework [Re: tonymro]
      #331111 - 09/20/06 09:55 AM

That's what I assumed Tony.

Could never envisage negating the need for QC at my current client as it's almost woven into the fabric of the company.

Wonder how he updates the run stats in the test lab (by hand ) Crikey! if he uses test factory code does that mean he hasn't really negated the need for QC.


Post Extras: Print Post   Remind Me!   Notify Moderator  
BoBoMonkey
Member


Reged: 06/09/05
Posts: 246
Re: QTP Framework VS Custom Framework [Re: mwsrosso]
      #331246 - 09/20/06 10:14 PM

mwrosso, QC to me is just a tracker there are a million different ways to substitute that.

Now staying with the topic can anyone think of more advantages of custom framework.


Post Extras: Print Post   Remind Me!   Notify Moderator  
IanFraserModerator
Super Member


Reged: 07/11/04
Posts: 2112
Loc: Brisbane
Re: QTP Framework VS Custom Framework [Re: BoBoMonkey]
      #331253 - 09/20/06 10:48 PM

As others have said QTP out of the box doesnt have a framework. What they do have is pretty features that impress managers who dont know anything about real automation.

A good framework allows users who dont know anything about automation to create automated test cases.

A good framework requires very little maintenance compared to traditional automation practises.

A good framework lets you work around things like QC which is after all an expensive defect tracker.

--------------------
You can buy my Art from: "Post Cards now available"
Ian Fraser Landscape Photography
World Wide Shipping.

http://mowogman.wordpress.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | >> (show all)



Extra information
0 registered and 76 anonymous users are browsing this forum.

Moderator:  IanFraser, Rajkumar_Rajangam, thorwath, TReddy, AJ, Tarun Lalwani, mwsrosso 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating: ****
Topic views: 36099

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5