Keeping Keyword-Driven Frameworks User-Friendly
being someone who tries to avoid work wherever possible, I spend a lot of time trawling through the forums on here, and for a long while I've been drawn to the idea of developing a keyword-driven framework. Initially I felt it was a little ambitious for me, but as I got to grips with concepts like error handling, test suites, intellisense (thanks to my old friend Coxy for that!), etc. I started to feel braver. However, within our company we have tests that go across a number of applications, and our core application has over 500 screens (and some screens have over 50 objects). The thought of maintaining the set up for all those applications and all those screens, on spreadsheets, just seems insane, and yet almost every paper and article on keyword-driven frameworks talks exclusively about using spreadsheets!
A lot of papers either hint at or state outright that the user is expected to know the names of the screen, object and action without any support from the system and so not surprisingly that is a criticism of these frameworks that you come across quite often. More sophisticated users of
spreadsheets talk about using named lists in the validation so that when a particular screen is selected, the objects dropdown is populated solely with objects from that screen. But then, what happens with the Actions list, since actions are defined by the object type (e.g. 'Click' for a button,
'GetValue', 'SetValue' and 'Validate' for an EditBox)? I guess with a bit of work it would still be possible to restrict the Actions dropdown list to only the relevant Actions for that object type, but it is starting to get a bit messy. And what happens when a developer decides to re-label an object, say from 'Customer 1 Forename' to 'Forenames: Customer 1'. Do you leave it as it was and hope that people can figure it out, or do you go through every spreadsheet making corrections where needed (we expect our system to eventually support thousands of scripts)?
My last point concerns functions (or 'tasks' as they are called in some papers). A common example of a function is 'Login' which takes 2 parameters: Username and Password. This is a simple function with fairly obvious inputs and would not take a lot of effort to remember. Now suppose you had 100 functions, and their inputs and the order of those inputs was less obvious? How do you support your users so that they know what goes where in the spreadsheet?
One of the reasons I drifted from Development into Testing is that I came to IT fairly late in life, and so I have always identified myself more with the users then the developers, and I worry that a lot of the discussion of keyword-driven frameworks focuses overly on the technical aspects of how to develop such a system, and not enough on how to develop a system that users will want to use and maybe even enjoy using.
After a lot of thought we decided to avoid using spreadsheets altogether and went with a VB.Net front-end with a SQLServer database for the back-end (it took 2 of us about two and a half months to build). If you have developed a keyword-driven framework using spreadsheets, I would like to hear how you have managed to keep the system maintainable and user-friendly. Have you come up with a quick way to collate all those spreadsheets into lists for when you build test suites from them? Will your system allow scripts to test that a value equals the current application date plus 1 month? If so, how have you kept the mechanism for this simple for users?
Re: Keeping Keyword-Driven Frameworks User-Friendly
We have been using spreadsheets in a keyword framework. In my experience I end up writing the work flows and test cases. The B.A.'s I work with read and review the test cases and results. They understand the spreadsheets but they don't write them.
We have some date specific test conditions. I don't think it is very clear to anyone but the B.A. who handles this area. It employs their jargon. 'Domain specific' I think is the term. This is how we have put the information into spreadsheets -
iUserStory, iDateTrack, iEffectiveDate, ...
'termination 28 day', -28 d, Default, ...
'termination < 28 day', < -28 d, Default, ...
'termination >28 day', -28 d, Default, ...
'termination >28 day', -28 d, Default, ...
'termination LTD >= 1 d', -1 d, Default, ...
'Waiting period < 6 mo.', < -6 m, Default, ...
'Waiting period 6 mo.', -6 m, Default, ...
'Waiting period Health < 42 day', < -42 d, Default, ...
'Waiting period Health 42 day', -42 d, Default, ...
The default above refers to the date the test is run. It could also be a specific date - 12/1/2010.