SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 6 of 6
  1. #1
    Member
    Join Date
    May 2009
    Location
    Canada
    Posts
    85
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Layers separation VS testing UI functionalities

    Hi all,

    So I'm still struggling at creating my first automation framework using WebDriver.

    Today I was refactoring my page objects so that they encapsulate any reference to the UI, and keep them private.

    My page objects won't let you "click a button", but they will let you "Navigate to a page", or "add a comment" etc. "Macro actions".

    But somtehing struck me: if you need to test some UI functionality, your framework is useless then. Yes, you spent all that time making sure that the UI was in its own separate layer, and now you can't directly access it from your test layer anymore.
    Isn't that a paradox? we have those UI automation tools, and we still use them only to validate business scenarios that could be validated without the UI (if an API is available of course).
    But a failure on a particular UI element can lead to block a business scenario, and if you don't cover this particular business scenario, you're screwed. However, you would have found the bug if you had some automation testing all the states of this particular UI element.

    Or maybe you just want to automate testing on a particular UI element that is known to be fragile.

    Now even if I stick to the layer separation approach, in my application I can come across complicated screens with forms where there are many possible combinations possible before submitting the form.
    It seems like a pain to make sure that my Page Object doesn't expose the UI, because it would mean I need a lot of different functions for my different combinations.
    Sometimes it seems it would even be preferable to have one function per UI element, and let the test layer set everything, rather than having bigger UI abstracted functions from my page object for each combination.

    It kind of kill the purpose of layers separation, doesn't it?

  2. #2
    SQA Knight
    Join Date
    May 2006
    Location
    Playa Del Rey, California, United States
    Posts
    2,594
    Post Thanks / Like
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Layers separation VS testing UI functionalities

    In automation, we like to avoid pure GUI tests, and try to form things closer to the requirements.

    For example, there might be a requirement that says, "User cannot create an account with a password less than 10 characters.". To test this, you can create a pure GUI test that tests if the submit button is enabled or not after typing in 10 characters. Or, on a higher level you try to submit a password less than 10 characters and assert that you have been thrown an error. The former requires more maintainence and probably will break with any UI tweaks. While the latter may not check if a pretty error message or button is disabled, but still verifies the requirement the user cannot create a new account with less than 10 characters for password.

    In the automated testing world, we want to keep the top layer test itself to be as static as possible. Because this represents the definition of the requirements as interpreted by the test engineer. While the actual UI steps, you want to encapsulate and hide them in the Page object, which encapsulates the implementation.
    David Lai
    SDET / Consultant
    LinkedIn profile

  3. #3
    Member
    Join Date
    May 2009
    Location
    Canada
    Posts
    85
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Layers separation VS testing UI functionalities

    After a lot of thinking on that I came to the conclusion that there are situations where automating UI scenarios is desirable (rather than automating business scenarios).

    I'm not saying that all your tests should directly refer to the UI, but you could have two sets of tests: those UI agnostic, and those testing the UI itself.

    It really depends on the context. I know that our application's UI changes rarely, and that our business people want to make sure that some UI parts is still working completely build after build.

  4. #4
    SQA Knight
    Join Date
    May 2006
    Location
    Playa Del Rey, California, United States
    Posts
    2,594
    Post Thanks / Like
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Layers separation VS testing UI functionalities

    Actually the opposite is true. Because the GUI changes so often, you don't want to rewrite you tests. Say if a password field is replaced with a custom password field. Do you really want to recode that portion of the test because of such a small trivial change? So anything done in the UI layer should be abstracted outside of the test definition into a different layer. So typically a good framework will have a Object recognition layer, and a implementation layer that handles the detailed UI work. At the very top level, we do not want to do any direct asserts on UI state. We want to verify whether or not we can perform the desired action.

    The reason why testing the behavior or UI components is not desired is because these are usually just off the shelf components. The developer rarely writes code defining the behavior of UI control itself, just the application state and data that the UI component reflects.
    David Lai
    SDET / Consultant
    LinkedIn profile

  5. #5
    Member
    Join Date
    May 2009
    Location
    Canada
    Posts
    85
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Layers separation VS testing UI functionalities

    [ QUOTE ]
    Actually the opposite is true. Because the GUI changes so often, you don't want to rewrite you tests. Say if a password field is replaced with a custom password field. Do you really want to recode that portion of the test because of such a small trivial change?

    [/ QUOTE ]

    No you don't want to. This is why your Business tests should not refer directly to the UI. I think we agree on this.
    However in a context when your UI is unlikely to change, UI tests less likely to break.
    I would of course always keep any reference to the UI (i.e.: element locators) unique.

    [ QUOTE ]
    So anything done in the UI layer should be abstracted outside of the test definition into a different layer. So typically a good framework will have a Object recognition layer, and a implementation layer that handles the detailed UI work. At the very top level, we do not want to do any direct asserts on UI state. We want to verify whether or not we can perform the desired action.

    [/ QUOTE ]

    In theory yes.
    Now let's say your application creates records (of any kind).
    On the screen used to create records, you have a lot of different controls.
    You will not have one business scenario for each combination of those controls. You may actually have some controls that your business scenarios never hit because you decided to spend your resource on hiting important business scenarios.
    Creating a test for one business scenario requires a bigger effort than creating a test for a handful of UI controls.
    So you might want to add UI controls tests as a complement to your existing business scenarios tests.
    Especially if you know that this screen is fragile.



    [ QUOTE ]
    The reason why testing the behavior or UI components is not desired is because these are usually just off the shelf components. The developer rarely writes code defining the behavior of UI control itself, just the application state and data that the UI component reflects.

    [/ QUOTE ]

    I have two points here:
    -off the shelf components can break, as can custom controls, especially in the context of the web with all the compatibility issues. I'm not saying you need to hit each button either. Depends on how complex/fragile your controls are.

    -Testing the UI doesn't only mean testing the components individually. You can test their interactions.

    Imagine you're testing Hotmail. You want to verify that when you click on the "ALL" checkbox, it does select all your e-mails on the page.
    This test can be easily automated, and is completely coupled to the UI. What would you have an abstraction layer for this test?

  6. #6
    SQA Knight
    Join Date
    May 2006
    Location
    Playa Del Rey, California, United States
    Posts
    2,594
    Post Thanks / Like
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Layers separation VS testing UI functionalities

    If you think about what a system level automated test gets you that a unit test or an integration test of 2 interacting components doesn't get you.. it's the acceptance of an end to end flow. That the components are hooked up right, the backend systems are hooked up right, the communications between the different dependent services are working, the big picture items.

    A simple test of checkbox for select all for instance does not affect anything down stream. So I can argue that it's less effort and less maintainence to test that using a unit test. This is the same for most test that testing GUI behavior. (I agree there might be some situations that might force you to have to, like a bad org structure where devs arn't doing the correct unit tests and QA is not allowed to modify the code to add unit tests the devs are neglecting.)

    Where end to end tests gives the most value is data flow through multiple pages and multiple components. You want to verify that the components and their back end systems are hooked up correctly. This is something that unit tests don't buy you. This is what I think system level testing should focus on.
    David Lai
    SDET / Consultant
    LinkedIn profile

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 11.54%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 06:07 AM.

Copyright BetaSoft Inc.