SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 6 of 6
  1. #1
    Senior Member
    Join Date
    Jul 2010
    Location
    Colombo , Srilanka
    Posts
    115
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    False Failures in Automation

    Hi ,

    I have been using selenium tool for a while now , normally i run 600 to 700 test cases at a time. After finishing my automated test cases there are errors , but those are not actual system errors. these errors are normally due to performance issue . if we re-run these failed test cases this will execute perfectly with zero errror , so i want to know , in my experience i didnt identified actual system errors , this feels me odd. , most of the errors are occurring due to performance issue either application server or db server,

    Is there a way or method to make my server performance higher level before my test automation suite run. or is there any pre_plan our system to be suit according to my automation script. to make my test scripts pass at first run.

    Also , as i mention there are less amount of errors are identifying through automation (selenium). what would be the best approach to automate , is it better to automate according to test scenarios or bases on available test cases. because i have never experienced by locating actual errors. unless development changes or modification.

    Still our manual testers find more bugs than my automated scripts.I was hoping to capture maximum number of errors through automation but this didnt worked out for me well.

    please help me to tune up my scripts.
    Muaaz ,

    Testing is my Geane --

  2. #2
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: False Failures in Automation

    [ QUOTE ]
    Is there a way or method to make my server performance higher level before my test automation suite run.

    [/ QUOTE ]
    Your automated scripts really should never have failures simply due to a slow server. If this happens, it's an indication that your scripts aren't designed and written well.

    That said, if they consistently do what you want them to do the second time you run them, the quick-and-dirty solution seems obvious - run them one time to prime your server and ignore the results, then immediately run them a second time to actually test your functionality.

    [ QUOTE ]
    Still our manual testers find more bugs than my automated scripts.

    [/ QUOTE ]
    That's typical. I always find more bugs when I'm creating my automated scripts than when I execute them.

    What you need to decide is if it is still worth the effort to maintain those scripts, or if the bugs they find could more effectively be found manually.
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  3. #3
    Moderator tristaanogre's Avatar
    Join Date
    Mar 2006
    Location
    Pennsylvania, USA
    Posts
    1,792
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: False Failures in Automation

    [ QUOTE ]
    [ QUOTE ]

    Still our manual testers find more bugs than my automated scripts.

    [/ QUOTE ]


    That's typical. I always find more bugs when I'm creating my automated scripts than when I execute them.

    [/ QUOTE ]

    I'm of the philosophy that automation has the best value in trapping regression bugs than it is in finding new bugs. Something worked last version and we wrote the automation to generally check for that feature. When code is changed, the automation should ensure that what used to work still works.

    In the shop that I worked at that had heavy automation use, our regression automation trapped a LOT of bugs in "old" features which freed up the testers to be able to spend more time in manual testing of the new features and code.
    -Robert Martin
    Automated Testing Evangelists
    TestComplete "expert"
    Definition expert - noun - Unknown drip under pressure

  4. #4
    Moderator
    Join Date
    Mar 2002
    Location
    Tampa Bay
    Posts
    7,673
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: False Failures in Automation

    [ QUOTE ]
    I'm of the philosophy that automation has the best value in trapping regression bugs than it is in finding new bugs.

    [/ QUOTE ]
    You could almost say that is "by definition." An automation script is designed to pass when the application works and fail when it doesn't (per the behavior at the time of script creation.)

    So its main strength is to find things that USED to work when the script was created, and DON'T any more.

  5. #5
    Member
    Join Date
    Dec 2006
    Location
    Seattle, Washington
    Posts
    230
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: False Failures in Automation

    To answer the question about your tests throwing a false positive, I agree with Joe this is likely a test design problem. One potential work around is to use a polling loop to give the server time to respond to the request and synchronize your test with the system.

    Most scripted automated tests can be effective in exposing regressions in the system (hence regression testing)

    But, well designed automated tests can also include variability more efficiently than manual testing and can also expose new bugs.
    - Bj -
    <font color="red">I.M. Testy blog</font>
    <font color="red">Testing Mentor</font>

  6. #6
    Senior Member
    Join Date
    Jul 2010
    Location
    Colombo , Srilanka
    Posts
    115
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: False Failures in Automation

    Thank you JOE , i have to re-run twice to make my scripts successful , as per peter's saying
    "
    An automation script is designed to pass when the application works and fail when it doesn't (per the behavior at the time of script creation.) "

    I agree with it ,

    I need to create the scripts which can fit most of the situation , Please share the good script designing concepts to overcome this kind of problem.

    I appreciate for your concerns ,

    Thumbs Up !
    Muaaz ,

    Testing is my Geane --

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 04:07 AM.

Copyright BetaSoft Inc.