SPONSORS:






User Tag List

Results 1 to 9 of 9
  1. #1
    lev
    lev is offline
    Junior Member
    Join Date
    Oct 1999
    Location
    Redwood City,CA
    Posts
    10
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Retry mechanism for testcases.

    Hi guys,
    I need to automatically rerun entire testcase, when any inconclusive error like "Network is down" or something is trapped with Do...Except statement. Has anyone done this before?
    Thanks,
    Lev.

  2. #2
    Senior Member
    Join Date
    Feb 2000
    Posts
    1,497
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    Put the recognition and retry logic into the code that received the "inconclusive error" instead.

    This may require that it be inserted at a single place deep in the class structure, but you'll benefit from reduced long term maintenance.

  3. #3
    Senior Member
    Join Date
    Jul 1999
    Location
    Burlingame CA 94010
    Posts
    502
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    Hi,

    You can't actually do what Nancy is suggesting. When you call a testcase using the @operator (syntax is @(sTestcase) ()), any error which occurs in the testcase still gets reported, then control passes back to the function which called it.

    See the attached file for a function which will execute a testcase a specified number of times depending on the error. You can call it from Organizer and pass from 0-5 arguments.

    John

    ------------------
    John W Green
    jwgreen@automationexpertise.com http://www.automationexpertise.com

    [This message has been edited by John Green (edited 02-21-2000).]

  4. #4
    Senior Member
    Join Date
    Jul 1999
    Location
    Burlingame CA 94010
    Posts
    502
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    Here are the files.

    Check TestCaseExit to check for your specific error messages.

    runsafe.t


    runsafe.pln


    ------------------
    John W Green
    jwgreen@automationexpertise.com
    http://www.automationexpertise.com

  5. #5
    Junior Member
    Join Date
    Nov 1999
    Location
    Chicago, IL USA
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by lev:
    Hi guys,
    I need to automatically rerun entire testcase, when any inconclusive error like "Network is down" or something is trapped with Do...Except statement. Has anyone done this before?
    Thanks,
    Lev.
    <HR></BLOCKQUOTE>


    I haven't actually done this before, but I would think the best way to attack this would be with a main() that looked something like this:

    ...and then just have a list of string that lists your testcases and their parameters. Of course, this won't work if you use QA Organizer, but if your scripts used to be just a main() listing all your testcases in order of required execution, this would just be replaced by a loop and a list.


    Now I'm curious enough to go try this...



    [This message has been edited by Nancy Pejril (edited 02-21-2000).]

    [This message has been edited by Nancy Pejril (edited 02-21-2000).]

  6. #6
    Junior Member
    Join Date
    Nov 1999
    Location
    Chicago, IL USA
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by John Green:
    Hi,

    You can't actually do what Nancy is suggesting. When you call a testcase using the @operator (syntax is @(sTestcase) ()), any error which occurs in the testcase still gets reported, then control passes back to the function which called it.
    <HR></BLOCKQUOTE>

    What's wrong with reporting an error and returning back to the main to try the testcase again?

    I would think I'd WANT to know that a testcase failed once...especially if it does so consistently. This could easily be an indication of a race condition or some other seemingly random bug.

    Also, the syntax would not be @(sTestcase) () because then you would lose all the parameters you had in the list of string.

  7. #7
    Senior Member
    Join Date
    Jul 1999
    Location
    Burlingame CA 94010
    Posts
    502
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    Nancy,

    I should have been clearer. What SilkTest does when you call a testcase inside a do..except is report the error anyway. In other words, you can't trap the error.

    In your earlier example, there weren't any arguments. In the case of no arguments being passed, the syntax is @(sTestcase) (). You have to add the parens after the testcase call. With an argument the syntax would be @(sTestcase) (iArg)

    ------------------
    John W Green
    jwgreen@automationexpertise.com http://www.automationexpertise.com

    [This message has been edited by John Green (edited 02-22-2000).]

  8. #8
    Junior Member
    Join Date
    Nov 1999
    Location
    Chicago, IL USA
    Posts
    29
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    I guess I should have been clearer. I stated that "just have a list of string that lists your testcases and their parameters" but I didn't actually show a sample list.

    The requirement was a testcase retry. There was nothing in the requirement that said trap the error and don't return it.

    My personal goal in automation is to keep the script running as long as possible and only avoid logging errors that aren't really errors.

    An exception situation that occurs *sometimes* is most definitely an error I want to log, with full details as to where in the script this error occurred (which I get quite nicely from the ExceptLog() call in TestCaseExit.)

    Let's not forget that our goal is not only to write elegant code but to also provide detailed information about the application under test...and if that means logging a few errors to prove that the test environment is horribly unstable, then so be it. Log a few of these retry warnings along with the exceptions and you got some pretty good stats and detailed info to fight the battle with.

    In my last job I worked on an application that CAUSED network instability, and it was only through detailed research into the cause of these often unreproducible errant exceptions that we found and fixed the problem.

    As a developer, I could very easily have written some elegant code to trap the errors and avoid them, but then would I have been doing my job as a tester?

    Beware the double-edged sword of QA automation!

  9. #9
    Senior Member
    Join Date
    Jul 1999
    Location
    Burlingame CA 94010
    Posts
    502
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Retry mechanism for testcases.

    The problem is that you can't execute those strings using the reference operator.

    For example,
    Given: STRING s = "VerifyListBoxContents (lsExpectedList)"

    @(s) won't work.

    Given STRING s2 = 'MyTest ()'

    @(s2) also won't work.




    ------------------
    John W Green
    jwgreen@automationexpertise.com
    http://www.automationexpertise.com

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 05:28 AM.

Copyright BetaSoft Inc.