SPONSORS:






User Tag List

Results 1 to 7 of 7
  1. #1
    Junior Member
    Join Date
    Mar 2003
    Location
    San Francisco, CA, USA
    Posts
    23
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    SilkTest Inconsistencies?

    I'm fairly new to silktest and I am experiencing some inconsistencies when running test cases. I have a Script file with ~50 test cases in it. When I run the test cases individually they work fine with no errors. However if I run the entire script file (F9), I get lots of random errors, and if I run the Script file(.t) from a .s file with a bunch of other script files I get even more random errors. Random errors include things like buttons not getting clicked, Cursor.GetType() returning wrong values, TypeKeys() not working etc...

    Running tests on IE 6.0 winXP SilkTest 6.0 sp1.

    Running tests in an application does not show nearly as many random errors!

    IE ~ 85% success rate
    App ~ 99% success rate

    If anyone has any suggestions on how get some more consistent results would be great!

    Thanks!


    ------------------

  2. #2
    Senior Member
    Join Date
    Aug 2001
    Location
    Manchester, NH
    Posts
    121
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    Random errors, or flukes, a pain in the side of automation. A lot of random errors really have a cause that is worth figuring out if you're looking for consistency. Spend some time investigating the random errors and you should be able to find the cause and work around it. Sometimes the window just needs more time to draw, or time for a button to enable, and sometimes unfortunately you'll find that a sleep will make a line more reliable and you'll simply have to put the sleep in and move onto other things. Other things you can do is check that the button was clicked and the window was closed or whatever, and if it wasn't, click the darn button again. I know Browser.LoadPage fails sometimes for us and we simply log a warning and repeat the call, and that usually works. There's a ton of things that could be wrong with any specific call. In the button click, the click could be missing because it's clicking on 0,0, or because the resolution is different, or because the button isn't enabled, or it could be a bug in the app, which does actually happen occasionally :-)

    One thing i've found helpful in debugging scripts you're not watching is having silk capture the screen when an error occurrs.

    Oh, I really don't see a reason for the errors to happen more/less depending on where you're running the testcase from, except that the longer the tests take the more likely something fluky will occur.

    good luck!

    ------------------
    Adam Kuczkowski
    akuczkowski@frametech.com

  3. #3
    Senior Member
    Join Date
    Nov 2001
    Posts
    375
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    Hi,

    If your tests are not organized properly - trust me its really not worth putting in any effort automating them. I used to face the same problems too. Your tests should take care that wahatever data your creating during the test should be deleted after the test is complete and the procede to the next test.

    Besides you can make sure that the objects / pages exist before you perform any action on them.

    Use as many print statements as possible! It sure helps you to debug!

    Best of Luck!
    Aatish

    ------------------

  4. #4
    Senior Member
    Join Date
    Jan 2001
    Posts
    750
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    akuczkowski: When you say capture the screen do you mean using CaptureBitmap() ?? or there is an option for doing this.

    People don't realize how important are the little things when everything is manageable in a small amount of time. Once you reach a limit where it takes almost a day(or more) to make a complete non stop automation run, then you realise how the little thing can help improve debugging(or code performance...).

    This is one of the best idea I have heard for a long time. You are my hero of the the month It's easy to think that you already know everything - especially with silk


    ------------------

  5. #5
    Senior Member
    Join Date
    Oct 2002
    Location
    Calgary, Alberta, Canada
    Posts
    101
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by GodKnows:
    akuczkowski: When you say capture the screen do you mean using CaptureBitmap() ?? or there is an option for doing this.

    <HR></BLOCKQUOTE>
    I get some inconsistencies when running as well which can be frustrating. Sometimes I tweak timing in the script to combat race conditions, and other times I just log a warning and continue on.

    I wrote an error handling function that logs a warning and captures a bitmap of the screen when there is some sort of inconsistency during execution. It helps debugging enormously. I also use typekeys("&lt;ctrl-a&gt;") and typekeys("&lt;ctrl-c&gt;") to grab page contents when there are 404s and 500s. I'd be curious if there is another way of getting this data as well.




    ------------------
    Beware of bugs in the above code; I have only proved it correct, not tried it."
    --Donald Knuth
    Beware of bugs in the above code; I have only proved it correct, not tried it.
    --Donald Knuth

  6. #6
    Senior Member
    Join Date
    Aug 2001
    Location
    Manchester, NH
    Posts
    121
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by GodKnows:
    akuczkowski: When you say capture the screen do you mean using CaptureBitmap() ?? or there is an option for doing this.

    <HR></BLOCKQUOTE>

    Yes, we just use a function that calls Desktop.CaptureBitmap, and raises an exception, so that a link to the screen shot will be in the results at the point the error occurred.

    [This message has been edited by akuczkowski (edited 03-27-2003).]

  7. #7
    Senior Member
    Join Date
    Jul 1999
    Location
    Bellingham, WA USA
    Posts
    1,323
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: SilkTest Inconsistencies?

    Mufasa,

    You said you're new to SilkTest, so first, welcome! Second, let not your heart be troubled. I understand the frustration of seemingly random errors. What you are seeing is the difference between "new" scripts that just work, and "mature" scripts that work and are robust!

    My recommendation is to give consideration to the things Adam Kuczkowski has said in the post above, and then also place as much as you can into shared functions. That way you can change your code in one place and that change will be effective for every test that calls the function.

    I'd recommend using Sleep statements as infrequently as possible. I'm more inclined to use an iWait and say while !xPage.Exists(1) && iWait &lt; 30 then sleep for 0.5 and increment iWait. That way once the page, or object you are looking for is there you can exit that loop. Otherwise, if you have a Sleep(30) you will sleep whether the object appears or not.

    Investigate each of your errors and surround the area in your code with all sorts of error checking. This is the difference between a "happy path" test where we code for how it SHOULD work, and an "off-normal" test where you ask yourself all the 'what if' questions as you code. You're seeing those "what if" situations.

    One last thing and I'll shut up here: Someone above mentioned use of Print statements for debugging. This can be tremendously helpful. We have a boolean Verbose= true/false where if true, you see all the print statements but if false, those are suppressed and you see only error messages and exceptions.

    I wish you the best with this. Don't be frustrated by this! You'll be fine!

    ------------------


    [This message has been edited by DJGray (edited 03-27-2003).]

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 02:34 AM.

Copyright BetaSoft Inc.