SPONSORS:






User Tag List

Results 1 to 7 of 7
  1. #1
    Junior Member
    Join Date
    Sep 2001
    Location
    columbia, mo, usa
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Is it worth the frustration?

    Our Company recently purchased Segue's silktest and silkperformer. The performer is great, but the test has basically been nothing but a pain. We have 2 main websites, each with 6 different home pages and not to mention our partners. We have had a consultant onsite for 2 weeks and he has even said it would cost more to keep up then to manually test. We have 2 installs a day, anywhere from 1 to 100 changes in those installs, is it worth all the scripting for testcases? We can only use DOM and that means we still have to manually test anyway for the different browsers. I went to the training for silktest, the beginners and the 4test language,we are so much more complex that the consultant is even frustrated with this product. Any feedback, of course there is more stipulations than we learned in class, and other look and feel applications is it worth all this time? ANYONE???

  2. #2
    Senior Member
    Join Date
    Jun 1999
    Location
    Fremont, CA, USA
    Posts
    355
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    As always, "It Depends..."

    If what you want to automate is only regression testcases (recommended), then it might be worth the time, if the pages themself that are being tested are not changing. If the pages that you want to automate are changing with each build, then I would say it isn't worth is.

    You want to be able to have a new build and run the automation. Not have a nwe build, fix the automation (by manually going through the application to see what changed) and then running it.

    Also, 2 weeks isn't much time to have someone come in to get a company up and running on a good automation framework (unless it is a very simple web site).

    It takes time to get automation working and stable. Sometimes, the cost of getting automation isn't worth the time.

    Just my 2-cents.

    ------------------
    David Genrich
    david@e2Open.com
    david@linuxpuppy.net

  3. #3
    Senior Member
    Join Date
    Jun 2000
    Location
    Chicago, IL USA
    Posts
    383
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    You should do a cost/benefit analysis to establish an appropriate balance between manual and automated testing.

    Some questions to ask:

    How much do bugs actually cost your company now ?
    How much would future minor/major/performance problems cost ?
    How much does it cost to maintain your automated testware ? manual testware ?
    How effective is your testware ?
    How many bugs go into production undetected ?

    Many companies go into automated testing with the idea thay they will "automate everything". This is rarely practical. It is not good testing either.

    Good manual testers will catch many problems that could never be anticipated with automated tests. Automated tests are really good at flagging regressions that humans have a tendency to overlook. Automated tests are also essential for testing load issues.

    ------------------
    Rob Kapteyn
    Rob Kapteyn, CSTE
    RobKapteyn@gmail.com

  4. #4
    Senior Member
    Join Date
    Jun 2000
    Location
    Chicago, IL USA
    Posts
    383
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    Looks like Segue's snake oil salesmen sold you their "magic", "no experience necessary" SilkTest tool.
    Sorry -- it doesn't work like that.

    SilkTest is notoriously finicky, buggy, hard to install, and difficult to setup.
    Even so -- it is the best tool on the market for many automated testing tasks.

    But what did you expect ?
    SilkTest is a very complicated program that ties into Microsoft's buggy and unstable GUI API in a way that no other program does.
    It then it tries to manipulate your application's admittedly buggy and unstable GUI.

    SilkPerformer does not have these stability problems because it does not deal with the complexity of a GUI interface.

    The reality is:
    (1) you must start with a fast computer and a stable OS (newly installed is best).
    (2) you must have an experienced SilkTest person (or a GUI developer) to get started with the tool.
    (3) Avoid all of the automatic "wizards" and "organizers" -- they are just marketing gimmicks.
    (4) Beware of Segue's expensive "training" -- most people are not satisfied.


    ------------------
    Rob Kapteyn

    [This message has been edited by RobKapteyn (edited 09-07-2001).]
    Rob Kapteyn, CSTE
    RobKapteyn@gmail.com

  5. #5
    Junior Member
    Join Date
    Sep 2001
    Location
    columbia, mo, usa
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    Did anyone else have as many problems when they first installed this program? I keep crashing, my agent doesnt connect, my errors are not consistant, when capturing properties on a page my Control/alt jams, takes FOREVER, or does not capture everything??? I keep getting those //WARNINGS in green that were never covered in class and sometimes my testcase passes when it should have failed???

  6. #6
    Senior Member
    Join Date
    Jun 2000
    Location
    Sunyyvale CA USA
    Posts
    422
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    I agree with the postings so far.

    Again, do not try to do everything with recording.

    Install on better system with stable OS. Do not record too much in a single instance.

    Do not move yr mouse too fast and out side AUT while recording.

    And the same while playing back. Do not touch yr mouse whie playing back.

    ------------------
    Sree
    Sree

  7. #7
    Senior Member
    Join Date
    Nov 2000
    Location
    Bloomington, IL
    Posts
    142
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Is it worth the frustration?

    I agree with all of the above. Almost 2 years ago we paid a SilkTest trainer to come and train the 15 of us that were going to be using SilkTest. All he was able to do was to give us the basics. We have found that using the product every day and sharing the sucesses and failures with the product has done more to move us to a more advanced use of SilkTest.

    After working with the tool for 1 year, I created a basic and advanced SilkTest class. The basics class spends some time (1/2 day) on the built in recorder and built in recovery parts of SilkTest but then spends the rest of the time talking about coding functions, and error handling, just like any other language. I do not tell the students not to use the recorer, but have them use the tool in a controlled environment and let them see the errors the recorder can create, and the amount of time you might spend fixing it. We then compare that time to the speed of coding the same thing by hand. 90% of the time coding by hand is faster.

    For the error handling, SilkTest's built in recovery system is great, if you always want to go back to the beginning of an application. Instead we have been building "happy path" functions for an application. There are ane "happy path" function per page of the application. The function contains all of the data entry needed to move thru the page successfully. We can then create other funtions to validate displayed information, create known errors, ect.. In this way we control the path thru the application, if we encounter an error, we log the error and go one. Most of the time we also create a function to return the screen to what is needed to continue thru the "happy path" and to the next screen.

    I have been using SilkTest for almost two years and the have been thru all of the classes that Segue has to offer. I have found that the information is good, but you have to ask lots of questions in order to really get all of the information you need in order to use the tool. Experience (and this forum) are a great source of information about SilkTest. You should use both.

    MikeF

    ------------------

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

vBulletin Optimisation provided by vB Optimise v2.6.0 Beta 4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.0.9 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Questions / Answers Form provided by vBAnswers (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominatevBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 03:54 PM.

Copyright BetaSoft Inc.