SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 5 of 5
  1. #1
    Member
    Join Date
    Jan 2001
    Posts
    33
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Automated web testing efforts?

    Hi everybody,

    I was wondering. How many of you working at online companies do automated web testing as a nightly effort? Do you come have testing results from a nightly test effort ready for analysis in the morning?

    The reason I ask this is that I am starting to develop automated web testing scripts, but don't really know where to start. So far, all the automation efforts here have been for specific areas of functionality to be tested before the release to the live web site.

    Are there automation efforts in place at your company that, for example, iteratively moniter the site for health on a 24 hours basis? Do you run automated tests to cover functionality of the site AFTER it's been pushed live?

    Thanks in advance for any comments!

    Andrew

    ------------------

  2. #2
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Automated web testing efforts?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by AndrewLance:
    I was wondering. How many of you working at online companies do automated web testing as a nightly effort? Do you come have testing results from a nightly test effort ready for analysis in the morning?

    The reason I ask this is that I am starting to develop automated web testing scripts, but don't really know where to start. So far, all the automation efforts here have been for specific areas of functionality to be tested before the release to the live web site.

    Are there automation efforts in place at your company that, for example, iteratively moniter the site for health on a 24 hours basis? Do you run automated tests to cover functionality of the site AFTER it's been pushed live?
    <HR></BLOCKQUOTE>

    I usually perform nightly regression runs that kick off after the daily code push.
    Each morning I review the results, investigate any anomalies further, and write bug reports as appropriate.

    As functionality is added to the system and becomes "testable" I can add to the regression suite.

    Separately, we also monitor the live site's health with scripts that check the major pieces of functionality. We look for major breakages or slowdowns. With the appropriate services, you can do this from many geographic areas at once.

    My suggestion would be to first build a framework for running your nightly regressions that allows simple insertion of new test cases.

    Work on making it very easy to analyze the results.

    Concentrate on the most important areas of functionality - particularly those that are not undergoing major changes in the near term.

    Run it every night.

    With some work, this can save a lot of testing time - time which can be used to further enhance your regressions.

    You'll also be able to catch bugs as they are introduced into the system - rather than much further down the road.

    Best of luck.

    ------------------
    - Joe (strazzerj@aol.com)
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  3. #3
    Senior Member
    Join Date
    Dec 2000
    Location
    Portland, OR
    Posts
    403
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Automated web testing efforts?

    We have various partners that leverage our technology on their sites, so while it appears as if you are on yahoo, or netscape, or whatever, its us underneath (the old powered by logo can be found on some of the results pages).

    Because production metrics are a part of our business, we do not run nightly regression on the live site., but we do have an automated suite that we run on any new cobranded versions of our site, that we run on the same software in our QA lab.

    The regression suite was born out of a lot of manual tests. I developed a framework that was easy to insert new test cases into (in fact, I have a test case template that people use to add new test cases to the regression suite), and a whole slew of customized library functions that handle the types of verifications that we would normally handle manually.

    A lot of work, and quite a bit of pencil and paper design, but well worth it. With the framework, we are able to do regression, HTML verification/validation, reliability tests rather eaily. The suite also saves the pageviews to disk to help facilitate browser testing. Now we can reference the disk based HTML pages rather than jump thru hoops to find a particular page view.

    But start small with a well defined idea. From there, only good things can happen and new expansions can occur that will make your life much easier in the long run. But we aware that this is an investment. Time, money will be used. But unlike a 401K, you just can't let it sit. Its more like a brokerage account/taxable account. Constantly tweaking to make it better.

    Good luck!




    ------------------
    It Depends.

  4. #4
    Senior Member
    Join Date
    Jan 2001
    Location
    Toronto, Ontario, Canada
    Posts
    1,621
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Automated web testing efforts?

    If you have automated tests, utilize them! Run them at night, monitor the sever responses, use any information you can!

    The problem with web is the short development cycles, so if you can squish in some testing while you are sleeping, all the better. Trust me, more than enough information on your product is MUCH better than not enough

    ------------------

  5. #5
    Senior Member
    Join Date
    Dec 1999
    Location
    Chicago,Illinois,USA
    Posts
    2,537
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Automated web testing efforts?

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by AndrewLance:
    How many of you working at online companies do automated web testing as a nightly effort?<HR></BLOCKQUOTE>

    I do this only if the automated tests are part of an automation framework and are centered on regression testing or combination/permutation testing. To a lesser extent, I do UI-variance testing with automated tools - again, assuming the automation framework is up to the task.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Do you come have testing results from a nightly test effort ready for analysis in the morning?<HR></BLOCKQUOTE>

    Generally, yes. That is mainly the whole point for running them at night. The key here, of course, is building up some good logic to parse out certain types of errors. For example, one script I did at a company was because we routinely had the servers going down for different reasons. Sometimes it was a 500 internal server, sometimes a general browser error (could not find the server), and sometimes a 404 error. So I wrote a SilkTest script that would gather the information from the error page and then write that into a specific format for easier analysis in the morning, complete with timestamps.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>The reason I ask this is that I am starting to develop automated web testing scripts, but don't really know where to start.<HR></BLOCKQUOTE>

    Automation should not even be considered until you have a testing metholdogy in place. Your automation will be an extension of that. Having said that, for me the next step is to build an automation architecture or framework of sorts that minimizes the need to drastically alter the automated test scripts. Rather, I rely on a test engine approach so that all logic goes into the engine - not the test scripts themselves.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Are there automation efforts in place at your company that, for example, iteratively moniter the site for health on a 24 hours basis?<HR></BLOCKQUOTE>

    Indeed. This is very critical. These are generally not behavioral/functional testing tools that do this, however, but tools that are geared to monitoring. Empirix, for example, has good tools for this but there are also a lot of freeware solutions you can get, particularly with Unix or Linux.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Do you run automated tests to cover functionality of the site AFTER it's been pushed live?<HR></BLOCKQUOTE>

    Sometimes. But that is generally a rarity, at least for pure fucntionality, and I will not go into all the exception cases here since that might be more than what you are asking for. However, this does sometimes have to happen if you push the site live to an environment that is different in any significant way from your test and/or staging environment. One example might be Cybercash. Some people do not have their environments set up where Cybercash recognizes their staging environment. So you might have to do this on a production box.

    Or consider going with some application service providers like NexPrise. For them, they will generally only point at one box on your setup and they require a lot of reconfiguration to point to different boxes. So, sometimes, it is easier to test this on a production box. However, that should be done before you are actually live. Generally you can take a production box out of production and make it your temporary staging server. If this is not feasible, then you have to work out how you can reliably test the functionality internally.

    However, all of that does not apply to the basic functionality of a given site. For example, you should be able to test any searching abilities, database lookups/retrievals, basic click-paths, etc. If you cannot, you more have a problem with how your environments are configured than anything else.



    ------------------

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 01:38 AM.

Copyright BetaSoft Inc.