I have been doing a web application automation. Has a set of 50 test cases now.
If I give them for run, they fail 50% of the times for one or the other reason. After that I I give again for the run without any modifications, they PASS !
Is the speed of the network could be an issue ?
Can any one point me to strategy for automating test cases without frustartion and get good results.
Whats the good practice to be followed for automating test cases? I have read many papers for Test Automation Strategy / Automation Planning, but they do not explain how to go about coding in test automation.
If network speed is a concern, then I would definitely say this could have an impact on what you're doing. There are a couple of ways to go about what you're doing.
Divide and Conquer:
You can drop log.message statements throughout your code and hope that there is a specific area in the code where it is continually failing. If this is the case then it could be traced back to a slow loading page or possibly a code glitch. If it seems to happen in random areas then I would start looking at the possibility of connectivity issues.
Slow Things Down:
This is a very simple method, but doesn't give you a whole lot of debugging information. However, it's a good follow-up, if the above turns out to be intermittent.
All you want to do here is slow down your execution of the tests. You can throw in delays or extended WaitForObjects. What you want to aim for here is simply to create the most stable environment possible. This will, however also lead to slower test cycles.
Whichever way you go, if you suspect network connectivity issues then you can also check to make sure that the page is not producing a 404 error. You can actually make use of the XMLHTTP object to get the contents of a remote page. If it isn't there or if there is a network problem then you will be able to immediately see that. For this you can write something like:
url = "http://yourdomain/" & thisPage
set xmlhttp = CreateObject("MSXML2.ServerXMLHTTP")
xmlhttp.open "GET", url, false
getContents = xmlhttp.responseText
set xmlhttp = nothing
In the above, thisPage comes in from another sub routine where you are defining the page that should be there.
writeLog points to a sub routine that will output the entire contents of the remote page (you can create and output an HTML page with the contents of the remote page so you can preview it for debugging)
This solution might take a little longer to perfect, but it will probably be the most effective.
Is there any possibility that you have overlooked a data set up condition? I know that every now and then I run into an automatin set that relies on data that is already in a database. I ask this because one thing that can happen is that your first run is failing BUT placing data in the DB and then second attempt is finding this data and using it.
It is a vague vague idea but I have seen it happen and whe you have a problem like this - well, anything is possible?
oooooo, good point Martin. Those suck the big one. Stepping through can help with this too. If it's at a code level, for sure. Also, you will be able to see if it's a particular set of data that's killing you. Maybe commas or apostrophes. If you are working with flat file data and using separators that are common in language, then this is a big pain.