Perf problem with SilkTest 2006 R2 datadriven .NET
I am experiencing performance trouble with a data driven testcase under SilkTest 2006 R2. My testcase has to run about 30 times, with one different dataset every time. The problem is that the execution of an iteration is about 10 seconds (or more) longer than that of the previous one. While I have noticed no particular performance problem with the first iterations, the execution gets quickly far too slow. Has anybody an idea of where that problem could come from?
Following are some details about my testing environment. Thanks very much in advance for any answer or hint.
- The data are stored in 1 Excel file containing 4 sheets with 1 column and up to 5 lines each. The testcase looks relatively simple to me.
- However, the application under test (AUT) is not so well designed and not really optimized. It runs with .NET Framework 2.0 with Infragistics and ActiveReports, and other proprietary components. It deals with an SQL Server database.
- SilkTest, its agent, the AUT, and SQL Server run within a VMWare virtual machine running Windows Server 2003 with 1 GB of RAM, on an Intel Xeon 3 GHz physical server with 4 GB of RAM (and the same OS).
Thank you again.
Re: Perf problem with SilkTest 2006 R2 datadriven .NET
One good think to do is ..
Instead of going to excel shet every time and getting the data store all the data into an array at once and use a loop and then run your testcase for different sets of data . I guess in this way we can improve the performance.
Correct me if I am wrong..
Re: Perf problem with SilkTest 2006 R2 datadriven
Thanks for the answer. But using my Excel sheets the way I explained seems to me the "natural" and "obvious" manner to take advantage of SilkTest data driven features.
Beside that, the additional times I have noticed don't seem to be linked to the moment when SilkTest gets a new dataset at the beginning of a new iteration. In fact, when the number of iterations increases, every action of this iteration gets slower: every click, every "key typing", etc. Moreover, the "partner.exe" process doesn't seem to take more RAM or CPU at any moment, which could have been the case if the problem came from the data access.
Indeed, I guess the main problem comes from the application. The RAM it takes grows quickly with the iterations, and after a while the SilkTest agent may have problems dealing with that (until about 140 MB of not-so-well-written .NET stuff ;-)). The SQL Server process becomes quite big too ('til 190 MB), possibly due to the application not being optimized.
As a result, I have done two things to diminish the problem:
1) reduce the number of actions in my testcase in order to avoid useless things that seem to end up overloading the system (e.g. close then immediately re-open a window);
2) close and re-open the application before my testcase executes, so that the system is not already too overloaded before execution.