Testing performance on platform different from the production platform.
Re: Testing performance on platform different from the production platform.
I have recently begun working on my master thesis project which has to do with performance testing (however you choose to define it).
In short, the thesis is about coming up with some kind of strategy for implementing component-level performance testing of a rather large-scale information system. This system is developed by a multinational company that is represented on several sites around the world.
Now, the thing is: On the site where I work, we don't have access to the platform on which the system is going to be run by the customers. We currently do functional testing on our test platform, but now we want to start testing performance too.
The machines we use for testing are Intel-based workstations in a multi-tier environment (clients, database server, web server, etc running on different machines) running different flavours of MS Windows 2000 (Pro edition, advanced server, etc). The customers are most likely going to use quite high-end servers (same OS of course) supporting a large number of users, so the performance of the computer system itself, not taking the application under development into account, we can expect is very much worse than the one the customers can expect from their machines.
According to what I have read (and according to common sense also), it is rather essential to do performance tests on the platform that the system is going to be ultimately run. At least for system-level tests.
Now, my question is: Is there any way of getting any kind of relevant insight regarding the performance of different _system_components_, when we are doing testing on machines that cannot be compared to the ones that the customers are going to use?
Different approaches to get such insights (it doesn't have to involve testing) could be performance modeling, simulation or anything of that kind. For example, perhaps one could model the customer platform and do performance analysis on our test platform by simulating the customer platform.
The purpose of these component-level performance tests is to get indications, early in the development/design process, on the performance of different components, finding potential performance/scalability problems at an early stage in the development cycle. Thus, the results from the tests we have in mind is by no means final, but rather early indicators.
It is a rather central issue in my thesis project that I find a way of drawing reliable conclusions regarding the performance of individual components by using our test platform.
Has anyone heard of this problem? Any suggestions, comments, etc? If anybody know of any litterature covering this kind of problems, I'd be glad to hear of it [img]images/icons/smile.gif[/img] .