| || |
Load Test Environment
We're testing a Siebel app at the clients location,the issue is pre-prod(on which we'll do load test) is completely different from the production environment,hence could any of you please suggest the best possible extrapolation method, or any other techniques to suggest the projected results to the client.
Re: Load Test Environment
You have two methods available to you, modeling or handicapping. With modeling you take the results of your tests and use these as inputs to a modeling system, such as the offerings from Hyperformix. Such tools provide a mechanism for anylyzing the "What if's" associated with a platform shift from Platform A to Platform B, Architecture A to Architecture B. The caveat here is that any model involves some degree of assumption and even the tool manufacturers for modeling and simulation tools will tell you that at some point you need to actually test against your modeled projections to minimize the chance that an errant assumption results in a dramatic difference between the model and the actual performance.
With handicapping the method is one of capturing the number of differences between test and production, resulting in an objective measure of the number of differences and a risk score. You will want to measure the hardware and software differences between each platform member. The key here is to remember this is an objective measure of the differences. That is, anyone can follow the methodology you use to note the differences and wind up with precisely the same score. In a subjective measure of the differences you would need to argue the overall performance impact of each difference between production and test. Get three people in a room and you are likely to have three different answers on the impact of the difference. Nobody will argue that a documented difference in a setting, version, data level, etc... exists but they will argue until they are blue in the face the level of impact. I use a scale of 1-1000, with 1000 being perfect match to production in all areas. I shoot for tests in the 750 and above scale ideally and express my results as follows, "requirements A,B,C,D,... have <passed|failed> with a <score diveded by 100> degree confidence...." Low match between test and production typically results in a low score. This provides a bit of cover for the inevitable difference in observed performance that occurs and also provides some push against management to provide test environments which are a closer match to production if they want higher confidence results.
Replace ineffective offshore contracts, LoadRunnerByTheHour
. Starting @ $19.95/hr USD.
Put us to the test, skilled expertise is less expensive than you might imagine.
Twitter: @LoadRunnerBTH @PerfBytes
Re: Load Test Environment
Thank you for your reply.