| || |
In terms of Performance Engineering, how do you describe each of these terms
I take a stab it it.
Baseline: I use the term baseline to refer to the results from a special set of tests. These test are run once we have achieved a certain configuration of hardware, middleware, and application. For example, with our current hardware, version 0.9 of the application, and our middleware tuned with settings current to October 18th, we can expect that with a 300 concurrent user test we get 10 pages/second and an average response time of .75 seconds per page. A month later, after a half dozen changes to the application and no changes to the hardware or configuration, I can run another 300 user baseline and compare it to my earlier baseline to see if I have impacted performance.
Benchmark: to me, a benchmark is the result from running a benchmark tool designed to test hardware performance. You run the same software during each test, but you vary the hardware configuration. The goal of running a benchmark is to find the differences between hardware configurations.
SLA: to me, SLA means service level agreement. From a perftest point of view, the SLA represents the targets you want the applicaiton to achieve. For example, the marketing team may determine that the application must meet an SLA of less than 5 seconds for all write operations and less than 2 seconds for all read operations.
-I use the Term Baseline to describe the results of a "good", usually long, load run. This run will usually take place during the last cycles of the current product version, and will be used as reference during the load cycles of the next version of the product.
-Benchmark: a performance test consisting of 15% of expected load. I run these in order to see wether the application, or the system, can handle relatively low amount of load, and it it's ready for some more rigorous testing
-SLA's are defined by R&D/Marketing. My job is to tell wether the product is within SLA or not (and why).