how to perform functional testing while performance testing is been executed to be clear, when a reliability testing is performed. If the client do want to know the functional behaviour of others testcases apart form the business test cases as performed in performance testing, please advise is it a right approach and how to meet the clients requirement, i find the client requirement as valid because the intention is to find out how other functionality scenarios behave when X load in on the system
please do provide some valid quantitaive solutions for teh same.
1. what i feel is the funtional testing thats done during the reliability testing would greatly impact the cpu and memory of the servers that are been tested.
2. The operational model of functinality testing is on presentation layer as compared to performance testing where it acts on application layer of the OSI model.
3. functionality tetsing carried on the same environment as that of performance testing has a great impact.
Please provide your suggestions/thoughts/informations/statistical and dynamic solutions.
The most efficient approach would be to integrate both these tasks. The scenarios that you want to be tested during the performance test can actually be made part of your performance test scenario.
so, in parallel to getting performance benchmarks you can also verify the reliability of the service across scaled loads.
Basically you will be scaling the load and doing a stress test on the system. The performance numbers that you collect along the way can itself suffice for your report on "continuity of service from the system".
Ramesh, I'm not quite sure what do you mean, so let me try first to tell how I understood you:
You have two sets of user scenarios: 1) those scripted for performance/reliability and 2) Those used for functional testing.
Now client request you doing following: launch performance tests scripts and execute functional (manual?) tests simultaneously. This is more than a reasonable request. I wonder why donít plan something like this yourself?
I assume you worry about (maybe a few of) the following issues:
- The load on the system will be greater than expected by performance tests, as the functional tests will create additional load.
- There will be technical issues to perform functional test because performance tests data may impact test results. For example manual test may open a business entity created by performance tests that is not filled in correctly (as expected by functional test)
- Anything else? For example there are a lot of fake (generated) data created for load tests, load tests include system overload, response times will be reasonably small, but will slow down functional testing, etc.
So I suggest make sure either that your load tests create correct data or your functional tests never interact with that data. You should analyze how big the additional load on the system will be created and probably reduce the performance load by that value.