Some guidance on performance testing
Iíve been tasked at my new place with getting performance testing set up ******ly, so Iíve been doing a lot of research online, but off the back of this I have a few questions To provide a bit of background on what we do, Iíve listed a few points below:
ē Our website sits on a .Net framework, hooked up to SQL Server
ē At the moment we have a maximum of 150 people using our site at peak times, but this is expected to ramp up considerably over the next couple of months
1) Is a Test Strategy Document required for Performance Testing? If so, what sort of stuff goes in here? Would you put the test conditions in here detailing what will be tested, or is this in a separate test plan?
2) What are the key things that always need to be tested when thinking about performance testing? Are there certain tests that are always carried out?
3) How closely should we get involved with the infrastructure of the Performance Testing environment? Is it the performance testers responsibility to have an understanding to ensure that it closely mimicks production?
4) Within performance testing there sits Load Testing; Stress Testing etc. Which types of performance testing are carried out?
5) Weíre starting from scratch hereÖ.so if you were to advise on the end-to-end user journey in terms of what we need to do from the outset, to how we actually start running performance tests, based on your experience, what are the different steps to take?
Any help would be much appreciated
I'm not a load testing expert, but I've done quite a bit of load testing in the past. Strategy and stuff is all up to personal preference.. on your other questions..
Originally Posted by tilston1001
Just to input into your first question regarding whether a test strategy is required or not, your next four questions are actually part of your test strategy - these questions (and the answers to them) are what forms part of the basis of your strategy of what to test, who is responsible for what, and how you will be approaching the testing.
Test conditions would go into your test plan though, not your strategy. Once your strategy/approach is reviewed and approved you can complete the detail of how you will implement your strategy in your plan.
Neither your test strategy nor test plan need to be full-blown formal documents, produce what you need in a format that satisfies the needs of your stakeholders, and ensure that the strategy is run past the .Net and SQL developers for their feedback as well - it would be a good idea to get the team involved from the start, as they should be able to implement some of the traffic stats and confirm forecasts that you will be using as a baseline.
Thank you for the response David, much appreciated.
Originally Posted by dlai
And meridian_05, also! A lot of useful information to take from both of your comments
I've just been going through some of your old blog post and responses....really impressed, and I'll take some time out to go through these as I think there is a lot of stuff that will get me thinking a bit more. Out of interest, do you have a blog that you post to as I think this would prove really beneficial to users on here
Please find my answers inlined in red
1) Is a Test Strategy Document required for Performance Testing? If so, what sort of stuff goes in here? Would you put the test conditions in here detailing what will be tested, or is this in a separate test plan? All definitions, scenarios, metrics, etc. in fact anything apart from load test scripts and results are to be defined in the test plan
2) What are the key things that always need to be tested when thinking about performance testing? Are there certain tests that are always carried out? It depends on the testing target. Web application is tested one way, complex banking or telecom hardware/software in another way, broadband country-level routers - third way, there is no swiss-army-knife recipe
3) How closely should we get involved with the infrastructure of the Performance Testing environment? Is it the performance testers responsibility to have an understanding to ensure that it closely mimicks production? As close as possible. If there is no possibility to have production-size staging environment at least consider several tests with increasing load to ensure that resource consumption linearly increases. It should give you enough inputs to calculate what will be the maximum amount of load which production can carry out. Besides you'll be able to identify bottlenecks
4) Within performance testing there sits Load Testing; Stress Testing etc. Which types of performance testing are carried out? All of them. Load testing is only a lesser subset of Performance testing, it stands for putting application under anticipated load. Other testing types are also extremely important, see Wiki article on Performance Testing for list and definitions and Why ĎNormalí Load Testing Isnít Enough clause for explanations
5) Weíre starting from scratch hereÖ.so if you were to advise on the end-to-end user journey in terms of what we need to do from the outset, to how we actually start running performance tests, based on your experience, what are the different steps to take?Again it depends
thanks for your comments glinius. much appreciated
As you are new and you appear to have been asked to perform in this role then there are certain items your management owes to you
* Training in processes and methods (tool independent)
* A thorough review of your foundation skills (tool independent) and brush up where needed
* A Mentor for a period of time. This could be as shot as 4-6 weeks if you have exceedingly robust foundation skills or as long as several years with weak ones. If you are still in an internship after a year it is time to consider moving to a different field.
Tools account for somewhere between 5-15% of the full skill set you need to have on any given project. They are critical, essential skills, but so are the other ones, such as a command of application architecture, expertise in the area of requirements collection and management, systems analysis skills related to bottleneck identification (independent of tool), development skills in the language of your tool (minimally) and ideally in the language of your tool and the language of your application, monitoring and reporting, capacity planning, testing (as a process independent of tool), project management, perhaps even politics on managing bad news.
There is a series of podcasts available which may help which have been running for three years, PerfBytes. ( PerfBytes | Helping IT Professionals to Improve Performance Practices )
Your management owes you certain items related to professional development to ensure high value and shore up opportunities for success. If your management will not consider this opportunity to own the success of your efforts, then take this as a preview of how much responsibility management is willing to own for low value as you learn on the job. This is not a reflection on you and your eagerness, but instead is a direct reflection on your management and an oft walked path in our industry of late to simply throw a person and a tool (that the person does not know) at a QA effort and expecting value.