RandyHayes
Member
Reged: 04/17/09
Posts: 31
Loc: Dallas, TX
|
|
Hey All,
If you are doing the Agile thing you know that unit testing and hopefully functional testing should be part of the nightly build cycle. But performance testing usually waits until the end, when it's too late to fix any issues that arise.
Instead, performance tests should begin as soon as the manual tests do, and they should be run nightly along with the build with email notification of the results so performance problems can be fixed as soon as they rear their ugly heads.
Is anybody out there doing this? If so, I would love to hear about it. If not, what keeps you from doing it?
Cheers
Randy
--------------------
"All computers wait at the same speed"
http://capcalblog.blogspot.com
|
richard.hill84
Member
Reged: 08/18/03
Posts: 34
|
|
Very good question. Our company has started to go down the agile route and i'm in the same boat as you.
I'm not sure how performance testing will fit in and how it will be managed.
Anybody doing performance testing in the agile world?
|
A$H
Active Member
Reged: 10/04/04
Posts: 1068
|
|
We do single user performance run daily like functional smoke test to identify obvious performance issue early / during dev. cycle. These need systematic approach to remove outliers and unnecessary noise.
We also setup such single user WS test for SOA architecture.
|
JakeBrake
Moderator
Reged: 12/19/00
Posts: 15264
Loc: St. Louis - Year 2025
|
|
We care not what the development methodology might be. We require performance testing early and often. Whether that actually occurs is another story.
|
Brad Johnson
Special Member
Reged: 07/10/06
Posts: 130
Loc: San Jose
|
|
I see key barriers to Agile Performance testing, or even, Iterative Performance Regression testing as:
1. A dev plan that includes performance requirements/expectations at a meaningful granularity
2. Performance engineering personnel (or performance-minded engineers) available for the entire project cycle vs shared across a dev org
3. Availability of test and staging environments capable of proper iterative testing
4. Tools tailored for rapid setup, simple test creation and frequent use (from a cost and/or a technical standpoint)
--------------------
** I work for SOASTA, the CloudTest Company **
|
SteveO44
Active Member
Reged: 07/08/04
Posts: 998
Loc: St. Louis, MO, USA
|
|
Our issue is that the environments used for unit and functional testing are very small subsets of what Prod looks like.
--------------------
House Construction
1. LR/PC 9.52
2. HTTP
3. No
4. Concurrent
5. n/a
6. New
7. 40k Web, 2k SOA, 1k DB
8. Yes
9. Server 2003
|
Alexander Podelko
Member
Reged: 06/11/03
Posts: 133
Loc: Stamford, CT
|
|
Well, if we talk about agile development I don't see any problem with performance testing at all: you get a working build each iteration - test it. What is a problem? It should be synonym, not oxymoron.
All issues mentioned above boil down to one: nobody wants to pay for it (so no testers, no equipment, etc). Don't see anything especially agile in it. Maybe companies still don't switch to agile development for major projects - and don't want to spend money for minor projects. Another explanation may be that in most traditional projects performance testing happens at the very end instead of every iteration (so require less resources) - but it not the way to do it properly.
I see significantly more problems doing performance testing properly during traditional development. I wrote a paper http://www.alexanderpodelko.com/docs/Agile_Performance_Testing_CMG08.pdf that has exactly the same title as the thread, but completely different content - how to do performance testing in a agile way for any kind of projects (where agile projects look like a trivial case for me).
Well, returning to the original question, there are a few articles in the Internet about the subject:
http://www.jamiedobson.co.uk/?q=node/53
http://www.logigear.com/newsletter/explanation_of_performance_testing_on_an_agile_team-part-1.asp
http://www.logigear.com/newsletter/explanation_of_performance_testing_on_an_agile_team-part-2.asp
http://msdn.microsoft.com/en-us/library/bb924361.aspx
http://perftestingguide.codeplex.com/Wik...%20Test%20Cycle
--------------------
Alex
http://www.alexanderpodelko.com
Edited by Alexander Podelko (04/23/09 06:31 PM)
|
AndriyMelnyk
Newbie
Reged: 07/24/09
Posts: 1
|
|
Alex,
What if you have new build every day and, say, 10 configurations to test on and 2-3 test engineers. Do you think managing performance testing in this case is a trivial task?
|
Previous
Index
Next
Threaded
Edit
Reply
Quote
