biscuits
Newbie
Reged: 01/28/08
Posts: 8
|
|
Hi all, long time lurker, first time poster here. I'm more after your experiences, rather than specific advice, but either are welcome.
I've just joined a small media company, to implement structured testing. We currently use the following environments throughout the software development lifecycle:
1. Dev checked out (developers local machine)
2. Dev checked in (the main development environment)
3. Staging
4. Production
Dev checked out and in share a database. Staging is used to check the release and client configurations.
The past few months I've been observing how QA is undertaken, and I'm now taking steps to improve the whole process, and to formalise testing with the software development team.
One of the "issues" that I have with the environmental set up is that Dev checked in is a hybrid of current and future work, therefore quality and confidence are reduced. I am proposing that we build a dedicated test environment, which only contains the current version of our web based software. I see a test environment as an area that is far safer than the sandbox that dev typically is, and it's an environment that QA can police and monitor. Costs will be minimum, as we have the licenses and hardware already.
A shared database alarms me, as does the hybrid codebase. It has been suggested that the development team tighten up their use of CVS, but I don't believe that's a solution for a stable and confident testing environment. I have the support of the senior developers that I have discussed this with, and I'm gradually working my way up the company heirarchy, gathering support, so hopefully I'll be able to report postive news soon! I have to admit, there are one or two individuals who do not see the benefit of a controlled and isolated environment, away from the perils of developers who incorrectly check data in and out - I'm working on those!
Previous companies I have worked at have all had distinct dev/test/staging/production environments, which I have found to be reliable in terms of testing confidence - do you run similar set ups, or have worked with similar, or do you test in a development environment and release into Production?
thanks in advance, I'm looking forward to hearing about your environments.
.:biscuits:.
--------------------
"Can't you just test it on the live server? It will save us a lot of time if you do."
|
canetti
Junior Member
Reged: 10/26/01
Posts: 168
Loc: Suffern, NY, USA
|
|
Except for the fact that you mentioned working for a media company, I'd swear we working at the same place 
We too use CVS with three environments, dev, staging and production. The developers have their own servers that they do most of their work on before uploading to dev, but we still run into the problem of having code change underneath us while we are testing. The good news is that we recently got approval for our own test environment and are in the process of getting it set up. We will benefit by having our own stable environment and not have to worry about being in the middle of a test when a developer decides to deploy.
--------------------
Sandy Canetti
|
blueinatl
Active Member
Reged: 10/20/06
Posts: 746
Loc: Atlanta, GA
|
|
Where I work for some of the products we have a separate QA environment, for others we have to test in development. The testing is a lot more thorough and accurate in the QA environment than it is in the development environment for exactly the reason you mention - development environment has "stuff" the developers are working on.
We are moving toward having a separate environment and for the project I'm specifically responsible for we will have a separate environment, including all downstream integration points.
Separate is better!
|
stth10
Junior Member
Reged: 01/26/06
Posts: 371
Loc: Stockholm, Sweden
|
|
Hi,
I would like to share how we handle test environments.
We have 13 separate test environments primaly for system testing (thank you lord for virtual server concept, no brands mentioned) and counting...
Why so many?
We develop a standard system which can be configured infinetly for each customer. In addition the test scenario cycles could span for months (system time) so sharing test environments between testers could be difficult.
Is it expensive?
Really not that much, except for initial costs for hardware and so on. Our App. server is Open Source and DB license does not increase when adding more databases to server.
Is it hard to maintain?
In order to automate batch jobs and so on some initial work is required of course but when framework is in place, you only have to deal with deviations which is available in real-time using a dashboard web page.
How to get new AUT versions?
Also done by automation where in a single web page, you choose which test environments to be upgraded or not.
How is this possible?
Each tester has pretty good knowledge about Databases and SQL which allows for freedom in the usage of one or more test environment for test execution in parallell. We aim to be independant from using other resources when handling the test environments.
What about test data?
We use different types of test data (small - or deindentified production sets) and use backup-restore a lot. I also developed some test tools which creates test data or gets random data based on test data requirements.
--------------------
/Stefan
http://abouttesting.blogspot.com/
http://advancedqtp.com/
|
JBatLM
Newbie
Reged: 06/12/07
Posts: 4
Loc: Utah, USA
|
|
First... separate test environment: This is a must for any sort of reliable testing. The test results are no more reliable than the stability of the test environment allows them to be... and, if the test environment is changing... then the results are not reliable.
Second... was it a rhetorical question about why not test in production? If not, I suspect those who wonder that are closely related to those that think reliable testing can/does occur in an unstable/unsecured environment. As for the answer to this 2nd question... it's simple... no one I've met, thus far, at any rate, writes perfect code... hence the need to test it and find the errors. Do your questioners really want those errors to be aired to their public... in production, even before the problems have surfaced? I'd hope not. 
|
michaeljf
Veteran
Reged: 09/17/01
Posts: 3456
Loc: Yankee Land
|
|
Rather than repeat, here is how I did it in one place.
Sharing Environments
--------------------
- M
Nothing learns better than experience.
"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown
Now wasting blog space at QAForums Blogs - The Lookout
|
DSquared
Moderator
Reged: 04/02/03
Posts: 4546
Loc: Wisconsin, USA
|
|
Why can't you move code into staging and test there before that environment is actually used to stage a production release?
|
biscuits
Newbie
Reged: 01/28/08
Posts: 8
|
|
Quote:
Why can't you move code into staging and test there before that environment is actually used to stage a production release?
Good thinking, but we use our staging environment to demo applications to clients, so that's not really possible - last thing I want is to be responsible for taking out the features being demonstrated!
Thanks for everyone who has replied so far, it's interesting to see what sort of environment set up people are using or prefer
--------------------
"Can't you just test it on the live server? It will save us a lot of time if you do."
|
Joe Strazzere
Moderator
Reged: 05/15/00
Posts: 9988
Loc: Massachusetts, USA
|
|
Quote:
Good thinking, but we use our staging environment to demo applications to clients, so that's not really possible - last thing I want is to be responsible for taking out the features being demonstrated!
So then it's not really a staging environment. It's a demo environment.
--------------------
- Joe
I speak only for me. I do not speak for my company, or for anyone else.
Visit my NEW blog All Things Quality now at http://strazzere.blogspot.com
|
DSquared
Moderator
Reged: 04/02/03
Posts: 4546
Loc: Wisconsin, USA
|
|
You beat me to it, Joe.
Bisquits - certainly sounds like you are one environment short.
|
michaeljf
Veteran
Reged: 09/17/01
Posts: 3456
Loc: Yankee Land
|
|
Yup, it looks like its really:
1. Dev checked out (developers local machine)
2. Dev checked in (the main development environment)
3. Demo
4. Production
But then if you are using your Demo for checking client configs does it also have its own database? Are you then set with 3 databases one for the two Dev env's? One for Demo and one for Production?
Seems like you need another layer in there, especially since what this looks like to me is that the Local Machine is really whatever sits in the Dev's desk and they build on the other one.
--------------------
- M
Nothing learns better than experience.
"So as I struggle with this issue I am confronted with the reality that noting is perfect."
- Unknown
Now wasting blog space at QAForums Blogs - The Lookout
|
Previous
Index
Next
Threaded
Edit
Reply
Quote
