At my current company we have a build numbering scheme that I'd like some opinions on.
We do nightly builds and we also do 5 "test" builds during the work day. The test builds are simply done to give the developers an indication that recent code changes are not causing problems.
In both cases an automated smoke test is executed.
The first test build of the day receives a number, let's say 1.0.107. When the next test build runs it deletes the old test build and re-uses the same number. This continues until the midnight build comes along and uses the same number. If the midnight build succeeds, the next number will be used for the next day's test builds, 1.0.108. If the midnight build does not build properly the number, 1.0.107, will be used for a second day.
Keep in mind that all builds are kept in the same location and they are labeled with "test_build" or "overnight_build". They are also labeled to indicate if the smoke test passed or failed. All engineers are instructed to not use the test builds.
Please give me your thoughts on our scheme.
I'd much prefer a 1 build / 1 build number scheme. We're using 4 digit build numbers, so why not? Build numbers are free, right?
I have now reported a handful of bugs on the same build number that were actually found with different builds. This happened because the bugs were found in smoke test, otherwise the test build is not used.
The test builds are not supposed to be used, but human nature suggests that when you look down the list of builds and the only one to pass a smoke test is a "test" build, you will most likely use it anyway.
On the flip side, my co-worker claims that having a large number of build labels hurts the performance in VSS. Is this true?
Re: Build numbers
I prefer each build to have a unique build number. But you do not have to label your source code control system with the test build numbers. You friend is right, the more labels you have in VSS the slower it will run -- eventually.
Check out a tool called Anthill (http://www.urbancode.com/projects/anthill/), it is a pretty mature Open Source project, a build management server that does exactly what you describe. You can set up multiple projects in Anthill. You would set up an integration project and put it on a 30 min schedule and tell anthill not to label VSS when this project runs. Then, you could set up another project which builds the same code but runs only once per day (say at 1am) and does label the repository (VSS).
Anthill can also send out emails ininterested parties when it does a build (either a succes or fail email or both). Plus a whole lot of other stuff, like I said -- it is a pretty mature tool.
Hope that helps.
Re: Build numbers
If the test builds are not supposed to be used, why do you use them?
It sounds like you should only be testing the midnight builds, which have unique numbers.
- Joe (email@example.com)