Building a Development Environment Part 1: Managing your sourcecode with Subversion
November 4, 2006
This first installment in a series of posts about building a development environment is about repository lay-out and getting the code out of your source control system.
I originaly was planning on writing a series of articles about how we implemented continuous integration (CI). However, while experimenting and searching the internet for information, I quickly came to the conclusion that this whole CI thing was just to narrow in scope. There is a lot of information and a lot of blogs discussing CI, but most of them are people starting to write a simple application and them being the only one working on it. So, no projects made of multiple subprojects, no projects shared by other projects, no developer interaction, etc…
But in a real company you do have several developers, you do enable reuse and thus use a project in multiple other projects, you do have organisational standards (how projects must be setup, where code must be commited in the source code repository, etc…) that must be applied, etc… Some of this interacts with CI and this is way I made the scope of the series bigger:
How Do I Build An Environment To Develop Software?
This post is by no means “The Way” of building a development environment. In fact, you will notice that at times I deviate from some of the practices advocated by the referred articles at the end of my articles. It’s our interpretation of the process and how we implement it. If you have any remarks or suggestions for improvement, please post a comment.
- Subversion (version 1.3.1): This is the source control application and is run on the source control server for central code management and the client part is on the developer box for retrieving the code to wrk on the application.
- TortoiseSvn (version 220.127.116.1104): This is an optional but very handy tool for windows shell integration of the Subversion client.
Installing the software
Installing the software is a no brainer. Just download the installation packages and execute them. The Subversion installation will add the path to the binary folder to your path environment variable. So you can execute commands directly in any folder.
Organizing the source control server
A universal thruth: “Everything must be seen in it’s proper context”. Weather it’s a real life event, a program method call or software proces. So here’s our context:
We are a shop with a few developers working mostly on a few products but which are closely related. So we have some code that is used in all our products, and some code that is just for a single product. However, this being “code for a single product” is relative: it might be that this code eventually wil end up in some other product. So, we treat all our products as being a collection of modules that could be reused.
The products evolve mostly by customer driven changes which can be implemented in a rather short timeframe. However, there are also sometimes major adaptations which take a long time to implement. What this all comes down to is that we don’t want an integration failure of a long term project jeopardize the integration of a short term project or vice versa. So for each project we create a codeline on the source control server which is completely seperate from eachother and from the main codeline.
Also, we would eventualy like to implement CI. And in this CI we will monitor of course our main code line and the project codelines. However, we do not want to monitor any release folders because the code placed in there is allready integrated: that code comes from the main code line or project code lines.
Thus the layout of our code lines is like this:
We have four basefolders in the root of our repository:
- Main: this is what the Subversion documentation calls “Trunk”
- Projects: this is a folder where we branch our sources for long term development, experiments, etc…
- Release: this is where we put our released sources. Subversion typically names this folder “Tags”
- Extern: this is where we put libraries from external sources
The Main folder has three subfolders: Core, Product and Unittest.
The Core folder contains the modules we develop. A module is a group of sourcecode files which produce one single target file of type exe or dll.
The Product folder contains subfolders with the name of the Products we develop. It contains Visual Studio solution files which represent components out of which the product exists and the buildscripts for the product.
And the Unittest folder contains the same Module folders as the Core folder but extended with Unittest to indicate thaht they are test modules.
The Project folder has subfolders for each project we are currently working on. These project folders have the same structure as the Main folder: they also have subfolders Core, Product and Unittest.
This is logical:
If the project is adding a new feature to an existing product, then this folder contains the modules that make this product.
If the project is about making a new product, then again it contains the existing modules that are used by this new product and some new modules which are not yet in the Main codeline.
This folder contains, as subfolders, all existing modules. As subfolders of these module folders, we have the release folders. They are named after the release label and contain the sources for that release.
This folder contains, as subfolders, all external modules. As subfolders of these module folders, we have the release folders. They are named after the complete version number of the external component. If you would for example use Nant as an external library, then your subfolder might be named “0.85-rc4″
Let’s get praticle
So, after this explanation of our code management strategy, let’s get our hands dirty and implement it.
Organising our experimentation environment
In practice, we can have a seperate Build Server, Source Code Management Server, and of course seperate Developer Workstations.
Me, I only have one portable computer. So, to be able to simulate all these computers, I simply used a few folders:
The devenv folder is simply a folder on my computer that I have made to experiment with developing a DEVelopment ENVironment
The buildsrvr subfolder mimics our BUILDSeRVeR and the scmsrvr folder mimics our Source Code Management SeRVeR.
We do not only have to organise our sourcecode, but also the access rights of our code repository. But for this we first need a repository.
Execute the following on the commandline:
(You can of course use your own folder and drive letter)
Now that we have a repsoitory, we will have to access it. We use svnserve to make our repository available. I’ve made a simple batch file with following line in it:
Now we can set the access rights to it.
In the folder D:\Serge\Devenv\simulation\scmsrvr\data\conf we have three files which provide this functionality:
- svnserve.conf: this files contains some default security settings (like anonymous access) and where to find the users and their passwords
- authz: contains the users
- passwd: contains the passwords
The svnserve.conf file looks like this
### This file controls the configuration of the svnserve daemon, if you
### use it to allow access to this repository. (If you only allow
### access through http: and/or file: URLs, then this file is
### Visit http://subversion.tigris.org/ for more information.
### These options control access to the repository for unauthenticated
### and authenticated users. Valid values are “write”, “read”,
### and “none”. The sample settings below are the defaults.
anon-access = read
auth-access = write
### The password-db option controls the location of the password
### database file. Unless you specify a path starting with a /,
### the file’s location is relative to the conf directory.
### Uncomment the line below to use the default password file.
password-db = passwd
### The authz-db option controls the location of the authorization
### rules for path-based access control. Unless you specify a path
### starting with a /, the file’s location is relative to the conf
### directory. If you don’t specify an authz-db, no path-based access
### control is done.
### Uncomment the line below to use the default authorization file.
authz-db = authz
### This option specifies the authentication realm of the repository.
### If two repositories have the same authentication realm, they should
### have the same password database, and vice versa. The default realm
### is repository’s uuid.
realm = CodeRepository
The authz file looks like this
autobuild = rw
serge = rw
* = r
Read access to anyone
Allthough in the svnserve.conf file we added the line “anon-access = read”, in the authz file you still have to add the line “* = r” (see last line of the file). In the file created by svnadmin when creating the repository this line siimply says “* =” which translates to everybody having no access to the root.
We will eventualy setup CI in this series and will use Cruisecontrol.NET. Cruisecontrol.NET uses the log command of subversion to decide if something changed in the repository, and thus if an integration must be build. Unfortunatly, this subversion command fails with a message “Item is not readable” if you didn’t provide read access to the root for anyone in the authz file. It even fails if you logged on to subversion with an account having read/write access rights.
It took me a wilhile to find this because one would think that providing read access to anonymous users and read/write access to others would be enough. However, it apparently is not sufficient. It somehow looks like logging the repsoitory history uses an anonymous account. I do not know if this is me or if it is a bug in subversion.
The passwd file looks like this
autobuild = autobuildpwd
serge = sergepwd
If we now start the batch file created above, our repository is available and we can create the folder structure described above in our repository using TortoiseSVN or by executing following commands in a commandline window:
svn mkdir -m “Main folder” svn://localhost/Main
svn mkdir -m “Core folder” svn://localhost/Main/Core
svn mkdir -m “Product folder” svn://localhost/Main/Product
svn mkdir -m “Unittest folder” svn://localhost/Main/Unittest
svn mkdir -m “Projects folder” svn://localhost/Projects
svn mkdir -m “Release folder” svn://localhost/Release
svn mkdir -m “Release folder” svn://localhost/Extern
Well, that’s it for this entry. In the next part we will setup CruiseControl.NET to monitor any changes in our repository. See you then…
 Version Control with Subversion
 Subversion Version Control: Using the Subversion Version Control System in Development Projects. There is a downloadable version available by following the Downloads link on the books page.
 Streamed Lines: Branching Patterns for Parallel Software Development
 The Joel on Software Discussion Group – Source Control and VS.NET
 Blogtastic – How to setup a .NET Development Tree Wrapup
 Better Builds with Maven
4 November 2006: original version
27 November 2006: added some references
10 januari 2007:
- restructured the Core folder: it now is made of several subfolders which contain modules, and modules can have only one buildresult, either an executable or dynamic library.
- Added commands to execute on commandline to build the repository.
- Added support for external libraries