Creating A Sane Build Process

I’m not proud of it (well maybe just a little), but I once created an insane build process once. If Pat (who maintained the build after me) posts in my comments, he’ll tell you about it. Take a stew of a proprietary microcomputer flavor of Fortran written in the 70s by a programmer most assuredly clad in polyester, churn it through a Visual Basic 6.0 preprocessor that spits out Fortran 90 code, all the while correcting memory bound issues, mix it together by compiling it with a custom NAnt fortran compiler task, and voila!, 20 or so compiled Win32 fortran dlls. At this point, the process compiled and sprinkled in some C# code.

I’m not sure that build process will ever run on another machine other than the one it runs on.

To create a sane build process, you need a sane development environment. I’m sure there are many important principles of a sound build process, but I have just one big one to impart for now.

The build must be location independent!

I can’t stress enough how important this principle is. I should be able to walk into your office (assuming you’d let me) and perform the following steps to get a fully working build on my machine.

Set up my computer
Hook it up to your source control system
Set the working folder to any old directory, say J:\IBetThisFolderIsNotOnYourMachine\NorThisOne
Get latest on a solution and open it up in VS.NET (or whichever IDE)
Compile it

If that doesn’t work because you have hard coded file paths, then FOO on you! But let’s not stop there, I should then be able to run your unit tests (what? You don’t have unit tests? A hex and octal on your code!) and they should all pass on my machine (assuming they pass on any machine).

But wait, I’m not done mucking around your office. Next, I should be able to head to your build server, copy the folder that serves as the root of your build process (or better yet, your CruiseControl.NET root) to any folder on my machine, and run your NAnt (or MSBuild) script, and have the system compile correctly and pass all unit tests.

How do you do it?

At first, it takes a bit of practice to get to this point. For example, there should not be a single hard-coded path in your code, nor in your build scripts. Find every way to get them out of there. Here’s a few tips for tricky situations you may run into.

NUnit/MbUnit configuration file

UPDATE: This section was rewritten due to changes in Visual Studio.NET 2005

In VisualStudio.NET 2005, you can include an App.config file in the root of any class library project. Compiling the project will automatically copy and rename the file appropriately (AssemblyName.dll.config) into the output directory. NUnit and MbUnit will use the settings in this file to run the tests. Make good use of this.

Testing With External Resource Files

Suppose your unit tests rely on some external files for testing (like an xml or html file to parse). If you store these files with the code, you can’t be guaranteed that your unit test will find them when running on a build server (since the directory structure may be quite different). You also don’t want to put these files within bin/debug for the reasons mentioned above.

Instead, follow Patrick Cauldwell’s lead and embed the files as resources. Now, your unit test can just unpack the file it needs into a known relative location when it runs, achieving location independence.

Limitations

Of course, there are limitations to location independence. You’re allowed a few assumptions. For example, in the scenario above where I stomp into your office and take your source code, you can assume that I’m running on the same platform you are and have a source control client and IDE installed. Try to reduce these assumptions as much as possible, but we have to agree on some basic axioms.

I’m working on a new build process right now, and I hope to make this one a sane one. Maybe I’ll post examples later when I get done. We’ll see.

Comments

7 responses

Patrick Gannon • August 26th, 2004
Sure, I could just TELL you about it. But what would be the fun in that? The reader can just as easily concoct his or her own mental image of an ominously complicated build process that checks everything in and out of source code after every micro-operation.

Just to contribute one actual tidbit of potentially useful information in this comment: in addition to the qualifications you've specified above, it also seems important that a build process be fast, and thus amenable to continuous integration (or a "continuous-integration-like" build process whereby people don't feel like they need to have the entire system bug free before it's "worth it" to do a build.) Ideally, you would have performance tests for the build (written for NUnit), to measure performance enhancements, then have those performance tests run by an uber build process, and then the uber build process gets sparked when the build process is modified. Interesting.. that started out as a lame joke but might actually be a good idea for someone with a ton of time on their hands.. and a slight case of OCD. ;)
Haacked • August 26th, 2004
Funny you mention it. I keep my build script in source control as a "Solution File". So when I run the build script, it will update itself from source control (though the changes don't apply to the current run).

At some point when I get CruiseControl.NET up and running (there's no Surround SCM support so I have to write a custom SCM task), changing anything including the build file will trigger the build process.

I am running into the perf issue with my current build. The Unit tests create a socket server, simulates multiple clients connecting simultaneously while issuing commands, and then closes the whole thing. This test alone takes a bit of time. Overall, under 10 minutes, but that's a bit long for unit tests to run.
Rick • March 7th, 2005
Im having similar issues with SQL reports. I have an assembly which originally gets its config info from web.config. I see two options, but I dont really like either of them.

If I use resource files, they have to be compiled into another assembly, right? So then I would need to recompile every time I want to change config settings. I could write a custom class to go out and get the config info like the appsettingsreader does, but how to resolve the file location? Not every possible client keeps them in bin Ex. SQL reports.

Am I right in assuming that microsoft has nothing in place for shared components in the GAC???
Community Blogs • April 27th, 2007
A long time ago Patrick Cauldwell wrote up a technique for managing external files within unit tests
Scruffy-Looking Cat Herder • July 18th, 2007
Self-Contained Unit Tests
Jacob • July 18th, 2007
The directions link above seems to be broken, which is too bad because I wanted to see if and how it differs from my approach.
beebe4 • July 21st, 2007
Sad thing is, I know you are working on our build process at work currently (July 2007), and I honestly didn't read the date on this post until I got to the comments. You really must get tired of reliving your life over and over again? Because your comments apply DIRECTLY to what we are going through now... so scary it is almost funny, almost.