I started taking a look at Paul Bett’s Surf project to do builds for things I work on. Currently I have things building in various other places, like Travis CI, Circle CI, etc. All of these options have one thing in common: they run your build in a Linux container that gets started on every build.
This worked fine for me, in fact I was really impressed with both services. But part of the build process was getting the environment in the right state. Installing packages with apt-get, pulling down some sources, building and installing them with make, etc. This got to the point where 90% of the build time was going to preparing the environment for the build. It eventually came to the point where we needed to be able to build our own container with all of the prerequisites already on it. We also needed something to do the actual building.
Enter Surf. Surf gives us exactly what Travis CI gave us. It checks out your repository, runs a build, and updates the GitHub PR status. That’s it. It’s hugely appealing because it’s stateless, built on node.js, and doesn’t even have a GUI. Contrast this with something like TeamCity or Jenkins, where you need to setup a database, spend time configuring remotes, builds, etc, finding the right plugins to update GitHub PR statuses, etc. Since Surf is stateless and very simple, it also made some sense to run it in a container.
Installing Surf is simple enough. It’s just a
npm install -g surf-build.
There isn’t anything more to it.
There are two commands that surf gives that are of interest at this point:
surf-build is the command that will actually check out a your git repository
and run a build. Surf will try its best to figure out how to build your project
for you, but the option that works best for me is to just have a file called
build.sh (or .ps1 on Windows) in the root of your repository. Whatever you
put in your build script is how your project gets built. It could run MSBuild,
Cake, Make, etc. If the exit code is zero, your build passed.
surf-build by itself simply just runs the build with the git hash you give it.
It works like this:
surf-build \ -s 56920f57db4afba1262b6969f577aaedd5e48b36 \ -r https://github.com/vcsjones/AuthenticodeLint.Core
As always, I experiment with new ideas on my own projects first. This will run my build on the Git hash with the GitHub repository. That’s all it takes.
Surf in a Docker image is especially useful because I can have my whole build environment wherever I am. If I have surf in a Docker container, all I need to do is pull-down my docker image (or build it locally) and simply do this:
docker run -e 'GITHUB_TOKEN=<github token>' \ -t 720adcff1217 \ surf-build \ -s 56920f57db4afba1262b6969f577aaedd5e48b36 \ -r https://github.com/vcsjones/AuthenticodeLint.Core
A few things.
surf-build expects an environment variable called
GITHUB_TOKEN to be able to update the pull-request status. It
will also use this token to publish a secret gist of the build’s log.
If you omit the
GITHUB_TOKEN, Surf will still build it, but only if
the repository is public, and it won’t set a pull-request status.
surf-build is fine and all, but it’s entirely manual. We don’t want
to have to run
surf-build ourselves, we want to have surf watch our
repository and run
surf-build for us. Enter
surf-run. This command
does exactly what I want - it runs
surf-build, or any command really,
whenever there is a new pull request, or when a commit is added to an
existing pull request.
It works like this:
surf-run \ -r https://github.com/vcsjones/AuthenticodeLint.Core \ -- surf-build -n 'surf-netcore-1.0.1'
surf-run watches the repository we specify, and starts whatever
process you want, as specified after then
--. It also sets
two environment variables,
SURF_REPO. This is how
surf-build knows what git hash to build instead of being passed in
Running in Docker
My Docker image needs a few things. It needs node.js to run Surf,
it also needs .NET Core, to start. I needed to pick a base image,
so I went with
nodejs:boron which is the 6.x LTS for node. I
chose this instead of one of the .NET Core images because I found
that installing .NET Core from scratch on an image was actually
easier than installing node.js. Now I need to put together a
Dockerfile with everything I need. To start I need all of the dependencies:
RUN apt-get install -y --no-install-recommends \ curl \ fakeroot \ libunwind8 \ gettext \ build-essential \ ca-certificates \ git
Some of these are dependencies I need for some projects, others are needed
by surf or .NET Core, like
libunwind8. These are the commands to install
.NET Core 1.0.1 on Debian Jessie, as verbatim from the Microsoft install
RUN curl -sSL -o dotnet.tar.gz https://go.microsoft.com/fwlink/?LinkID=827530 \ && mkdir -p /opt/dotnet && tar zxf dotnet.tar.gz -C /opt/dotnet \ && ln -s /opt/dotnet/dotnet /usr/local/bin
This next step is a bit of a work around. I wanted my images as ready-to-go
as possible before actually running them. The
dotnet command will do some
“first run” activities, like pulling down a bunch of nuget packages for the
.NET Core runtime. To do this when making the Docker image, I simply create
a new .NET Core project with
dotnet new in the temp directory, then remove
RUN mkdir -p /var/tmp/dotnet-prime \ && cd /var/tmp/dotnet-prime && dotnet new && cd ~ \ && rm -rf /var/tmp/dotnet-prime
There is an open issue on GitHub to facilitate this first-run behavior without side effects, like creating a new project or needing a dummy project.json to restore.
Next, we install Surf:
RUN npm install -g email@example.com
I locked to beta.15 of surf right now, but that might not be something you want to do.
Finally, we specify our command:
CMD surf-run \ -r https://github.com/vcsjones/AuthenticodeLint.Core \ -- surf-build -n 'surf-netcore-1.0.1'
Now we have a Dockerfile for .NET Core with surf on it. With my Docker image running, I tested a pull request:
Success! This is exactly what I wanted. Surf publishes the build log as a gist, a simple way to view logs.
The actual build script in
build.sh is a simple
dotnet restore and
dotnet test in the test directory. As far as the container itself, I
have it running in AWS ECS which works well enough.
All in all I’m super happy with surf. It does nothing more than I need it to, and I don’t have anything complex set up. If the container instance starts misbehaving, I can terminate it and let another takes its place. Having everything in a container also means my whole build environment is portable.