So while I was working on setting up a Continuous Delivery toolchain using Kubernetes and Spinnaker I stumbled upon a little problem. Spinnaker can be used with Kubernetes for CD when you have a docker image. But you need some kind of step that transforms your application in to an image. You can trigger this step from Spinnaker by invoking the build
in your favorite Continuous Integration tool.
Packaging your application in to an image often follows align with one of these scenarios:
create the image as a last step of your ordinary build invoking docker build
have a separate step after your ordinary build that pulls
your application artifact from some kind of repository and builds the image
build your application using a build image and let the build artifact be an image
The third alternative has become much more attractive with Docker multi-stage build, which is awesome, where you can have multiple FROM (stage) and copy artifacts from earlier stages.
You can now have both small images and fully reproducible builds, without any complicated scripts.
There’s is a great plugin for Kubernetes that lets you dynamically provision Jenkins agents to run a single build. This is exactly what I want, and I want the agent to build and push an image on every build.
So far we have the Kubernetes plugin in Jenkins that:
* creates a build agent pod
* checks out the code and delivers a reproducible build
docker run -v /var/run/docker.sock:/var/run/docker.sock \
This wont work in this case because we are dependent on the hosts docker version and in my case the host is running docker 1.12.6
with no support for multi-stage and build --network host.
The problem with running sibling containers is that you introduce a dependency on the host where your build container is running. Our hosts and our build containers have very different purposes. Our hosts should deliver a rock solid infrastructure for executing containers. The build environment, on the other hand, can be using the latest and greatest to
give us the best tools available.
So instead of running sibling containers I decided to try dind (docker in docker). I’m now using two container in the build agent pod:
one jnlp with the latest and greatest Docker CLI
one latest and greatest dockerd daemon
The docker daemon is using the vfs storage driver that should work everywhere and accepts connections on tcp://localhost:2375. The Docker daemon must run in --privileged mode.
One of the downsides with this solution is that your build have to pull all images on every build which increases network traffic and slows down the build. One solution to this is that you can easily “pre-pull” the images used during the build, in a container, and commit that as the new
image for your dockerd container.