In the last few years, software development teams have had a revolution in which teams are exponentially accelerating the rate at which they develop software. One key factor in this revolution is the concepts of continuous integration and continuous delivery. In order for teams to successfully implement these practices, they need to leverage tools to help automate how their projects are built, tested, and deployed. This creates reliable and automated deliverables that can be deployed quickly. Using tools like Drone http://try.drone.io/, teams can quickly get feedback from their code, their end users, and help close the feedback loop.
Continuous Integration (CI) is the idea that developers should leverage version control software and automated tools to continually integrate their changes with the rest of their team. This reduces the pains of integrating codebases, and speeds up development.
Continuous Delivery (CD) expands on that concept, by suggesting that all code check-ins should automatically integrate changes, build the codebase, run tests, enforce code quality, create deliverables, and even deploy. This process is referred to as the build pipeline.
In order for continuous integration and delivery to be effective, your build pipeline needs to be automated. There are many tools available to help achieve this automation, but most CI/CD tools revolve around the basic concept that your build should be scripted and that you should have a dedicated server to executing your build pipeline.
This blog post will be focusing on Drone as our build server to implement a scripted, automated build pipeline. While this example is basic, in can be modified and expanded to your own projects’ needs.
So what is Drone, and what makes it special? Drone is a build automation server, which means it is designed around building, testing, and deploying your code base. Drone relies heavily on Docker to achieve this functionality. Projects define their steps and dependencies using a superscript of Docker Compose. Each step references a base Docker image and specifies the commands to be run. These specialized Docker images are known as plugins. Drone provides a handful of Docker images for various language environments as well as cloud deployment tools. You can run Drone locally, but typically it will be hosted on a server. In both instances, Drone will utilize either a local or remote Docker host to run the containers. If you need to modify or create your own plugins, you can host them in your Docker repository.
First, let’s imagine what an ‘ideal’ CI/CD build pipeline would look like:
- A developer is working on code for their project in their own branch. When their code is checked in, the CI/CD server should automatically run the code tests and check code quality.
- When a developer is ready to merge their code into master, they open a pull request. We would like our CI/CD server to merge and test our code for us (but not push the merge yet), to ensure that if the pull request is accepted, that there should be no failures.
- Once the pull request is accepted and merged, we should test one final time for good measure, and then create a deliverable.
- The team should be notified on a build failure, that way we can address the issue quickly.
- Our tests should be self-contained and know how to create any dependencies it needs in an isolated environment.
Automatically Build Our Code
For the first step, set up a Drone pipeline to build code with every commit. This is configured in a file called `drone.yml` located at the root of the project. This file is a superset of Docker compose and looks very familiar.
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
All pipeline steps will be located under the top level property `pipeline`. You can name your steps whatever you’d like, in this example, it’s ‘build’. Then, the image is specified. Since Drone is based on Docker images, you can use any docker image https://hub.docker.com/explore/ or, a pre-configured Drone plugin http://plugins.drone.io/ Also, specify the commands you want to be run on this image, in this case, Maven will compile the project.
Drone will automatically check out the git project associated with this project, and each pipeline step will share a volume with one another, so if you build in one step, you can test in the next step without re-building the project. Each build will start a fresh instance though, so there’s no need to worry about artifacts from previous builds.
Adding External Dependencies
Many projects have external dependencies that need to be configured for use. Drone can define services, also based on Docker images, that will be created before the build pipeline is fired. In this example, a PostgreSQL database is created before integration tests are run. This is then destroyed at the end of the test.
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
services:
database:
image: postgres
environment:
- POSTGRES_USER=db_test_user
- POSTGRES_PASSWORD=db_test_password
- POSTGRES_DB=my_database
In the above example, a secondary top-level property is defined, `services`. Drone will stand up an instance of each sub-property. In this case, a `database` instance will be created. The image references a Docker image. We can additionally add multiple `environment` parameters to be sent as environment variables. This functionality is available on all pipeline and services steps.
This allows applications to create the necessary dependencies they need to run their integration tests, and ensure a consistent testing environment.
Testing
Now that the application’s dependencies are available, a test pipeline step can be added.
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
test:
image: openjdk
commands: mvn install
services:
database:
image: postgres
environment:
- POSTGRES_USER=db_test_user
- POSTGRES_PASSWORD=db_test_password
- POSTGRES_DB=my_database
Another `pipeline` step is defined, this one called `test`. This is based on the same Docker `image`, and instead runs `mvn install` (which runs our tests).
Conditional Builds
By default, each pipeline step is run on all push events. Ideally, Drone should also test any pull requests. Conditional scenarios can be defined as such:
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
test:
image: openjdk
commands: mvn test
when:
events: [push, pull_request]
services:
database:
image: postgres
environment:
- POSTGRES_USER=db_test_user
- POSTGRES_PASSWORD=db_test_password
- POSTGRES_DB=my_database
Here, conditionals steps are added to the `test` pipeline step as `when` and `events`. This tells Drone to run these build steps on not only code pushes, but pull requests as well. Drone will automatically pull the pull request, perform the merge locally, and test the merged code.
Deploy Code
Next, have Drone deploy the codebase, but only when code is merged to a specific branch (`master` in this example). This means that developers can work on code in their branches, and once they merge to the release branch, code can be deployed.
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
test:
image: openjdk
commands: mvn test
when:
event: [push, pull_request]
deploy:
image: cheslip/drone-cloudfoundry
api: api.run.pivotal.io
org: pivotal
space: production
name: my-app
when:
branch: master
services:
database:
image: postgres
environment:
- POSTGRES_USER=db_test_user
- POSTGRES_PASSWORD=db_test_password
- POSTGRES_DB=my_database
There’s a few things happening in the `deploy` step. This step uses the `drone-cloudfoundry` plugin to push to the defined cloud foundry environment. Drone has similar plugins for all major cloud providers, as well as most service orchestration tool.
The `when` property should be familiar with the previous step, but it has a new property called `branch` which defines which branch should trigger this step. Looking back at the desired build pipeline, we only want code to be deployed when it has been merged into the `master` branch. The `branch` conditional can be used to drive any plugin, so if you have branch specific scenarios, you can declare the branch, exclude the branch, or use regex matching to restrict.
The deployment step assumes that you created your artifacts earlier in the `build` step. In this case, our Maven package phase should build a deployable artifact. But, if you’re working with another build tool, you can substitute as needed.
Update Status
Drone does come with an administrative and display page for you to monitor your builds. In order to make sure developers are aware of build failures, messages can be sent too. Using the conditionals from the above example, and a Slack plugin, we can post a message whenever the build fails. Email, sms, and other messaging platforms are supported as well.
pipeline:
build:
image: openjdk
commands: mvn package -DskipTests=true
test:
image: openjdk
commands: mvn test
when:
event: [push, pull_request]
deploy:
image: cheslip/drone-cloudfoundry
api: api.run.pivotal.io
org: pivotal
space: production
name: my-app
when:
branch:
notify:
image: plugins/slack
webhook: https://hooks.slack.com/services/GET_YOUR_OWN_WEBHOOK
services:
database:
image: postgres
environment:
- POSTGRES_USER=db_test_user
- POSTGRES_PASSWORD=db_test_password
- POSTGRES_DB=my_database
Closing Notes
The next step would be setting up a Drone server http://readme.drone.io/admin/installation-guide/ and connecting to your git repository. Once setup, Drone will be ready to start automating your builds. Now you can get instant feedback as you develop your code, and automatically deploy your code to your desired environment.
While this is a simplified example, hopefully, this can inspire those interested in implementing their own CI/CD pipeline. There are many other tasks that should be automated, like the creation of build artifacts (i.e. `npm version` or `mvn release`), the creation of GitHub releases, and launching Docker images. See Drone’s docs for more details on enabling other pipelines: http://readme.drone.io/
Leveraging tools like Drone can help teams enable continuous integration and delivery. This, in turn, reduces the amount of time spent building, testing, and deploying applications and reduces the time to feedback in an agile development cycle.
