Mastering Elixir CI pipeline

Developer time is precious for business. Don't waste it on things that can be easily automated. One of these things is a part of the code review process - quality checks. Continuous integration (CI) is where this automation is put in place, and in this article, I'd love to walk you through our setup.

Why should you use Continuous integration?

Simply put, we're just humans that tend to forget about stuff, but even more importantly, we get bored as soon as the activities become monotonous. In project quality checks, these two disadvantages can be fixed thanks to Continuous Integration. Here are some benefits of CI:

• you make sure that your project builds (or runs)
• you make sure that tests are not failing
• you make sure that the quality guidelines are met
• you save time on repeatable manual code review checks

Does it mean that we don't need a developer to verify the code? Not really, we're still not at this stage, but you can treat CI as a solid tool in the code development process that should be a part of every Pull Request cycle. In other words, don't merge anything until your CI is green.

Step by step, what can we verify in Elixir CI

My first pipeline in Elixir probably included only build and testing steps. It was back in the time when Elixir formatter was not even a thing. The toolset got more mature and advanced, and so did my pipeline. I'm a fan of putting as many steps in Elixir CI as possible. Why? Well, because it means I automated everything I could (at least in my understanding) and the code reviewer has fewer things to concentrate on.

Here is a list of every step I would currently put into Elixir CI. I would love to hear from you in the comments if you have even more to suggest (after all, as I said, I'm a fan of big CI's 😉).

1. Fetching dependencies

mix deps.get

Simply put, you want to make sure that all of the dependencies declared in mix.exs can be fetched. These are needed to build your project, as it's nearly sure that your code depends on some sort of external libraries.

Check more: https://hexdocs.pm/mix/1.12/Mix.Tasks.Deps.Get.html

2. Checking retired deps

mix hex.audit

This simple & cool check will make sure that packages in your project are not retired and are no longer recommended to be used by their maintainers. Right after fetching the deps, this step is yet another way to verify the dependencies' validity.

Check more: https://hexdocs.pm/hex/Mix.Tasks.Hex.Audit.html

3. Finding unused dependencies

mix deps.unlock --check-unused

With this simple mix task, we're able to detect the libraries that are a part of mix.lock while at the same time are not used in the code. Why compile a project with something we don't use?

Check more: https://hexdocs.pm/mix/1.12/Mix.Tasks.Deps.Unlock.html

4. Security audit of deps

mix deps.audit

Mix audit is an external dependency, so make sure to install it first.

This step will scan Mix dependencies for security vulnerabilities. The author of this library was inspired by tools like npm audit and bundler-audit. The list of found vulnerabilities is far from perfect, but still, I would advise putting it into the pipeline.

This might not, however, be the best choice for you very soon. If you're using GitHub, you can take a look at GitHub Advisory Database which just welcomed Erlang and Elixir packages: https://github.blog/2022-06-27-github-advisory-database-now-supports-erlang-and-elixir-packages/.

Check more: https://github.com/mirego/mix_audit

5. Checking code formatting

mix format --dry-run --check-formatted

I strongly advise you to include a mix formatter in your project. There are three perfect reasons for that:

1. Your code starts to look nice. Although everyone has their own style of code formatting, we tend to forget to format some parts of the code sometimes, and in the end, it always becomes a mix of different ideas. The result - the code is ugly. Formatter fixes this problem.
2. No need to even for a second concentrate on code style in code review and this is for me the biggest win. I saw this countless times - two devs arguing on the best way to indent the code, using brockets or not, or... you know what I mean. These kinds of discussions are counterproductive and can be replaced with activities that do actually push the project forward (developing new features).
3. Unification of code styling rules.

There is one disadvantage I can name, but it just doesn't outweigh the advantages - disrupting the git history. If you integrate formatter at a late stage of the project, your commit history is already big enough to make some use of git blame. Formatting the whole project creates one big commit, and you don't know who to blame. This problem disappears in a couple of weeks usually, as soon as new commits go on top of history.

As you can see, in this step, we're using --dry-run --check-formatted options, which will make sure that:

1. A proper status code is returned if the project is not formatted properly.
2. No saves will be made to the project files.

Check more: https://hexdocs.pm/mix/main/Mix.Tasks.Format.html

6. Compiling the code

mix compile --all-warnings --warning-as-errors

Finally the build! Why not sooner? Shouldn't we compile first instead of performing 3 steps of dependencies verification?

The compilation itself may take some time. The bigger the project, the more time compiling consumes. Some of the steps in Elixir CI depend on another, and the previous ones could be run without code being compiled.

Deps verification steps are fast, so I believe that if we can get the failure status sooner, then let's take this chance. That's why compiling happens now.

The clarification of this step is simple - we should not ship a product that doesn't run. You may be wondering, though, why I also used the --warning-as-errors flag. This option will make sure that the code won't compile if there are any warnings. From my experience, the warnings are actually pretty important stuff, so I think it's not a good idea to allow these in the main branch. You can read more about it in one of my TILs: Treating warnings as errors in Elixir's mix compile.

Check more: https://hexdocs.pm/mix/1.12.3/Mix.Tasks.Compile.html

7. Verify DB migrations ability to fully rollback

mix ecto.create && mix ecto.migrate && mix ecto.rollback --all

!!! Important !!! This might not be required. If your project doesn't need a database, you can move on to step 8.

---

If given migration can run successfully, but can't rollback, it means that it's not defined in a proper way. Some of the operations require you to define both up and down functions in migration. This step will make sure that you can run migrations, and then rollback them all. In other words, if there are any migrations that can't be rollback, you'll get notified.

---

Check more: https://hexdocs.pm/ecto_sql/Mix.Tasks.Ecto.Rollback.html

8. Static Elixir code analysis with Credo

mix credo --strict

Refactoring opportunities, complex code fragments, spotting common mistakes, showing inconsistencies in your naming scheme and - if needed - enforcing a desired coding style. This is a short description of Credo.

I strongly advise putting this into the pipeline. Credo allows you to specify which rules you'd like to verify so that you and your team can decide on checks that you perceive as desired.

Credo assigns priorities to reported suggestions, and only the ones with the highest priority are treated as a failing scenario. I like to include --strict flag to also include the ones with lower priority to failing cases.

Read more: https://hexdocs.pm/credo/basic_usage.html

9. Security-focused analysis with Sobelow

mix sobelow

Sobelow performs security-oriented static analysis of your code. You can't check whether your app is 100% secured with this step, but it can help you prevent introducing common vulnerabilities.

Here is a list of checks it can perform:

• Insecure configuration
• Known-vulnerable Dependencies
• Cross-Site Scripting
• SQL injection
• Command injection
• Code execution
• Denial of Service
• Directory traversal
• Unsafe serialization

Read more: https://github.com/nccgroup/sobelow

10. Static Elixir code analysis with Dialyzer & Dialyxir

mix dialyzer

Thanks to Dialyxir we're able to include Dialyzer in our Elixir CI. As you could see, we already performed static Elixir code analysis in step 6 with Credo. These are, however, two different tools.

Dialyzer, amongst other information, will print out type mismatches, which is at the moment the best thing we can do to assure type integrity in the Elixir app. As the documentation states, Dialyzer is just useful in any repo, but its performance can be boosted with spec's which I personally consider a must in every Elixir app (until there is a better way to do it).

Important! The first usage of Dialyzer takes time, keep this in mind. Once the plt file is generated every new run will take much less time.

Read more: https://hexdocs.pm/dialyxir/readme.html

11. Verifying documentation coverage

mix doctor

Documenting a project can be done in many different ways. I always put some pressure on function and argument naming, as this is the first place where I try to understand function meaning. Sometimes though it's not enough, and that's why we needs documentation.

The doctor library ensures that your project is well documented, or in other words, has proper documentation coverage. The specification of thresholds can be done inside of .doctor.exs config file, so it's really up to you and your team what type of coverage you can accept.

Check more: https://github.com/akoutmos/doctor

12. Running tests

mix test

And of course, the step that we just can't miss - running tests 🙂 I believe this step doesn't need a description, but let me give you an idea of how you can improve it.

With the use of one of the coverage analysis modules, you can use the coverage option to perform some checks. One of these checks can for instance verify test coverage percentage which seems like a good thing to enforce in the project. You can read more about this option here as well as check ExCoveralls module as an example of a module that can be used for analysis.

Check more: https://hexdocs.pm/mix/1.12/Mix.Tasks.Test.html

Setting up a local Elixir CI to prevent git commit

All of the above steps can be set in remote CI sysytem, like:

• GitHub Actions
• CircleCI
• Buddy
• Travis

or really, any other CI tool that allows you to run Elixir code (possibly in Docker). But what if you'd prefer to catch repo issues before it even reaches remote CI? What if we could do it before git commit ?

Setting up a git pre-commit hook

The first solution is pretty simple, we can use a git pre-commit hook. There are a couple of easy steps here.

First, copy the pre-commit sample file:

cp .git/hooks/pre-commit.sample .git/hooks/pre-commit

---

Set the pre-commit file as executable:

chmod +x .git/hooks/pre-commit

Open the pre-commit file with an editor of choice. I'll use VS Code here:

code .git/hooks/pre-commit

Add #!/bin/bash to the top of the file (if you're using bash) and right after that, in the next line, you can set the chain of commands that should succeed before committing to the repo:

#!/bin/bash

mix format --check-formatted && mix credo

For simplicity, I just presented two operations here, but it's really up to you what will land in your local CI. I'm mentioning this because I spotted a bunch of discussions about using or not using local CI's. Most of the developers who were against this idea noted that it's quite painful to run everything and wait for the commit to happen.

I'm definitely a fan of local CI, and here is why:

• I really would love to spot issues sooner, as CI tends to block or run slower than on a local machine
• no one says that we have to put everything here, some checks can be disabled
• git history is free of mistakes that we could have spotted before git commit

Keeping the above in mind I suggest:

• putting all of the operations that can't break the app (checking the correctness of migrations will break it, as it runs migrations and then rollbacks all of them) if your app is very small, and only when it starts to take too much time remove some of the checks
• or including only the operations that happen to perform quickly, like formatter, credo, etc

In general, using the git pre-commit hooks is a very flexible and nice-to-have local CI solution.

Synchronizing the local CI configuration within a team of Elixir devs

Pre-commits are very cool, but what if we'd like our whole team to perform the same local checks? After all, hooks are not transferred as part of the repository, therefore it's only a local, per-user setting.

This type of shared behaviour has to be a mix of Elixir & Git solutions, and this is exactly what git_hooks does.

Setting up git hooks in your repo

First, add a new dependency to your project:

def deps do
  [
    {:git_hooks, "~> 0.6.5", only: [:dev], runtime: false}
  ]
end

Disclaimer: At the moment of writing this article, version 0.6.5 seems to be stable on the contrary to the most recent one. Please verify if that's still the case and choose the one that works for you.

Next, fetch git_hooks dependency:

mix deps.get

The last thing we have to do is add a config in config.exs :

use Mix.Config

# somewhere in your config file
if Mix.env() == :dev do
  config :git_hooks,
    auto_install: true,
    verbose: true,
    hooks: [
      pre_commit: [
        tasks: [
          {:cmd, "mix format --check-formatted"}
        ]
      ]
    ]
end

The above is just a sample configuration. Let's take a look at what it does:

• The auto_install: true option will make sure that when you compile the project, the pre_commit Git file will be prepared to run local CI. You can make it false, but then keep in mind that you'll have to manually run the installation.
• The verbose: true option will disable tasks output logging.
• Hooks define one by one the type of hooks you'd like to install in your project. Within each of the hooks, you define tasks, which can be any task performed in bash (or another shell). Mostly, you'll run Elixir tasks here.

If the auto_install option is enabled, hooks will be installed during project compilation:

Let's take a look at what's actually injected into the pre_commit hook in this case:

#!/bin/sh

[ "/Users/user/local_ci/deps/git_hooks" != "" ] && cd "/Users/user/local_ci/deps/git_hooks"

mix git_hooks.run pre_commit "$@"
[ $? -ne 0 ] && exit 1
exit 0

As you can see, it simply runs a mix task against the pre_commit defined within the config.

Here is what happens when you'll try to run git commit :

↗ Running hooks for :pre_commit
** (Mix) mix format failed due to --check-formatted.
The following files are not formatted:

  * mix.exs
  * config/prod.exs

× `pre_commit`: `mix format --check-formatted` execution failed

As my code wasn't formatted, git commit failed which is exactly what should happen.

Local Continuous integration summary

From my point of view using the git_hooks library is a great way to perform local Continuous integration checks within the team. Make sure that the whole team agrees on the given steps, and choose them wisely accordingly to project size (some operations might just take too much time).

Summary

Continuous integration is a must in every Elixir project. In this article, I showed you what the options are with Elixir, but keep in mind that it might not be a definite guide. I'm still looking for more steps to include in this process, because, as I said, I'm a fan of putting as much as possible into automatic checks to spare some of it during the manual code review process.

FAQ

What is Continuous Integration (CI) in Elixir and why is it important?

Continuous Integration (CI) in Elixir ensures automated quality checks during code development, promoting reliable code builds, successful tests, and adherence to quality guidelines, saving time on manual reviews.

How can I ensure my Elixir project's dependencies are correctly managed in CI?

In Elixir CI, manage dependencies by using mix deps.get for fetching, mix hex.audit for checking retired dependencies, and mix deps.unlock --check-unused for identifying unused libraries.

What steps should be included in an Elixir CI pipeline?

A comprehensive Elixir CI pipeline includes steps for fetching dependencies, checking for retired or unused dependencies, auditing for security, ensuring code formatting, compiling code, verifying database migrations, static code analysis, and running tests.

How can I ensure code quality and security in my Elixir project?

Ensure code quality by integrating mix formatter, and enhance security by using tools like mix deps.audit for vulnerability scanning and mix sobelow for security-focused static analysis.

How can local Continuous Integration be implemented in Elixir projects?

Local Continuous Integration in Elixir can be implemented using git pre-commit hooks to run checks like code formatting and credo analysis before committing, and by synchronizing configurations across the team with tools like git_hooks.

What are the advantages of local CI compared to remote CI systems in Elixir projects?

Local CI allows for quicker detection of issues on a local machine, reducing the number of mistakes in the git history. It's customizable, enabling developers to include or exclude checks based on project size and requirements.

What are some best practices for setting up an Elixir CI pipeline?

Best practices include automating as many steps as possible, ensuring all code passes through checks before merging, and setting up local CI to catch issues early. Engage with your team to agree on checks that are essential and relevant to your project's size and complexity.