Setup - Toolchain, IDEs, CI

• June 06, 2020
• 3989 words
• 20 min

This article is a sample from Zero To Production In Rust, a hands-on introduction to backend development in Rust.
You can get a copy of the book at zero2prod.com.

Chapter #1

1. Getting Started
2. Installing The Rust Toolchain
   • 2.1. Compilation targets
   • 2.2. Release channels
   • 2.3. What toolchains do we need?
3. Project Setup
4. IDEs
   • 4.1. Rust-analyzer
   • 4.2. IntelliJ Rust
   • 4.3. What should I use?
5. Inner Development Loop
   • 5.1. Faster Linking
   • 5.2. cargo-watch
6. Continuous Integration
   • 6.1. CI steps
   • 6.2. Ready-to-go CI pipelines

1. Getting Started

There is more to a programming language than the language itself: tooling is a key element of the experience of using the language. It often has a disproportionate impact on the uptake (or the demise) of the technology itself.

Tooling should therefore be treated as a first-class concern both when designing and teaching the language itself.

The Rust community has put tooling at the forefront since its early days: it shows.
We are now going to take a brief tour of a set of tools and utilities that are going to be useful in our journey. Some of them are officially supported by the Rust organisation, others are built and maintained by the community.

2. Installing The Rust Toolchain

There are various ways to install Rust on your system, but we are going to focus on the recommended path: via rustup.

Instructions on how to install rustup itself can be found at https://rustup.rs.

rustup is more than a Rust installer - its main value proposition is toolchain management.

A toolchain is the combination of a compilation target and a release channel.

2.1. Compilation targets

The main purpose of the Rust compiler is to convert Rust code into machine code - a set of instructions that your CPU and operating system can understand and execute.
Therefore you need a different backend of the Rust compiler for each compilation target, i.e. for each platform (e.g. 64-bit Linux or 64-bit OSX) you want to produce a running executable for.
The Rust project strives to support a broad range of compilation targets with various level of guarantees. Targets are split into tiers, from "guaranteed-to-work" Tier 1 to "best-effort" Tier 3.

An exhaustive and up-to-date list can be found here.

2.2. Release channels

The Rust compiler itself is a living piece of software: it continuously evolves and improves with the daily contributions of hundreds of volunteers.

The Rust project strives for stability without stagnation. Quoting from Rust's documentation:

[..] you should never have to fear upgrading to a new version of stable Rust. Each upgrade should be painless, but should also bring you new features, fewer bugs, and faster compile times.

That is why, for application development, you should generally rely on the latest released version of the compiler to run, build and test your software - the so-called stable channel.
A new version of the compiler is released on the stable channel every six weeks¹ - the latest version at the time of writing is v1.59.0².

There are two other release channels:

• beta, the candidate for the next release;
• nightly, built from the master branch of rust-lang/rust every night, thus the name.

Testing your software using the beta compiler is one of the many ways to support the Rust project - it helps catching bugs before the release date³.

nightly serves a different purpose: it gives early adopters access to unfinished features⁴ before they are released (or even on track to be stabilised!).
I would invite you to think twice if you are planning to run production software on top of the nightly compiler: it's called unstable for a reason.

2.3. What toolchains do we need?

Installing rustup will give you out of the box the latest stable compiler with your host platform as a target. stable is the release channel that we will be using throughout the book to build, test and run our code.

You can update your toolchains with rustup update, while rustup toolchain list will give you an overview of what is installed on your system.

We will not need (or perform) any cross-compiling - our production workloads will be running in containers, hence we do not need to cross-compile from our development machine to the target host used in our production environment.

3. Project Setup

A toolchain installation via rustup bundles together various components.
One of them is the Rust compiler itself, rustc. You can check it out with

rustc --version

You will not be spending a lot of quality time working directly with rustc - your main interface for building and testing Rust applications will be cargo, Rust's build tool.
You can double-check everything is up and running with

cargo --version

Let's use cargo to create the skeleton of the project we will be working on for the whole book:

cargo new zero2prod

You should have a new zero2prod folder, with the following file structure:

zero2prod
├── Cargo.toml
├── .gitignore
├── .git
└── src
   └── main.rs

The project is already a git repository, out of the box.
If you are planning on hosting the project on GitHub, you just need to create a new empty repository and run

cd zero2prod
git add .
git commit -am "Project skeleton"
git remote add origin [email protected]:YourGitHubNickName/zero2prod.git
git push -u origin main

We will be using GitHub as a reference given its popularity and the recently released GitHub Actions feature for CI pipelines, but you are of course free to choose any other git hosting solution (or none at all).

4. IDEs

The project skeleton is ready, it is now time to fire up your favourite editor so that we can start messing around with it.
Different people have different preferences but I would argue that the bare minimum you want to have, especially if you are starting out with a new programming language, is a setup that supports syntax highlighting, code navigation and code completion.

Syntax highlighting gives you immediate feedback on glaring syntax errors, while code navigation and code completion enable "exploratory" programming: jumping in and out of the source of your dependencies, quick access to the available methods on a struct or an enum you imported from a crate without having to continuously switch between your editor and docs.rs.

You have two main options for your IDE setup: rust-analyzer and IntelliJ Rust.

4.1. Rust-analyzer

rust-analyzer⁵ is an implementation of the Language Server Protocol for Rust.
The Language Server Protocol makes it easy to leverage rust-analyzer in many different editors, including but not limited to VS Code, Emacs, Vim/NeoVim and Sublime Text 3.

Editor-specific setup instructions can be found here.

4.2. IntelliJ Rust

IntelliJ Rust provides Rust support to the suite of editors developed by JetBrains.

If you don't have a JetBrains license⁶, IntelliJ IDEA is available for free and supports IntelliJ Rust.
If you have a JetBrains license, CLion is your go-to editor for Rust in JetBrains' IDE suite.

4.3. What should I use?

As of March 2022, IntelliJ Rust should be preferred.
Although rust-analyzer is promising and has shown incredible progress over the last year, it is still quite far from delivering an IDE experience on par with what IntelliJ Rust offers today.

On the other hand, IntelliJ Rust forces you to work with a JetBrains' IDE, which you might or might not be willing to. If you'd like to stick to your editor of choice look for its rust-analyzer integration/plugin.

It is worth mentioning that rust-analyzer is part of a larger library-ification effort taking place within the Rust compiler: there is overlap between rust-analyzer and rustc, with a lot of duplicated effort.
Evolving the compiler's codebase into a set of re-usable modules will allow rust-analyzer to leverage an increasingly larger subset of the compiler codebase, unlocking the on-demand analysis capabilities required to offer a top-notch IDE experience.
An interesting space to keep an eye on in the future⁷.

5. Inner Development Loop

While working on our project, we will be going through the same steps over and over again:

• Make a change;
• Compile the application;
• Run tests;
• Run the application.

This is also known as the inner development loop.
The speed of your inner development loop is as an upper bound on the number of iterations that you can complete in a unit of time.
If it takes 5 minutes to compile and run the application, you can complete at most 12 iterations in an hour. Cut it down to 2 minutes and you can now fit in 30 iterations in the same hour!

Rust does not help us here - compilation speed can become a pain point on big projects. Let's see what we can do to mitigate the issue before moving forward.

5.1. Faster Linking

When looking at the inner development loop, we are primarily looking at the performance of incremental compilation - how long it takes cargo to rebuild our binary after having made a small change to the source code.
A sizeable chunk of time is spent in the linking phase - assembling the actual binary given the outputs of the earlier compilation stages.

The default linker does a good job, but there are faster alternatives depending on the operating system you are using:

• lld on Windows and Linux, a linker developed by the LLVM project;
• zld on MacOS.

To speed up the linking phase you have to install the alternative linker on your machine and add this configuration file to the project:

# .cargo/config.toml

# On Windows 
# ```
# cargo install -f cargo-binutils
# rustup component add llvm-tools-preview
# ```
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]

[target.x86_64-pc-windows-gnu]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]

# On Linux:
# - Ubuntu, `sudo apt-get install lld clang`
# - Arch, `sudo pacman -S lld clang`
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "linker=clang", "-C", "link-arg=-fuse-ld=lld"]

# On MacOS, `brew install michaeleisel/zld/zld`
[target.x86_64-apple-darwin]
rustflags = ["-C", "link-arg=-fuse-ld=/usr/local/bin/zld"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "link-arg=-fuse-ld=/usr/local/bin/zld"]

There is ongoing work on the Rust compiler to use lld as the default linker where possible - soon enough this custom configuration will not be necessary to achieve higher compilation performance⁸

5.2. cargo-watch

We can also mitigate the impact on our productivity by reducing the perceived compilation time - i.e. the time you spend looking at your terminal waiting for cargo check or cargo run to complete.
Tooling can help here - let's install cargo-watch:

cargo install cargo-watch

cargo-watch monitors your source code to trigger commands every time a file changes.
For example:

cargo watch -x check

will run cargo check after every code change.

This reduces the perceived compilation time:

• you are still in your IDE, re-reading the code change you just made;
• cargo-watch, in the meantime, has already kick-started the compilation process;
• once you switch to your terminal, the compiler is already halfway through!

cargo-watch supports command chaining as well:

cargo watch -x check -x test -x run

It will start by running cargo check.
If it succeeds, it launches cargo test.
If tests pass, it launches the application with cargo run.

Our inner development loop, right there!

6. Continuous Integration

Toolchain, installed.
Project skeleton, done.
IDE, ready.

One last thing to look at before we get into the details of what we will be building: our Continuous Integration (CI) pipeline.

In trunk-based development we should be able to deploy our main branch at any point in time.
Every member of the team can branch off from main, develop a small feature or fix a bug, merge back into main and release to our users.

Continuous Integration empowers each member of the team to integrate their changes into the main branch multiple times a day.

This has powerful ripple effects.
Some are tangible and easy to spot: it reduces the chances of having to sort out messy merge conflicts due to long-lived branches. Nobody likes merge conflicts.
Some are subtler: Continuous Integration tightens the feedback loop. You are less likely to go off on your own and develop for days or weeks just to find out that the approach you have chosen is not endorsed by the rest of the team or it would not integrate well with the rest of the project.
It forces you to engage with your teammates earlier than when it feels comfortable, course-correcting if necessary when it is still easy to do so (and nobody is likely to get offended).

How do we make it possible?

With a collection of automated checks running on every commit - our CI pipeline.
If one of the checks fails you cannot merge to main - as simple as that.

CI pipelines often go beyond ensuring code health: they are a good place to perform a series of additional important checks - e.g. scanning our dependency tree for known vulnerabilities, linting, formatting, etc.

We will run through the different checks that you might want to run as part of the CI pipeline of your Rust projects, introducing the associated tools as we go along.
We will then provide a set of ready-made CI pipelines for some of the major CI providers.

6.1. CI steps

6.1.1. Tests

If your CI pipeline had a single step, it should be testing.
Tests are a first-class concept in the Rust ecosystem and you can leverage cargo to run your unit and integration tests:

cargo test

cargo test also takes care of building the project before running tests, hence you do not need to run cargo build beforehand (even though most pipelines will invoke cargo build before running tests to cache dependencies).

6.1.2. Code coverage

Many articles have been written on the pros and cons of measuring code coverage.
While using code coverage as a quality check has several drawbacks I do argue that it is a quick way to collect information and spot if some portions of the codebase have been overlooked over time and are indeed poorly tested.

The easiest way to measure code coverage of a Rust project is via cargo tarpaulin, a cargo subcommand developed by xd009642. You can install tarpaulin with

# At the time of writing tarpaulin only supports x86_64 CPU architectures running Linux.
cargo install cargo-tarpaulin

while

cargo tarpaulin --ignore-tests

will compute code coverage for your application code, ignoring your test functions.

tarpaulin can be used to upload code coverage metrics to popular services like Codecov or Coveralls - instructions can be found in tarpaulin's README.

6.1.3. Linting

Writing idiomatic code in any programming language requires time and practice.
It is easy at the beginning of your learning journey to end up with fairly convoluted solutions to problems that could otherwise be tackled with a much simpler approach.

Static analysis can help: in the same way a compiler steps through your code to ensure it conforms to the language rules and constraints, a linter will try to spot unidiomatic code, overly-complex constructs and common mistakes/inefficiencies.

The Rust team maintains clippy, the official Rust linter⁹.
clippy is included in the set of components installed by rustup if you are using the default profile. Often CI environments use rustup's minimal profile, which does not include clippy.
You can easily install it with

rustup component add clippy

If it is already installed the command is a no-op.

You can run clippy on your project with

cargo clippy

In our CI pipeline we would like to fail the linter check if clippy emits any warnings.
We can achieve it with

cargo clippy -- -D warnings

Static analysis is not infallible: from time to time clippy might suggest changes that you do not believe to be either correct or desirable.
You can mute a warning using the #[allow(clippy::lint_name)] attribute on the affected code block or disable the noisy lint altogether for the whole project with a configuration line in clippy.toml or a project-level #![allow(clippy::lint_name)] directive.
Details on the available lints and how to tune them for your specific purposes can be found in clippy's README.

6.1.4. Formatting

Most organizations have more than one line of defence for the main branch: one is provided by the CI pipeline checks, the other is often a pull request review.

A lot can be said on what distinguishes a value-adding PR review process from a soul-sucking one - no need to re-open the whole debate here.
I know for sure what should not be the focus of a good PR review: formatting nitpicks - e.g. Can you add a newline here?, I think we have a trailing whitespace there!, etc.

Let machines deal with formatting while reviewers focus on architecture, testing thoroughness, reliability, observability. Automated formatting removes a distraction from the complex equation of the PR review process. You might dislike this or that formatting choice, but the complete erasure of formatting bikeshedding is worth the minor discomfort.

rustfmt is the official Rust formatter.
Just like clippy, rustfmt is included in the set of default components installed by rustup. If missing, you can easily install it with

rustup component add rustfmt

You can format your whole project with

cargo fmt

In our CI pipeline we will add a formatting step

cargo fmt -- --check

It will fail when a commit contains unformatted code, printing the difference to the console¹⁰.

You can tune rustfmt for a project with a configuration file, rustfmt.toml. Details can be found in rustfmt's README.

6.1.5. Security vulnerabilities

cargo makes it very easy to leverage existing crates in the ecosystem to solve the problem at hand.
On the flip side, each of those crates might hide an exploitable vulnerability that could compromise the security posture of your software.

The Rust Secure Code working group maintains an Advisory Database - an up-to-date collection of reported vulnerabilities for crates published on crates.io.

They also provide cargo-audit¹¹, a convenient cargo sub-command to check if vulnerabilities have been reported for any of the crates in the dependency tree of your project.
You can install it with

cargo install cargo-audit

Once installed, run

cargo audit

to scan your dependency tree.

We will be running cargo-audit as part of our CI pipeline, on every commit.
We will also run it on a daily schedule to stay on top of new vulnerabilities for dependencies of projects that we might not be actively working on at the moment but are still running in our production environment!

6.2. Ready-to-go CI pipelines

Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime.

Hopefully I have taught you enough to go out there and stitch together a solid CI pipeline for your Rust projects.
We should also be honest and admit that it can take multiple hours of fidgeting around to learn how to use the specific flavour of configuration language used by a CI provider and the debugging experience can often be quite painful, with long feedback cycles.
I have thus decided to collect a set of ready-made configuration files for the most popular CI providers - the exact steps we just described, ready to be embedded in your project repository:

• GitHub Actions;
• CircleCI;
• GitLab CI;
• Travis.

It is often much easier to tweak an existing setup to suit your specific needs than to write a new one from scratch.

This article is a sample from Zero To Production In Rust, a hands-on introduction to backend development in Rust.
You can get a copy of the book at zero2prod.com.

Footnotes

Book - Table Of Contents