8. Use tox and pre-commit#

Date: 2023-01-18

Status#

Accepted

Context#

We require an easy way to locally run the same checks as CI. This provides a rapid inner-loop developer experience.

Decision#

Use tox and pre-commit.

tox is an automation tool that we use to run all checks in parallel, see https://tox.wiki/en/latest/.

pre-commit provides a hook into git commit which runs some of the checks against the changes you are about to commit.

Decision detail#

There are a number of things that CI needs to run:

  • pytest

  • black

  • mypy

  • flake8

  • isort

  • build documentation

The initial approach this module took was to integrate everything under pytest that had a plugin, and isort under flake8:

digraph initial { bgcolor=transparent graph [fontname="Lato" fontsize=10 style=filled fillcolor="#8BC4E9"] node [fontname="Lato" fontsize=10 shape=box style=filled fillcolor="#8BC4E9"] subgraph cluster_0 { label = "pytest" "pytest-black" "pytest-mypy" subgraph cluster_1 { label = "pytest-flake8" "flake8-isort" } } }

This had the advantage that a pytest tests run in CI would catch and report all test failures, but made each run take longer than it needed to. Also, flake8 states that it does not have a public, stable, Python API so did not recommend the approach taken by pytest-flake8.

To address this, the tree was rearranged:

digraph rearranged { bgcolor=transparent graph [fontname="Lato" fontsize=10 style=filled fillcolor="#8BC4E9"] node [fontname="Lato" fontsize=10 shape=box style=filled fillcolor="#8BC4E9"] pytest black mypy subgraph cluster_1 { label = "flake8" "flake8-isort" } }

If using VSCode, this will still run black, flake8 and mypy on file save, but for those using other editors and for CI another solution was needed. Enter pre-commit. This allows hooks to be run at git commit time on just the files that have changed, as well as on all tracked files by CI. All that is needed is a one time install of the git commit hook:

$ pre-commit install

Finally tox was added to run all of the CI checks including the documentation build. mypy was moved out of the pre-commit and into tox because it was quite long running and therefore intrusive. tox can be invoked to run all the checks in parallel with:

$ tox -p

The graph now looks like:

digraph rearranged { bgcolor=transparent graph [fontname="Lato" fontsize=10 style=filled fillcolor="#8BC4E9"] node [fontname="Lato" fontsize=10 shape=box style=filled fillcolor="#8BC4E9"] subgraph cluster_0 { label = "tox -p" pytest mypy "sphinx-build" subgraph cluster_1 { label = "pre-commit" black subgraph cluster_2 { label = "flake8" "flake8-isort" } } } }

Now the workflow looks like this:

  • Save file, VSCode runs black, flake8 and mypy on it

  • Run ‘tox -p’ and fix issues until it succeeds

  • Commit files and pre-commit runs black and flake8 on them (if the developer had not run tox then this catches some of the most common issues)

  • Push to remote and CI runs black, flake8, mypy once on all files (using tox), then pytest multiple times in a test matrix

Consequences#

Running tox -p before pushing to GitHub verifies that the CI will most likely succeed.

Committing changes to git will run all of the non-time critical checks and help avoid some of the most common mistakes.