8. Use tox and pre-commit#

Date: 2023-01-18

Status#

Accepted

Context#

We require an easy way to locally run the same checks as CI. This provides a rapid inner-loop developer experience.

Decision#

Use tox and pre-commit.

tox is an automation tool that we use to run all checks in parallel, see https://tox.wiki/en/latest/.

pre-commit provides a hook into git commit which runs some of the checks against the changes you are about to commit.

Decision detail#

There are a number of things that CI needs to run:

pytest
black
mypy
flake8
isort
build documentation

The initial approach this module took was to integrate everything under pytest that had a plugin, and isort under flake8:

This had the advantage that a pytest tests run in CI would catch and report all test failures, but made each run take longer than it needed to. Also, flake8 states that it does not have a public, stable, Python API so did not recommend the approach taken by pytest-flake8.

To address this, the tree was rearranged:

If using VSCode, this will still run black, flake8 and mypy on file save, but for those using other editors and for CI another solution was needed. Enter pre-commit. This allows hooks to be run at git commit time on just the files that have changed, as well as on all tracked files by CI. All that is needed is a one time install of the git commit hook:

$ pre-commit install

Finally tox was added to run all of the CI checks including the documentation build. mypy was moved out of the pre-commit and into tox because it was quite long running and therefore intrusive. tox can be invoked to run all the checks in parallel with:

$ tox -p

The graph now looks like:

Now the workflow looks like this:

Save file, VSCode runs black, flake8 and mypy on it
Run ‘tox -p’ and fix issues until it succeeds
Commit files and pre-commit runs black and flake8 on them (if the developer had not run tox then this catches some of the most common issues)
Push to remote and CI runs black, flake8, mypy once on all files (using tox), then pytest multiple times in a test matrix

Consequences#

Running tox -p before pushing to GitHub verifies that the CI will most likely succeed.

Committing changes to git will run all of the non-time critical checks and help avoid some of the most common mistakes.