I recently discovered the ✨ awesomeness that is pre-commit. I steered away from it for so long because it seemed like a big daunting thing to set up, but really it's easy. It will automatically run checks for you. In some cases, it will even automatically fix them for you. Out of the box, it will do things like automatically trim extra whitespace, fix file endings, and ensure file sizes are not too large for git.
It comes with a
sample-config that is pretty general purpose and use for just about any project in git.
pip instal pre-commit pre-commit sample-config > .pre-commit-config.yaml pre-commit install git add . git commit -m "added pre-commit"
Once someone has created the
.pre-commit-config.yaml everyone on the team will want to be running it for consistency's sake. (make sure everyone agrees with the config you have chosen first). Simply install the existing config.
pip install pre-commit git clone <repo> pre-commit install git add . git commit -m "added pre-commit"
The sample configuration does some really basic, file ending, trailing-whitespace fixing. And checks for files too large for git. This one saved me when I tried to commit linux
rpm once 🤦♀️.
# See https://pre-commit.com for more information # See https://pre-commit.com/hooks.html for more hooks repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.4.0 hooks: - id: trailing-whitespace - id: end-of-file-fixer - id: check-yaml - id: check-added-large-files
Adding some extras from pre-commit themselves
Here I have added a couple of extra ones form pre-commit
- id: check-case-conflict # Check for files that would conflict in case-insensitive filesystems - id: check-merge-conflict # Check for files that contain merge conflict strings. - id: debug-statements # Check for debugger imports and py37+ `breakpoint()` calls in python source. - id: requirements-txt-fixer # Sorts entries in requirements.txt - id: forbid-new-submodules # Check for git submodules - id: flake8 # runs python flake8
The submodules one is big. I have seen several folks trying to learn git for the first time mistakenly start nesting all of their projects underneath each other and eventually losing a lot of work. Trying to learn the command line and git all at once can be really confusing.
So you have a big codebase and you are trying to get pre-commit ready, but you just need your changes in.
git commit -m "commiting wihout pre-commit" --no-verify
manually run pre-commit
If you have an existing repo and want to run pre-commit on everything, since it was pre-existing, you can do that manually.
pre-commit run --all-files
So pre-commit changed some files
pre-commit only runs against staged files, but makes changes to the local files you need to add them.
Here is a git status after committing with some trailing whitespace issues.
❯ git status On branch main Changes to be committed: (use "git restore --staged <file>..." to unstage) modified: README.md Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: README.md
pre-commit will keep yelling at you about
trailing whitespace until you add the files.
git add README.md
Community Developed plugins
give me more
These almost make it too easy. Sharing your process to get up and running without
pre-commit can involve a lot of instructions. Installing several different tools, then running them manually, probably forgetting to do so sometimes. These will automatically install and only run scoped to the files that have changed, not on the whole repo.
- repo: https://github.com/asottile/seed-isort-config rev: v2.1.1 hooks: - id: seed-isort-config - repo: https://github.com/pre-commit/mirrors-isort rev: v4.3.21 hooks: - id: isort
In order to get isort to play nicely with black, I found great success with the following config placed in the root of the repo at
.isort.cfg. Without these settings, I found that you commits will consistently fail checks because
black are fighting each other.
[settings] multi_line_output=3 include_trailing_comma=True force_grid_wrap=0 use_parentheses=True line_length=88
Just as with
isort flake8 tends to complain about a few things that black does. To get them to play nicely together place this file in the root of the repo at
# taken from black # added E231 as is conflicts with black formatting [flake8] ignore = E203, E266, E501, W503, E231, F541 max-line-length = 88 max-complexity = 18 select = B,C,E,F,W,T4,B9
Black is an amazing CLI tool the python community has been blessed with. It was developed by python core dev Lukasz Langa after deep research of real python projects. It will autoformat your project and will check that the AST before and after remains the same ensuring that the code will run exactly the same. It only makes it more readable. I keep black installed and set to run on save. Many times I will bang out some sloppy code with long lines or poor indentation hit save and let black take care of the easy work.
- repo: https://github.com/asottile/blacken-docs rev: v1.7.0 hooks: - id: blacken-docs - additional_dependencies: [black] - repo: https://github.com/psf/black rev: 19.3b0 hooks: - id: black
I have recently fallen in love with mypy. It has saved me from shipping some bugs that would not have been caught with tests, even with 100% coverage. I don't have 100% coverage across every possible type entered.
- repo: https://github.com/pre-commit/mirrors-mypy rev: v0.720 hooks: - id: mypy exclude: tests/
Your own plugin
Sometimes you have a CLI tool that you want to run, but there is no plugin. No worries, you can install manually set the repo to local, and add an entry for your CLI command to run.
- repo: local hooks: - id: interrogate name: "Interrogate docstring coverage check" types: [file, python] entry: interrogate -f 100 -vv
I have been writing short snippets about my mentality breaking into the tech/data industry in my newsletter, 👇 check it out and lets get the conversation started.