Skip to content

[Housekeeping] Update Python & third party packages. Add ruff. #395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions .github/workflows/django.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,8 @@ jobs:
- uses: actions/checkout@v3
- name: Build docker
run: docker compose build
- name: Run isort
run: docker compose run django isort styleguide_example/ --check
- name: Run black
run: docker compose run django black styleguide_example/ --check
- name: Run flake8
run: docker compose run django flake8
- name: Run ruff
run: docker compose run django ruff check styleguide_example/
- name: Run mypy
run: docker compose run django mypy --config mypy.ini styleguide_example/
- name: Run tests
Expand Down Expand Up @@ -50,12 +46,8 @@ jobs:
key: python-${{ hashFiles('requirements/local.txt') }}-${{ hashFiles('requirements/base.txt') }}
- name: Install dependencies
run: pip install -r requirements/local.txt
- name: Run isort
uses: isort/isort-action@master
- name: Run black
uses: psf/black@stable
- name: Run flake8
run: flake8
- name: Run ruff
uses: chartboost/ruff-action@v1
- name: Type check
run: mypy --config mypy.ini styleguide_example/
- name: Run tests
Expand Down
30 changes: 11 additions & 19 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,12 @@
default_language_version:
python: python3.12.0
repos:
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
hooks:
- id: flake8
name: flake8

- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
name: isort
args: [--check]

- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: black
name: black
args: [--check]
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.2.2
hooks:
# Run the linter.
- id: ruff
args: [ --fix ]
# Run the formatter.
- id: ruff-format
49 changes: 7 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,61 +388,26 @@ _Coming soon_

In all our Django projects we use:

- [flake8](https://flake8.pycqa.org/en/latest/) - a linter that ensures we follow the PEP8 conventions.
- [black](https://github.com/psf/black) - a code formatter that ensures we have the same code style everywhere.
- [isort](https://github.com/PyCQA/isort) - a code formatter that ensures we have the same import style everywhere.
- [ruff](https://docs.astral.sh/ruff/) - an extremely fast Python linter and code formatter, written in Rust.
- [pre-commit](https://pre-commit.com/) - a tool that triggers the linters before each commit.

To make sure all of the above tools work in symbiosis, you'd need to add some configuration:

1. Add `.pre-commit-config.yaml` file to the root of your project. There you can add the instructions for `pre-commit`
2. Add `pyproject.toml` file to the root of your project. There you can add the `black` config. **NOTE:** `black` [does not respect any other config files.](https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html)
3. Add the following to `setup.cfg` for the `isort` config:

```
[isort]
profile = black
```

This will tell `isort` to follow the `black` guidelines.

```
[isort]
filter_files = true
skip_glob = */migrations/*
```

This will tell `pre-commit` to respect the `isort` config.

4. You can add a custom `flake8` configuration to `setup.cfg` as well. We usually have the following config in all our projects:

```
[flake8]
max-line-length = 120
extend-ignore = E203
exclude =
.git,
__pycache__,
*/migrations/*
```

5. Make sure the linters are run against each PR on your CI. This is the config you need if you use GH actions:
2. Add `pyproject.toml` file to the root of your project. There you can add the `ruff` config.
3. Make sure the linters are run against each PR on your CI. This is the config you need if you use GH actions:

```
build:
runs-on: ubuntu-latest
steps:
- name: Run isort
uses: isort/isort-action@master
- name: Run black
uses: psf/black@stable
- name: Run flake8
run: flake8
- name: Run ruff
uses: chartboost/ruff-action@v1
```

6. Last but not least, we highly recommend you to setup you editor to run `black` and `isort` every time you save a new Python file.
4. Last but not least, we highly recommend you to setup you editor to run `ruff` every time you save a new Python file.

In order to test if your local setup is up to date, you can either:

1. Try making a commit, to see if `pre-commit` is going to be triggered.
1. Or run `black --check .` and `isort --check .` in the project root directory.
2. Or run `ruff check .` in the project root directory.
2 changes: 1 addition & 1 deletion config/django/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

import os

from config.env import BASE_DIR, APPS_DIR, env
from config.env import APPS_DIR, BASE_DIR, env

env.read_env(os.path.join(BASE_DIR, ".env"))

Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version: "3.9"
version: "3.12.0"

services:
db:
Expand Down
2 changes: 1 addition & 1 deletion docker/local.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This docker file is used for local development via docker-compose
# Creating image based on official python3 image
FROM python:3.10.8
FROM python:3.12.0

# Fix python printing
ENV PYTHONUNBUFFERED 1
Expand Down
2 changes: 1 addition & 1 deletion docker/production.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This docker file is used for production
# Creating image based on official python3 image
FROM python:3.10.8
FROM python:3.12.0

# Installing all python dependencies
ADD requirements/ requirements/
Expand Down
80 changes: 66 additions & 14 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,66 @@
[tool.black]
line-length = 120
# By default, `black` will ignore skip configuration when paths are explicitly provided.
# In order for `pre-commit` to respect this configuration, `force-exclude` needs to be explicitly set.
force-exclude = 'migrations'

[tool.isort]
# https://black.readthedocs.io/en/stable/guides/using_black_with_other_tools.html#isort
profile = "black"
# By default, `isort` will ignore skip configuration when paths are explicitly provided.
# In order for `pre-commit` to respect this configuration, `filter_files` needs to be set to true.
# https://jugmac00.github.io/blog/isort-and-pre-commit-a-friendship-with-obstacles/
filter_files = true
skip_glob = ["*/migrations/*", "config/*"]
[tool.ruff]
# Exclude a variety of commonly ignored directories.
exclude = [
"migrations",
".bzr",
".direnv",
".eggs",
".git",
".git-rewrite",
".hg",
".ipynb_checkpoints",
".mypy_cache",
".nox",
".pants.d",
".pyenv",
".pytest_cache",
".pytype",
".ruff_cache",
".svn",
".tox",
".venv",
".vscode",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"node_modules",
"site-packages",
"venv",
]

# Same as Black.
line-length = 88
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we use default length we do not need to add to pyproject.toml. If you choose some custom length (120 for example) it will make sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets just continue using 120 and we are good to go.

indent-width = 4

# Assume Python 3.12
target-version = "py312"

[tool.ruff.lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.
select = ["E4", "E7", "E9", "F", "I"]
ignore = ["E722"]

# Allow fix for all enabled rules (when `--fix`) is provided.
fixable = ["ALL"]
unfixable = []

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[tool.ruff.format]
# Like Black, use double quotes for strings.
quote-style = "double"

# Like Black, indent with spaces, rather than tabs.
indent-style = "space"

# Like Black, respect magic trailing commas.
skip-magic-trailing-comma = false

# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

[tool.ruff.lint.isort]
case-sensitive = true
30 changes: 15 additions & 15 deletions requirements/base.txt
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
Django==4.1.9
django-environ==0.9.0
psycopg2==2.9.5
Django==4.2.10
django-environ==0.11.2
psycopg2==2.9.9
djangorestframework==3.14.0

celery==5.2.7
django-celery-results==2.4.0
django-celery-beat==2.4.0
celery==5.3.6
django-celery-results==2.5.1
django-celery-beat==2.5.0

whitenoise==6.5.0

django-filter==23.2
django-extensions==3.2.1
django-cors-headers==4.1.0
django-storages==1.13.1
django-filter==23.5
django-extensions==3.2.3
django-cors-headers==4.3.1
django-storages==1.14.2

drf-jwt==1.19.2

boto3==1.26.0
attrs==22.1.0
boto3==1.34.44
attrs==23.2.0

gunicorn==20.1.0
sentry-sdk==1.14.0
gunicorn==21.2.0
sentry-sdk==1.40.5

requests==2.31.0

Expand All @@ -30,4 +30,4 @@ google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0

pyotp==2.8.0
qrcode==7.4.2
qrcode==7.4.2
24 changes: 11 additions & 13 deletions requirements/local.txt
Original file line number Diff line number Diff line change
@@ -1,23 +1,21 @@
-r base.txt

pytest==7.4.0
pytest-django==4.5.2
pytest==8.0.1
pytest-django==4.8.0

factory-boy==3.2.1
Faker==18.6.0
factory-boy==3.3.0
Faker==23.2.1

ipdb==0.13.13
ipython==8.10.0

django-debug-toolbar==3.8.1
django-debug-toolbar==4.3.0

mypy==1.2.0
mypy==1.8.0

django-stubs==1.14.0
djangorestframework-stubs==1.8.0
boto3-stubs==1.26.81
django-stubs==4.2.7
djangorestframework-stubs==3.14.5
boto3-stubs==1.34.48

flake8==6.0.0
isort==5.12.0
black==23.3.0
pre-commit==3.2.2
ruff==0.2.2
pre-commit==3.6.2
2 changes: 1 addition & 1 deletion runtime.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
python-3.10.8
python-3.12.0
10 changes: 0 additions & 10 deletions setup.cfg

This file was deleted.

12 changes: 9 additions & 3 deletions styleguide_example/common/models.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from django.db import models
from django.db.models.query import F, Q
from django.db.models import F, Q
from django.utils import timezone


Expand Down Expand Up @@ -29,7 +29,13 @@ class RandomModel(BaseModel):
start_date = models.DateField()
end_date = models.DateField()

simple_objects = models.ManyToManyField(SimpleModel, blank=True, related_name="random_objects")
simple_objects = models.ManyToManyField(
SimpleModel, blank=True, related_name="random_objects"
)

class Meta:
constraints = [models.CheckConstraint(name="start_date_before_end_date", check=Q(start_date__lt=F("end_date")))]
constraints = [
models.CheckConstraint(
name="start_date_before_end_date", check=Q(start_date__lt=F("end_date"))
)
]
Loading