Skip to content

Modernize Python repos: pyproject.toml + uv + semantic-release #506

Open
4 of 13 issues completed
Open
Modernize Python repos: pyproject.toml + uv + semantic-release#506
4 of 13 issues completed
Assignees
salman2013
Labels
epicLarge unit of work, consisting of multiple tasks

Description

@feanil
feanil
opened on Apr 27, 2026

Context

The openedx org is standardizing on modern Python tooling across all maintained Python
packages. This issue tracks the work of applying that standard to repos that still use
legacy tooling.

Why:

  • pip-tools is being retired in favour of uv, which is faster and produces a single
    reproducible uv.lock lockfile (no more per-environment .txt files)
  • pyproject.toml (PEP 621/735) consolidates package metadata and dependency
    declarations in one file, eliminating setup.py, setup.cfg, and requirements/*.txt
  • python-semantic-release automates version bumps and PyPI publishing from conventional
    commit messages, removing manual release steps

Reference implementation: openedx/sample-plugin- The files to
model after are backend/pyproject.toml, backend/tox.ini, backend/Makefile, and
.github/workflows/backend-ci.yml.

Supporting infrastructure:

Which repos qualify?

A repo should be modernized if it meets all of the following:

  • Is a Python package published (or intended to be published) to PyPI
  • Uses pip-compile / requirements/*.txt for dependency management
  • Uses setup.py or setup.cfg for package metadata (or a pyproject.toml that
    still uses dynamic = [..., "dependencies"] pointing to a requirements file)

Work per repo

1. Consolidate package metadata into pyproject.toml

Move all package metadata into a single pyproject.toml using PEP 621. This means
replacing setup.py/setup.cfg, declaring a static dependencies list under
[project], and using setuptools-scm for version management from git tags.

Reference: backend/pyproject.toml ([project], [build-system], and [tool.setuptools_scm] sections)

  • Migrate metadata from setup.cfg/setup.py into [project]
  • Declare dependencies as a static list (not dynamic from a requirements file)
  • Configure setuptools-scm for version discovery
  • Delete setup.py and setup.cfg

Note: The sample-plugin houses the python library in a subdirectory and so has
to set the root setting in tool.setuptools_scm to ... This should not be
necessary for any other repo and should be omitted.

2. Switch dependency management from pip-compile to uv

Replace all requirements/*.in + requirements/*.txt files with PEP 735
[dependency-groups] in pyproject.toml and a single uv.lock lockfile. The
standard groups are test-base, test, quality, doc, ci, and dev — with
additional version-matrix groups as needed. Update tox.ini to use tox-uv's
uv-venv-lock-runner, update the Makefile to use uv lock/uv sync, and update
CI to install uv and run uv run tox.

Reference: backend/pyproject.toml ([dependency-groups] and [tool.uv] sections), backend/tox.ini, backend/Makefile, .github/workflows/backend-ci.yml

  • Add [dependency-groups] to pyproject.toml covering test, quality, doc, ci, and dev
  • Add [tool.edx_lint].uv_constraints for any repo-specific version overrides,
    then run edx_lint write_uv_constraints to populate [tool.uv].constraint-dependencies
  • Generate uv.lock and commit it
  • Delete the requirements/ directory
  • Update tox.ini to use tox-uv>=1 and uv-venv-lock-runner with dependency_groups
  • Update Makefile targets (upgrade, compile-requirements, requirements)
  • Update CI to install uv via astral-sh/setup-uv, install deps via uv sync --group ci,
    and run tests via uv run tox

3. Add semantic-release

Configure python-semantic-release so that pushing a conventional commit to main
automatically cuts a version, tags it, and publishes to PyPI. Add commitlint to enforce
conventional commit format on PRs.

Reference: backend/pyproject.toml ([tool.semantic_release] section), .github/workflows/release.yml

Note: The sample plugin overrides minor_tags because it wants to publish new
minor versions on docs changes. That is not needed in our other libraries. They
should use the default value unless you know for sure you need to use something
different.

  • Add [tool.semantic_release] config to pyproject.toml with a build_command
    that sets SETUPTOOLS_SCM_PRETEND_VERSION at build time
  • Add a release.yml workflow that runs CI then publishes to PyPI via OIDC
  • Add a commitlint.yml workflow to enforce conventional commits on PRs
  • Confirm PyPI trusted publisher (OIDC) is configured for the repo

Notes

  • [tool.uv].constraint-dependencies is machine-managed. Never edit it directly.
    Repo-specific version overrides belong in [tool.edx_lint].uv_constraints.
  • uv sync does not put tools on PATH. Use uv run tox in CI, not bare tox.
  • A new framework version may require bumping requires-python. If the new version
    drops an older Python, bump accordingly and remove that Python from the tox envlist.
  • OEP-67 ADR documenting this migration is in progress (link TBD once merged).

Activity

  1. feanil
    added this to Aximprovements Teamon Apr 27, 2026
  2. feanil
    moved this to 🆕 New in Aximprovements Teamon Apr 27, 2026

  3. xitij2000 commented on Apr 27, 2026

    @xitij2000
    xitij2000
    on Apr 27, 2026

    @feanil Sorry if this was already discussed and I'm jumping in too late, but I'd like to recommend hatch instead of uv as the build backend. It supports build environments which will allow replacing tox as well. It's a pypa project so it follows PEP standards closely and can still use uv for installing packages.
    Sorry if this has already been discussed and eliminated.

  4. feanil commented on Apr 29, 2026

    @feanil
    feanil
    on Apr 29, 2026
    ContributorAuthor

    @xitij2000 My understanding is that the build backend is still setuptools and we're just using uv for dependency management. Hatch seems promising as a setuptools replacement but I haven't evaluated it too deeply yet because I didn't want to add complexity to the switch from setup.py to pyproject.toml by also changing the build backend at the same time. Let me know if I misunderstood something about the ecosystem here.

  5. farhan
    added
    epicLarge unit of work, consisting of multiple tasks
    on Apr 29, 2026

  6. farhan commented on Apr 29, 2026

    @farhan
    farhan
    on Apr 29, 2026
  7. farhan
    moved this from 🆕 New to 📋 Backlog in Aximprovements Teamon Apr 29, 2026
  8. farhan
    added a sub-issue on Apr 29, 2026

  9. bradenmacdonald commented on Apr 30, 2026

    @bradenmacdonald
    bradenmacdonald
    on Apr 30, 2026

    Reference implementation: openedx/sample-plugin- The files to model after are backend/pyproject.toml, backend/tox.ini, backend/Makefile, and .github/workflows/backend-ci.yml.

    I just want to point out that sample-plugin/backend is not using a src/ directory layout, and I would highly recommend that we try to adopt that where feasible, for these reasons and this reason (editable installs of repos like opaque-keys will not be usable by important tools like mypy because they're not using a src/ directory, but repos like openedx-core that do use a src directory work fine).

  10. feanil commented on Apr 30, 2026

    @feanil
    feanil
    on Apr 30, 2026
    ContributorAuthor

    @bradenmacdonald thanks, that seems like another general improvement we should make across all of our libraries as we modernize.

  11. feanil commented on May 1, 2026

    @feanil
    feanil
    on May 1, 2026
    ContributorAuthor

    PR for that refactor in the sample-plugin repo: openedx/sample-plugin#32 if anyone has a sec to review.

  12. salman2013 commented on May 6, 2026

    @salman2013
    salman2013
    on May 6, 2026

    Uses setup.py/setup.cfg — pip-compile / requirements/*.txt — PUBLISHED on PyPI

    Uses setup.py/setup.cfg + requirements/*.txt — NOT on PyPI

    Uses setup.py/setup.cfg — NO requirements/*.txt — NOT on PyPI

  13. FuaadZam
    assigned
    salman2013
    on May 6, 2026
  14. FuaadZam
    moved this from 📋 Backlog to 🏗 In progress in Aximprovements Teamon May 13, 2026
  15. salman2013
    added a sub-issue on May 19, 2026

  16. 136 remaining items

  17. feanil
    added 4 commits that reference this issue on Sep 30, 2026

    feat: consolidate package metadata into pyproject.toml

    6988232

    feat: switch dependency management from pip-compile to uv

    c2479a5

    feat: add semantic-release and commitlint workflows

    7cd52e2

    feat: modernize Python tooling (pyproject.toml + uv + semantic-releas…

    91a2d4f
  18. irfanuddinahmad
    added a commit that references this issue on Oct 1, 2026

    feat: modernize to uv + pyproject.toml + python-semantic-release

    86e6eb0
  19. feanil
    added 4 commits that reference this issue on Oct 1, 2026

    refactor: move package to src/ layout

    69e64ec

    feat: modernize to uv + pyproject.toml + python-semantic-release

    1ec3dc0

    feat: modernize to uv + pyproject.toml + semantic-release (#381)

    abed7fb

    feat: modernize to uv + pyproject.toml + semantic-release (#190)

    16ef652
  20. farhan
    added 3 commits that reference this issue on Oct 8, 2026

    build: modernize edx-submissions repo to uv + pyproject.toml

    b741e29

    build: modernize edx-submissions repo to uv + pyproject.toml

    2a080b8

    build: modernize edx-submissions repo to uv + pyproject.toml

    d62bc7d
  21. irfanuddinahmad
    added a commit that references this issue on Oct 12, 2026

    feat: modernize Python tooling (pyproject.toml + uv + semantic-releas…

    81253aa
  22. salman2013
    added a commit that references this issue on Oct 14, 2026

    fix: address review-modernize-python-tooling findings

    6536598
  23. farhan
    added a commit that references this issue on Oct 20, 2026

    build: modernize edx-submissions repo to uv + pyproject.toml

    0a4f68f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

epicLarge unit of work, consisting of multiple tasks

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions