Publish Python packages with PyPI Trusted Publishing

[Jaydeep869]

Addresses #228

Summary

• migrate package publishing from API tokens to PyPI Trusted Publishing
• publish release candidates to TestPyPI and stable releases to PyPI
• publish all six public packages, including darnit-csl
• enable PyPI attestations and update package metadata, container builds, smoke tests, and release documentation
• build the release container from this run's Darnit wheel artifacts, while resolving external dependencies from PyPI

Testing

• built all six public Python packages
• ruff check .
• pytest tests/packaging/test_wheel_install_config.py tests/darnit_reproducibility/test_container_pinning.py -q (22 passed)
• workflow YAML parsing
• git diff --check

The broader test run previously reached 3074 passed, with existing parity harness timeouts and the product source guard failure unrelated to these packaging-only changes.

AI assistance

• No AI assistance was used
• AI assistance was used

@mlieberman85 and @Marc-cn, could you please take another look, especially at the Trusted Publisher setup and public package list?

[mlieberman85]

Thanks, this is a solid migration, and it also fixes the smoke matrix (darnit -> darnit-core) and aligns release.yml with the smoke workflow's existing TestPyPI expectation for rc tags.

Two things before merge:

1. rc container dependency confusion. With --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/, pip picks the highest version across both indexes, and anyone can register any name on TestPyPI. A squatted mcp, pydantic, or darnit-* release on TestPyPI with a higher version would be installed into the :rc image we sign and push. Could the image build install the darnit wheels from this run's build artifacts (for example pip install --no-index --find-links ./dist darnit-mcp==$VERSION for the darnit packages, then dependencies from pypi.org only), or at least install the darnit packages from TestPyPI with --no-deps and resolve dependencies from PyPI alone?
2. rc release notes. The GitHub Release body still says pip install darnit-mcp==${VERSION} (PyPI). For rc tags that version now only exists on TestPyPI, so the install instructions should switch to the TestPyPI form, as docs/install/pypi.md does.

Smaller:

3. attestations: write isn't needed for PyPI's PEP 740 attestations, which use the OIDC token (id-token: write). Could you drop it for least privilege?
4. Please add the "AI assistance" section from the PR template (and an Assisted-by: trailer if AI was used).

@Marc-cn and I (and anyone else who's interested) will handle setting up the Trusted Publishers and the release environment on the PyPI/TestPyPI side.

[Marc-cn]

I can take the PyPI/TestPyPI side, Trusted Publishers plus the release environment on both indexes. From the workflow that's five projects (darnit-core, darnit-baseline, darnit-gittuf, darnit-reproducibility, darnit-mcp), so ten publisher configs in total, each pointing at release.yml with environment release.

Two questions before I set it up:

@mlieberman85, do you already own these five names on PyPI and TestPyPI, or do some need registering? Trusted Publishing needs either an existing project or a pending publisher, and the names must match exactly. Also: do you want me added as owner on the PyPI orgor would you rather configure and I verify?

@Jaydeep869darnit-csl isn't in the publish set. Deliberate (it's new), or an oversight? If it should ship, it needs a sixth job and I'll add two more publisher configs.

[Jaydeep869]

@Marc-cn Thanks for the review. On the darnit-csl i followed the existing public package list, so leaving out darnit-csl was not intentional. Since it is an installable plugin and its README already documents pip install darnit-csl, I think it should be included as the sixth published package. I will update the workflow and docs for it. For the container image, I will keep it out unless we decide the MCP image should include optional plugins too.

[Jaydeep869]

Thanks Mike and Marco. I pushed updates for the review points: the container now installs Darnit packages from this run's built wheel artifacts and uses PyPI only for external dependencies, rc release notes now show the TestPyPI install command, publish jobs no longer request attestations: write, and the PR now includes the AI assistance section plus an Assisted-by trailer. I also added darnit-csl as the sixth public package and included it in publish docs and smoke tests.

[Marc-cn]

Checked the updates, container installs the darnit wheels from this run's dist with PyPI for externals only, rc notes point at TestPyPI, "attestations: write" dropped, "assisted-by" okay and darnit-csl is in as the job. All four points look addressed to me. Name status, so we know what the PyPI side needs:
darnit-core pypi: exists testpypi: free
darnit-baseline pypi: exists testpypi: free
darnit-gittuf pypi: exists testpypi: free
darnit-reproducibility pypi: exists testpypi: free
darnit-mcp pypi: exists testpypi: free
darnit-csl pypi: free testpypi: free

So five Trusted Publisher configs on existing PyPI projects, plus seven pending publishers (six on TestPyPI, one for darnit-csl on PyPI). The existing five are Kusari's 0.1.0 from before the org move, "darnit-core"'s metadata still lists "kusari-oss/darnit" as homepage and repository. The publisher config needs "darnitdevorg/darnit2and the project URLs are worth refreshing on the next release.

@mlieberman85, the five existing PyPI projects need an owner to add the publisher. Can you add me as owner or would you rather configure those five while I do the seven pending ones? Either works. Also yours to clear the requested-changes review.

[mlieberman85]

Thanks for reworking this; the earlier review points are addressed. We want to land Trusted Publishing, and #558 already moves the publish jobs into the protected release environment. Before merge:

1. container-edge.yml builds the same Dockerfile without dist/, so COPY dist /tmp/dist would break every push to main. It needs a source-install path, or dist built in that workflow.
2. Drop --pre from the Dockerfile; it pulls prerelease dependencies into stable images.
3. darnit-csl is still 0.1.0, which fails the release preflight. Either bump it or leave it out of the public list for now; we'll decide which.
4. Update the local-build instructions in packaging/container/README.md, and delete or align publish.yml.
5. Rebase onto #558 once it merges.

Drafted with Claude Code; reviewed and posted by me.