Skip to content

ci: mirror a release's engine and models to dl.godwit.email - #1

Merged
baranwang merged 1 commit into
mainfrom
mirror-assets
Oct 19, 2026
Merged

baranwang merged 1 commit into
mainfrom
mirror-assets

Conversation

@baranwang

Copy link
Copy Markdown
Member

Summary

Mirror a release's registry assets (the llama.cpp engine and the two models) to dl.godwit.email/plugins/startlux/, and print the assets block for the registry entry.

  • scripts/mirror.py WORK_DIR [--upload]
    • Engine: scripts/engine.py lays out the pinned build (third_party/llama/VERSION, b11707) for each of the six platforms.
    • Models: the GGUF and decision_config.json of each model are downloaded from startlux-models on Hugging Face at pinned revisions and checked against fixed sha256s.
    • With --upload, each file goes to s3://godwit-downloads/plugins/startlux/{engine/b11707/<platform>,models/v1.1/<variant>}/<name>. Each file is uploaded once, with its sha256 as object metadata. A path that already holds other bytes fails the run, because a published file never changes.
    • Writes assets.json. Memory requirements come from the measurements in docs/parity.md:
      • macOS arm64: 4b-q4 (8 GB) and 2b-q8 (6 GB).
      • Every other platform: 2b-q8 only (6 GB; windows-* 12 GB).
  • .github/workflows/mirror.yml: manual only (tag, upload). The default is a dry run, which downloads, checks and prints but writes nothing to the bucket. It runs this branch's script against the tag's checkout, so v0.1.0 can be mirrored too.

Before running with upload

This repo needs the secrets R2_ACCESS_KEY_ID and R2_SECRET_ACCESS_KEY (the same R2 token the plugins repo uses, or a token limited to godwit-downloads).

Test plan

  • Local dry run: 6 engine layouts (6–19 files, 5–26 MB each); both models match their pinned hashes
  • A draft registry entry for v0.1.0 (the release's six libraries plus this assets.json) passes godwit-registry check
  • Dry run on Actions after merge, then upload: true

🤖 Generated with Claude Code

A manual workflow downloads the pinned llama.cpp build for each platform and
the models at pinned Hugging Face revisions, checks their hashes, uploads each
file once (a path never changes once published) and prints the registry assets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@baranwang
baranwang merged commit 5e962dd into main Oct 19, 2026
7 checks passed
@baranwang
baranwang deleted the mirror-assets branch October 19, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant