From 89989b85c02c84501b52069f3d70392ab525c1c3 Mon Sep 17 00:00:00 2001 From: johannestreutlein Date: Wed, 28 Oct 2026 17:30:27 -0400 Subject: [PATCH 1/4] Add the upstream sync job: merge upstream commits 7 days after first seen A daily GitHub Actions job (.github/workflows/upstream-sync.yml) runs .github/upstream-sync/sync.sh. It records each upstream head with the time it was first seen on the branch upstream-sync-state, and merges into main the newest recorded commit that was first seen at least 7 days ago and is still on upstream's main. The merge driver fork-manifest (.gitattributes) and a check after each merge keep the version fields of the plugin manifests deleted. The job uses plain git and jq, no third-party actions. Pull requests that change these files run test.sh and a dry run. Co-Authored-By: Claude Opus 5.5 --- .gitattributes | 4 + .github/upstream-sync/README.md | 51 ++++++ .github/upstream-sync/fork-manifest.sh | 36 ++++ .github/upstream-sync/sync.sh | 192 ++++++++++++++++++++ .github/upstream-sync/test.sh | 235 +++++++++++++++++++++++++ .github/workflows/upstream-sync.yml | 51 ++++++ 6 files changed, 569 insertions(+) create mode 100644 .gitattributes create mode 100644 .github/upstream-sync/README.md create mode 100755 .github/upstream-sync/fork-manifest.sh create mode 100755 .github/upstream-sync/sync.sh create mode 100644 .github/upstream-sync/test.sh create mode 100644 .github/workflows/upstream-sync.yml diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000..5f29f5167 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +# The plugin manifests are merged with the driver "fork-manifest", which the upstream sync job +# configures: the result is upstream's file without the version fields (.github/upstream-sync/README.md). +plugins/codex/.claude-plugin/plugin.json merge=fork-manifest +.claude-plugin/marketplace.json merge=fork-manifest diff --git a/.github/upstream-sync/README.md b/.github/upstream-sync/README.md new file mode 100644 index 000000000..51cb25f5b --- /dev/null +++ b/.github/upstream-sync/README.md @@ -0,0 +1,51 @@ +# Upstream sync + +This fork takes the changes of its upstream, [Crazytieguy/codex-plugin-cc](https://github.com/Crazytieguy/codex-plugin-cc), automatically, but only once they are at least 7 days old. The delay is a defense against supply-chain attacks: every machine that installs the plugin updates itself from this fork's `main`, so a malicious commit pushed to upstream should have a week to be noticed and removed before it gets here. + +## What the job does + +The workflow `.github/workflows/upstream-sync.yml` runs `sync.sh` once a day. Each run: + +1. Fetches upstream's `main`, this fork's `main`, and the branch `upstream-sync-state`. That branch holds `seen.tsv`, one line per upstream head the job has seen (the full commit hash and the time the job first saw it, in UTC), and `last_checked`, the time of the last run. The first run creates the branch. +2. Adds upstream's current head to `seen.tsv` if it is not there yet. +3. Picks the newest commit in `seen.tsv` that was first seen at least 7 days ago and is still on upstream's `main`. Commit dates play no part, since anyone can backdate them. A commit that a force-push removed from upstream is skipped. +4. If that commit is not yet in `main`, merges it into `main` with a merge commit that names it and the time it was first seen. +5. Pushes `upstream-sync-state`, on every run, then `main` if it merged something. The daily push is also meant to keep the repository active: GitHub turns off scheduled workflows in a public repository after 60 days without activity. + +The job uses only `git` and `jq`: no third-party actions, and no upstream code is run. + +## Plugin versions + +Claude Code updates a plugin only when its version changes. Without a `version` field it uses the git commit, so each new commit of this fork counts as an update. The fork therefore keeps `version` deleted in `plugins/codex/.claude-plugin/plugin.json` and in the plugin entries of `.claude-plugin/marketplace.json`, while upstream changes the version in most commits. + +`.gitattributes` gives both files the merge driver `fork-manifest` (`fork-manifest.sh`), which `sync.sh` configures: where both sides changed the file, the result is upstream's file without the version fields. After each merge, `sync.sh` also deletes version fields that only upstream changed. Any other change this fork makes to these two files is replaced by upstream's next change to them. + +## When the job fails + +A failed run never changes `main`, and GitHub emails the person who last changed the workflow's schedule. The run's log says why it failed and which upstream commit it tried to merge. Merge that commit, not upstream's newer head: it is the one that has been on upstream for 7 days. + +- **Upstream changed `.github/workflows/`.** The job's token may not push workflow files. Read the workflow changes first: once merged, they run in this repository. Then run the job's script from a clone of `main` on the laptop, where git pushes with your own GitHub login, which needs the `workflow` scope. It applies the same 7-day rule: + + ```sh + git clone https://github.com/TruthfulAI-research/codex-plugin-cc.git && cd codex-plugin-cc + bash .github/upstream-sync/sync.sh + ``` + +- **The merge conflicts.** Merge by hand, keeping the version fields deleted: + + ```sh + git clone https://github.com/TruthfulAI-research/codex-plugin-cc.git && cd codex-plugin-cc + git config merge.fork-manifest.driver "bash '$PWD/.github/upstream-sync/fork-manifest.sh' %B %A %P" + git fetch https://github.com/Crazytieguy/codex-plugin-cc.git main + git merge --no-ff + # resolve the conflicts, commit, then: + git push origin main + ``` + + The next run sees that the commit is in `main` and goes on from there. + +## Dry runs and tests + +- A pull request that changes the sync files runs `test.sh` and a dry run, which merges locally and pushes nothing. +- A dry run by hand: `gh workflow run upstream-sync.yml -R TruthfulAI-research/codex-plugin-cc -f dry_run=true`, or `SYNC_DRY_RUN=1 bash .github/upstream-sync/sync.sh` locally. +- `bash .github/upstream-sync/test.sh` tests `sync.sh` on scratch repos (a fake upstream and a fake fork, with the clock set by `SYNC_NOW`). It needs `git` and `jq`. diff --git a/.github/upstream-sync/fork-manifest.sh b/.github/upstream-sync/fork-manifest.sh new file mode 100755 index 000000000..b5c08b4e8 --- /dev/null +++ b/.github/upstream-sync/fork-manifest.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash +# Writes a plugin manifest without its version fields: usage fork-manifest.sh . +# Git runs it as the merge driver "fork-manifest" (see .gitattributes) with upstream's file as the source, +# so the merge result is upstream's file without the version fields. sync.sh also runs it after each merge. +# Without a version, Claude Code updates the plugin on every new commit of the fork. +set -euo pipefail + +src=$1 dest=$2 path=$3 +if [ "$src" -ef "$dest" ]; then + echo "fork-manifest.sh: source and destination must be different files" >&2 + exit 1 +fi + +case $path in + plugins/codex/.claude-plugin/plugin.json) + has_version='has("version")' + strip='del(.version)' + ;; + .claude-plugin/marketplace.json) + has_version='any(.plugins[]?; has("version"))' + strip='del(.plugins[]?.version)' + ;; + *) + echo "fork-manifest.sh: not a plugin manifest: $path" >&2 + exit 1 + ;; +esac + +# Exits here on invalid JSON, which makes git report a conflict. +has=$(jq "$has_version" "$src") +if [ "$has" = true ]; then + jq "$strip" "$src" > "$dest" +else + # Keep the file byte for byte when there is nothing to delete. + cat "$src" > "$dest" +fi diff --git a/.github/upstream-sync/sync.sh b/.github/upstream-sync/sync.sh new file mode 100755 index 000000000..89eb1589d --- /dev/null +++ b/.github/upstream-sync/sync.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +# Merges upstream's main into this fork's main, but only commits first seen on upstream main at least +# 7 days ago. See README.md in this folder. +# +# Environment (all optional): +# SYNC_DRY_RUN=1 do everything locally, push nothing, print what would happen +# SYNC_TOKEN token for pushing to the fork over HTTPS (the job's GITHUB_TOKEN) +# SYNC_NOW the current time, as 2026-10-28T17:00:00Z (tests) +# SYNC_ORIGIN_URL the fork (default: this GitHub repository) +# SYNC_UPSTREAM_URL upstream (default: Crazytieguy/codex-plugin-cc) +set -euo pipefail +export LC_ALL=C + +here=$(cd "$(dirname "$0")" && pwd) +origin_url=${SYNC_ORIGIN_URL:-https://github.com/${GITHUB_REPOSITORY:-TruthfulAI-research/codex-plugin-cc}.git} +upstream_url=${SYNC_UPSTREAM_URL:-https://github.com/Crazytieguy/codex-plugin-cc.git} +state_branch=upstream-sync-state +min_age_seconds=$((7 * 24 * 60 * 60)) +manifests="plugins/codex/.claude-plugin/plugin.json .claude-plugin/marketplace.json" +tab=$(printf '\t') +time_re='^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$' + +die() { + if [ "${GITHUB_ACTIONS:-}" = true ]; then echo "::error::$*"; else echo "error: $*" >&2; fi + exit 1 +} + +command -v jq > /dev/null || die "jq is not installed" + +case ${SYNC_DRY_RUN:-0} in + 1 | true) dry_run=1 ;; + 0 | false | '') dry_run=0 ;; + *) die "SYNC_DRY_RUN must be 0 or 1" ;; +esac + +now=${SYNC_NOW:-$(date -u +%Y-%m-%dT%H:%M:%SZ)} +[[ $now =~ $time_re ]] || die "SYNC_NOW must look like 2026-10-28T17:00:00Z, not $now" +# Times in this format compare correctly as strings. +cutoff=$(jq -nr --arg now "$now" --argjson age "$min_age_seconds" '$now | fromdateiso8601 - $age | todate') + +work=$(mktemp -d "${RUNNER_TEMP:-${TMPDIR:-/tmp}}/upstream-sync.XXXXXX") +trap 'rm -rf "$work"' EXIT +git init -q -b sync "$work/repo" +cd "$work/repo" +if [ "${GITHUB_ACTIONS:-}" = true ]; then + git config user.name 'github-actions[bot]' + git config user.email '41898282+github-actions[bot]@users.noreply.github.com' +fi +git config merge.fork-manifest.name 'upstream manifest without version fields' +git config merge.fork-manifest.driver "bash '$here/fork-manifest.sh' %B %A %P" + +if [ -n "${SYNC_TOKEN:-}" ]; then + auth=$(printf 'x-access-token:%s' "$SYNC_TOKEN" | base64 | tr -d '\n') + if [ "${GITHUB_ACTIONS:-}" = true ]; then echo "::add-mask::$auth"; fi + # The header goes only to the fork's URL, and through the environment, so the token is in no file. + export GIT_CONFIG_COUNT=1 + export GIT_CONFIG_KEY_0="http.$origin_url.extraheader" + export GIT_CONFIG_VALUE_0="AUTHORIZATION: basic $auth" +fi + +git remote add origin "$origin_url" +git remote add upstream "$upstream_url" +git fetch -q --no-tags upstream +refs/heads/main:refs/remotes/upstream/main +git fetch -q --no-tags origin +refs/heads/main:refs/remotes/origin/main +status=0 +git ls-remote --exit-code origin "refs/heads/$state_branch" > /dev/null || status=$? +case $status in + 0) + have_state=1 + git fetch -q --no-tags origin "+refs/heads/$state_branch:refs/remotes/origin/$state_branch" + git show "refs/remotes/origin/$state_branch:seen.tsv" > "$work/seen.tsv" + ;; + 2) + have_state=0 + echo "The state branch $state_branch does not exist yet; this run creates it." + : > "$work/seen.tsv" + ;; + *) die "could not list the branches of $origin_url" ;; +esac + +if grep -Evq "^[0-9a-f]{40}${tab}[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z\$" "$work/seen.tsv"; then + die "seen.tsv on $state_branch has a malformed line" +fi +duplicates=$(cut -f1 "$work/seen.tsv" | sort | uniq -d) +if [ -n "$duplicates" ]; then + die "seen.tsv on $state_branch lists a commit twice" +fi + +head=$(git rev-parse refs/remotes/upstream/main) +head_seen=$(awk -F "$tab" -v head="$head" '$1 == head { print $2 }' "$work/seen.tsv") +if [ -n "$head_seen" ]; then + echo "Upstream main is at $head, first seen $head_seen." +else + printf '%s\t%s\n' "$head" "$now" >> "$work/seen.tsv" + echo "Upstream main is at $head, first seen now ($now)." +fi + +# The candidate: the newest commit first seen at least 7 days ago that is still on upstream main. +# Commit dates are never used: an attacker can backdate them. +candidate='' candidate_seen='' +sort -t "$tab" -k2,2r "$work/seen.tsv" > "$work/newest-first.tsv" +while IFS="$tab" read -r sha first_seen; do + if [[ $first_seen > $cutoff ]]; then continue; fi + if git cat-file -e "$sha^{commit}" 2> /dev/null && git merge-base --is-ancestor "$sha" refs/remotes/upstream/main; then + candidate=$sha candidate_seen=$first_seen + break + fi + echo "Skipping $sha (first seen $first_seen): it is no longer on upstream main." +done < "$work/newest-first.tsv" + +merged=0 conflict=0 +if [ -z "$candidate" ]; then + echo "No commit still on upstream main was first seen at least 7 days ago (by $cutoff): nothing to merge." +elif git merge-base --is-ancestor "$candidate" refs/remotes/origin/main; then + echo "$candidate (first seen $candidate_seen) is already in main: nothing to merge." +else + echo "Merging $candidate (first seen $candidate_seen). Upstream commits it brings:" + git log --oneline --no-decorate "refs/remotes/origin/main..$candidate" + git checkout -q --detach refs/remotes/origin/main + if git merge -q --no-ff --no-commit "$candidate"; then + # The merge driver runs only where both sides changed a manifest. This also deletes version + # fields that only upstream added or changed. + for path in $manifests; do + if [ -f "$path" ]; then + bash "$here/fork-manifest.sh" "$path" "$work/manifest" "$path" + if ! cmp -s "$work/manifest" "$path"; then + cp "$work/manifest" "$path" + git add "$path" + echo "Deleted the version fields in $path." + fi + fi + done + { + echo "Merge upstream commit ${candidate:0:12}" + echo + echo "Upstream: $upstream_url" + echo "Commit: $candidate" + echo "First seen on upstream main: $candidate_seen" + echo + echo "Merged by the upstream sync job, which merges an upstream commit only 7 days after" + echo "it first appeared on upstream main (.github/upstream-sync/README.md)." + } > "$work/message" + git commit -q --no-verify -F "$work/message" + merged=1 + else + conflict=1 + echo "The merge conflicts in:" + git diff --name-only --diff-filter=U + fi +fi + +workflow_commits='' +if [ "$merged" = 1 ]; then + workflow_commits=$(git log --format='%h %s' --full-history refs/remotes/origin/main..HEAD -- .github/workflows) +fi + +printf '%s\n' "$now" > "$work/last_checked" +seen_blob=$(git hash-object -w "$work/seen.tsv") +checked_blob=$(git hash-object -w "$work/last_checked") +tree=$(printf '100644 blob %s\tlast_checked\n100644 blob %s\tseen.tsv\n' "$checked_blob" "$seen_blob" | git mktree) +message="Upstream sync check at $now (upstream main $head)" +if [ "$have_state" = 1 ]; then + state_commit=$(git commit-tree "$tree" -p "refs/remotes/origin/$state_branch" -m "$message") +else + state_commit=$(git commit-tree "$tree" -m "$message") +fi + +if [ "$dry_run" = 1 ]; then + echo "Dry run: would push $state_branch ($(wc -l < "$work/seen.tsv" | tr -d ' ') rows in seen.tsv, last_checked $now)." +else + git push -q origin "$state_commit:refs/heads/$state_branch" + echo "Pushed $state_branch (last_checked $now)." +fi + +if [ "$conflict" = 1 ]; then + die "merging upstream commit $candidate into main conflicts; main was not changed. Merge by hand: see .github/upstream-sync/README.md." +fi +if [ "$merged" = 1 ]; then + if [ -n "$workflow_commits" ]; then + echo "These upstream commits change .github/workflows/, which the job's token may not push:" + echo "$workflow_commits" + fi + if [ "$dry_run" = 1 ]; then + echo "Dry run: would push main as $(git log -1 --format='%h %s')." + elif git push -q origin HEAD:refs/heads/main; then + echo "Pushed main: $(git log -1 --format='%h %s')." + elif [ -n "$workflow_commits" ]; then + die "pushing main failed: upstream changed .github/workflows/, which the job's token may not push; main was not changed. Merge by hand from the laptop: see .github/upstream-sync/README.md." + else + die "pushing main failed; main was not changed." + fi +fi diff --git a/.github/upstream-sync/test.sh b/.github/upstream-sync/test.sh new file mode 100644 index 000000000..af9445665 --- /dev/null +++ b/.github/upstream-sync/test.sh @@ -0,0 +1,235 @@ +#!/usr/bin/env bash +# Tests sync.sh on scratch repos: a fake upstream and a fake fork, with SYNC_NOW as the clock. +# Needs git and jq. Usage: bash .github/upstream-sync/test.sh +set -euo pipefail +export LC_ALL=C + +here=$(cd "$(dirname "$0")" && pwd) +repo_root=$(cd "$here/../.." && pwd) +root=$(mktemp -d "${TMPDIR:-/tmp}/upstream-sync-test.XXXXXX") +trap 'rm -rf "$root"' EXIT + +# Keep git away from the user's and the system's config, and from the job's settings. +export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 +export GIT_AUTHOR_NAME=test GIT_AUTHOR_EMAIL=test@example.com +export GIT_COMMITTER_NAME=test GIT_COMMITTER_EMAIL=test@example.com +unset GITHUB_ACTIONS RUNNER_TEMP SYNC_TOKEN SYNC_DRY_RUN SYNC_NOW GIT_CONFIG_COUNT + +up=$root/up # upstream (a plain repo, so a force-push is a reset) +fork=$root/fork.git # the fork's GitHub repo +tab=$(printf '\t') +plugin=plugins/codex/.claude-plugin/plugin.json +marketplace=.claude-plugin/marketplace.json +failures=0 + +pass() { echo "ok - $1"; } +fail() { + echo "FAIL - $1" + sed 's/^/ | /' "$root/out" + failures=$((failures + 1)) +} +check() { + local name=$1 + shift + if "$@"; then pass "$name"; else fail "$name"; fi +} +not() { ! "$@"; } + +# A fork whose main is upstream's first commit plus the fork's own commit (versions deleted, +# .gitattributes from this repo). Its pre-receive hook refuses workflow changes on main, as +# GitHub does for the job's token. +setup() { + rm -rf "$up" "$fork" "$root/clone" + git init -q -b main "$up" + mkdir -p "$up/plugins/codex/.claude-plugin" "$up/.claude-plugin" + printf '{\n "name": "codex",\n "version": "1.0.0",\n "description": "d"\n}\n' > "$up/$plugin" + printf '{\n "name": "m",\n "owner": { "name": "o" },\n "plugins": [\n { "name": "codex", "source": "./plugins/codex" }\n ]\n}\n' > "$up/$marketplace" + printf 'line 1\nline 2\n' > "$up/a.txt" + up_commit "upstream 0" > /dev/null + git clone -q --bare "$up" "$fork" + git clone -q "$fork" "$root/clone" + cp "$repo_root/.gitattributes" "$root/clone/.gitattributes" + jq 'del(.version)' "$up/$plugin" > "$root/clone/$plugin" + git -C "$root/clone" add -A + git -C "$root/clone" commit -q -m "fork: delete version, add merge driver" + git -C "$root/clone" push -q origin main + cat > "$fork/hooks/pre-receive" << 'EOF' +#!/bin/sh +while read -r old new ref; do + if [ "$ref" = refs/heads/main ] && [ -n "$(git rev-list --full-history "$old..$new" -- .github/workflows)" ]; then + echo "refusing to allow a GitHub App to create or update workflow without workflows permission" >&2 + exit 1 + fi +done +EOF + chmod +x "$fork/hooks/pre-receive" + main0=$(main_head) +} + +up_commit() { + git -C "$up" add -A + git -C "$up" commit -q -m "$1" + git -C "$up" rev-parse HEAD +} + +# run_sync [dry run: 0 or 1]: output in $root/out, exit status in $status. +run_sync() { + status=0 + SYNC_NOW=$1 SYNC_DRY_RUN=${2:-0} SYNC_ORIGIN_URL=$fork SYNC_UPSTREAM_URL=$up \ + bash "$here/sync.sh" > "$root/out" 2>&1 || status=$? +} + +main_head() { git -C "$fork" rev-parse main; } +has_ref() { git -C "$fork" rev-parse -q --verify "$1" > /dev/null; } +state_file() { git -C "$fork" show "upstream-sync-state:$1"; } +in_main() { git -C "$fork" merge-base --is-ancestor "$1" main 2> /dev/null; } +said() { grep -qF -- "$1" "$root/out"; } +succeeded() { [ "$status" = 0 ]; } +equal() { [ "$1" = "$2" ]; } + +echo "# a missing state branch is created" +setup +u0=$(git -C "$up" rev-parse HEAD) +run_sync 2026-01-01T00:00:00Z +check "first run succeeds" succeeded +check "state branch exists" has_ref refs/heads/upstream-sync-state +check "seen.tsv lists upstream's head with the time" equal "$(state_file seen.tsv)" "$u0${tab}2026-01-01T00:00:00Z" +check "last_checked is the time" equal "$(state_file last_checked)" 2026-01-01T00:00:00Z +check "main is unchanged" equal "$(main_head)" "$main0" + +echo "# no merge before 7 days, a merge after" +setup +run_sync 2026-01-01T00:00:00Z +printf 'line 1\nupstream line 2\n' > "$up/a.txt" +u1=$(up_commit "upstream 1") +run_sync 2026-01-02T00:00:00Z +run_sync 2026-01-08T23:59:59Z +check "a run 1 second before 7 days succeeds" succeeded +check "it merges nothing" equal "$(main_head)" "$main0" +check "it updates last_checked" equal "$(state_file last_checked)" 2026-01-08T23:59:59Z +run_sync 2026-01-09T00:00:00Z +check "a run at 7 days succeeds" succeeded +check "main's first parent is the old main" equal "$(git -C "$fork" rev-parse main^1)" "$main0" +check "main's second parent is the upstream commit" equal "$(git -C "$fork" rev-parse main^2)" "$u1" +check "the merge message names the commit" grep -qF "Commit: $u1" <(git -C "$fork" log -1 --format=%B main) +check "the merge message says when it was first seen" grep -qF "First seen on upstream main: 2026-01-02T00:00:00Z" <(git -C "$fork" log -1 --format=%B main) +merged=$(main_head) +run_sync 2026-01-10T00:00:00Z +check "a later run succeeds" succeeded +check "it says the commit is already in main" said "is already in main" +check "it merges nothing" equal "$(main_head)" "$merged" + +echo "# the newest commit seen at least 7 days ago is merged, not a newer one" +setup +printf 'line 1\nx\n' > "$up/a.txt" +u1=$(up_commit "upstream 1") +run_sync 2026-01-01T00:00:00Z +printf 'line 1\ny\n' > "$up/a.txt" +u2=$(up_commit "upstream 2") +run_sync 2026-01-04T00:00:00Z +run_sync 2026-01-08T00:00:00Z +check "the run succeeds" succeeded +check "the commit seen 7 days ago is in main" in_main "$u1" +check "the commit seen 4 days ago is not" not in_main "$u2" + +echo "# an upstream version bump merges without a conflict and the version stays deleted" +setup +printf '{\n "name": "codex",\n "version": "1.0.1",\n "description": "d2"\n}\n' > "$up/$plugin" +jq '.plugins[0].version = "1.0.1"' "$up/$marketplace" > "$root/m.json" && mv "$root/m.json" "$up/$marketplace" +up_commit "bump to 1.0.1" > /dev/null +run_sync 2026-01-01T00:00:00Z +run_sync 2026-01-08T00:00:00Z +check "the run succeeds" succeeded +check "main has the merge" equal "$(git -C "$fork" rev-parse main^1)" "$main0" +check "plugin.json is upstream's without the version" \ + equal "$(git -C "$fork" show "main:$plugin" | jq -c .)" '{"name":"codex","description":"d2"}' +check "marketplace.json has no version" \ + equal "$(git -C "$fork" show "main:$marketplace" | jq -c '[.plugins[] | has("version")]')" '[false]' + +echo "# a conflict fails the job and leaves main unchanged" +setup +printf 'fork line 1\nline 2\n' > "$root/clone/a.txt" +git -C "$root/clone" commit -q -am "fork change" +git -C "$root/clone" push -q origin main +main0=$(main_head) +printf 'upstream line 1\nline 2\n' > "$up/a.txt" +up_commit "upstream change" > /dev/null +run_sync 2026-01-01T00:00:00Z +run_sync 2026-01-08T00:00:00Z +check "the run fails" not succeeded +check "it names the conflict" said "conflicts" +check "main is unchanged" equal "$(main_head)" "$main0" +check "the state branch is still updated" equal "$(state_file last_checked)" 2026-01-08T00:00:00Z + +echo "# a backdated commit does not count as old" +setup +printf 'line 1\nold\n' > "$up/a.txt" +git -C "$up" add -A +GIT_AUTHOR_DATE=2020-01-01T00:00:00Z GIT_COMMITTER_DATE=2020-01-01T00:00:00Z git -C "$up" commit -q -m "backdated" +run_sync 2026-01-01T00:00:00Z +run_sync 2026-01-02T00:00:00Z +check "the run succeeds" succeeded +check "it merges nothing" equal "$(main_head)" "$main0" + +echo "# a commit that a force-push removed from upstream is skipped" +setup +printf 'line 1\nbad\n' > "$up/a.txt" +bad=$(up_commit "bad") +run_sync 2026-01-01T00:00:00Z +git -C "$up" reset -q --hard HEAD~1 +printf 'line 1\ngood\n' > "$up/a.txt" +good=$(up_commit "good") +run_sync 2026-01-02T00:00:00Z +run_sync 2026-01-08T01:00:00Z +check "the run succeeds" succeeded +check "it skips the removed commit" said "Skipping $bad" +check "it merges nothing" equal "$(main_head)" "$main0" +run_sync 2026-01-09T00:00:00Z +check "a run 7 days after the new head succeeds" succeeded +check "the new head is in main" in_main "$good" +check "the removed commit is not" not in_main "$bad" + +echo "# a dry run pushes nothing" +setup +printf 'line 1\nupstream line 2\n' > "$up/a.txt" +up_commit "upstream 1" > /dev/null +refs_before=$(git -C "$fork" for-each-ref) +run_sync 2026-01-01T00:00:00Z 1 +check "a dry run without a state branch succeeds" succeeded +check "it creates no state branch" equal "$(git -C "$fork" for-each-ref)" "$refs_before" +run_sync 2026-01-01T00:00:00Z +refs_before=$(git -C "$fork" for-each-ref) +run_sync 2026-01-08T00:00:00Z 1 +check "a dry run with a commit to merge succeeds" succeeded +check "it says it would push main" said "Dry run: would push main" +check "it pushes nothing" equal "$(git -C "$fork" for-each-ref)" "$refs_before" + +echo "# an upstream change to .github/workflows/ fails the push with a message" +setup +mkdir -p "$up/.github/workflows" +printf 'on: push\n' > "$up/.github/workflows/ci.yml" +up_commit "add a workflow" > /dev/null +run_sync 2026-01-01T00:00:00Z +run_sync 2026-01-08T00:00:00Z +check "the run fails" not succeeded +check "it says to merge by hand from the laptop" said "Merge by hand from the laptop" +check "main is unchanged" equal "$(main_head)" "$main0" +check "the state branch is still updated" equal "$(state_file last_checked)" 2026-01-08T00:00:00Z + +echo "# a malformed seen.tsv fails the job and pushes nothing" +setup +run_sync 2026-01-01T00:00:00Z +blob=$(printf 'not a commit\t2026-01-01T00:00:00Z\n' | git -C "$fork" hash-object -w --stdin) +tree=$(printf '100644 blob %s\tseen.tsv\n' "$blob" | git -C "$fork" mktree) +bad_state=$(git -C "$fork" commit-tree "$tree" -p upstream-sync-state -m "bad state") +git -C "$fork" update-ref refs/heads/upstream-sync-state "$bad_state" +run_sync 2026-01-02T00:00:00Z +check "the run fails" not succeeded +check "it names the problem" said "malformed line" +check "the state branch is unchanged" equal "$(git -C "$fork" rev-parse upstream-sync-state)" "$bad_state" + +if [ "$failures" -gt 0 ]; then + echo "$failures check(s) failed" + exit 1 +fi +echo "All checks passed." diff --git a/.github/workflows/upstream-sync.yml b/.github/workflows/upstream-sync.yml new file mode 100644 index 000000000..051f6850f --- /dev/null +++ b/.github/workflows/upstream-sync.yml @@ -0,0 +1,51 @@ +name: Upstream sync + +# Merges upstream's main (Crazytieguy/codex-plugin-cc) into this fork's main, but only commits +# first seen on upstream at least 7 days ago. See .github/upstream-sync/README.md. + +on: + schedule: + - cron: "17 6 * * *" + workflow_dispatch: + inputs: + dry_run: + description: Print what would happen and push nothing + type: boolean + default: false + pull_request: + paths: + - .github/workflows/upstream-sync.yml + - .github/upstream-sync/** + - .gitattributes + +permissions: + contents: write + +concurrency: + # One real run at a time; pull request runs are dry runs and queue on their own. + group: upstream-sync-${{ github.event_name == 'pull_request' && github.ref || 'main' }} + +jobs: + sync: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + # Plain git instead of actions/checkout, so the job runs no third-party code. + - name: Fetch this commit's sync scripts + env: + REPO_URL: https://github.com/${{ github.repository }}.git + run: | + git init -q -b src "$RUNNER_TEMP/src" + git -C "$RUNNER_TEMP/src" fetch -q --no-tags --depth=1 "$REPO_URL" "$GITHUB_SHA" + git -C "$RUNNER_TEMP/src" checkout -q --detach FETCH_HEAD + + - name: Test the sync scripts + if: github.event_name == 'pull_request' + run: bash "$RUNNER_TEMP/src/.github/upstream-sync/test.sh" + + - name: Sync + env: + SYNC_DRY_RUN: ${{ (github.event_name == 'pull_request' || inputs.dry_run) && '1' || '0' }} + # Dry runs push nothing, so they get no token. + SYNC_TOKEN: ${{ !(github.event_name == 'pull_request' || inputs.dry_run) && github.token || '' }} + run: bash "$RUNNER_TEMP/src/.github/upstream-sync/sync.sh" From 55e3e10090faaf2d24ed6a44a3af30e741fe65bb Mon Sep 17 00:00:00 2001 From: johannestreutlein Date: Wed, 28 Oct 2026 17:31:22 -0400 Subject: [PATCH 2/4] upstream-sync: shorter dry-run message; make test.sh executable Co-Authored-By: Claude Opus 5.5 --- .github/upstream-sync/sync.sh | 2 +- .github/upstream-sync/test.sh | 0 2 files changed, 1 insertion(+), 1 deletion(-) mode change 100644 => 100755 .github/upstream-sync/test.sh diff --git a/.github/upstream-sync/sync.sh b/.github/upstream-sync/sync.sh index 89eb1589d..f857eb311 100755 --- a/.github/upstream-sync/sync.sh +++ b/.github/upstream-sync/sync.sh @@ -166,7 +166,7 @@ else fi if [ "$dry_run" = 1 ]; then - echo "Dry run: would push $state_branch ($(wc -l < "$work/seen.tsv" | tr -d ' ') rows in seen.tsv, last_checked $now)." + echo "Dry run: would push $state_branch (last_checked $now)." else git push -q origin "$state_commit:refs/heads/$state_branch" echo "Pushed $state_branch (last_checked $now)." diff --git a/.github/upstream-sync/test.sh b/.github/upstream-sync/test.sh old mode 100644 new mode 100755 From f4debaf32076aba57eff9503176da92ba44f4a94 Mon Sep 17 00:00:00 2001 From: johannestreutlein Date: Wed, 28 Oct 2026 17:41:03 -0400 Subject: [PATCH 3/4] upstream-sync: log every head change, so a re-pushed commit waits 7 days again seen.tsv is now a log of upstream's head: a row whenever the head differs from the last row's commit, even if that commit is in an earlier row. The candidate is the newest row at least 7 days old whose commit is an ancestor of every later row's commit, so it stayed in upstream's history without a break. A commit that a force-push removed and that came back later counts only from its new row. As with a package manager's minimum release age, a commit that upstream reverts instead of removing is still merged on day 7; the README says so. Rows must be in time order. Co-Authored-By: Claude Opus 5.5 --- .github/upstream-sync/README.md | 24 +++++++--- .github/upstream-sync/sync.sh | 73 ++++++++++++++++++----------- .github/upstream-sync/test.sh | 65 +++++++++++++++++++++---- .github/workflows/upstream-sync.yml | 4 +- 4 files changed, 121 insertions(+), 45 deletions(-) diff --git a/.github/upstream-sync/README.md b/.github/upstream-sync/README.md index 51cb25f5b..ddad3a0d4 100644 --- a/.github/upstream-sync/README.md +++ b/.github/upstream-sync/README.md @@ -1,16 +1,20 @@ # Upstream sync -This fork takes the changes of its upstream, [Crazytieguy/codex-plugin-cc](https://github.com/Crazytieguy/codex-plugin-cc), automatically, but only once they are at least 7 days old. The delay is a defense against supply-chain attacks: every machine that installs the plugin updates itself from this fork's `main`, so a malicious commit pushed to upstream should have a week to be noticed and removed before it gets here. +This fork takes the changes of its upstream, [Crazytieguy/codex-plugin-cc](https://github.com/Crazytieguy/codex-plugin-cc), automatically, but only once they are at least 7 days old. The delay is a defense against supply-chain attacks: every machine that installs the plugin updates itself from this fork's `main`, so a malicious commit pushed to upstream should have a week to be noticed and removed before it gets here. The rule is the one package managers use for a minimum release age, such as npm's `min-release-age` or uv's `exclude-newer`: take the newest version that upstream has had for at least 7 days. ## What the job does The workflow `.github/workflows/upstream-sync.yml` runs `sync.sh` once a day. Each run: -1. Fetches upstream's `main`, this fork's `main`, and the branch `upstream-sync-state`. That branch holds `seen.tsv`, one line per upstream head the job has seen (the full commit hash and the time the job first saw it, in UTC), and `last_checked`, the time of the last run. The first run creates the branch. -2. Adds upstream's current head to `seen.tsv` if it is not there yet. -3. Picks the newest commit in `seen.tsv` that was first seen at least 7 days ago and is still on upstream's `main`. Commit dates play no part, since anyone can backdate them. A commit that a force-push removed from upstream is skipped. -4. If that commit is not yet in `main`, merges it into `main` with a merge commit that names it and the time it was first seen. -5. Pushes `upstream-sync-state`, on every run, then `main` if it merged something. The daily push is also meant to keep the repository active: GitHub turns off scheduled workflows in a public repository after 60 days without activity. +1. Fetches upstream's `main`, this fork's `main`, and the branch `upstream-sync-state`. That branch holds `seen.tsv`, a log of upstream's head with one row per change (the full commit hash and the time the job saw it, in UTC), and `last_checked`, the time of the last run. The first run creates the branch. +2. Adds a row to `seen.tsv` when upstream's head differs from the last row's commit, even if that commit is in an earlier row. +3. Picks the newest row that is at least 7 days old and whose commit has stayed in upstream's history since: it is an ancestor of every later row's commit, the last of which is upstream's current head. Commit dates play no part, since anyone can backdate them. +4. If that commit is not yet in `main`, merges it into `main` with a merge commit that names it and the time of its row. +5. Pushes `upstream-sync-state`, on every run, then `main` if it merged something. + +So a commit that a force-push removes from upstream within the 7 days is never merged, and a commit that comes back later counts only from its new row. + +What the rule does not cover: a bad commit that upstream reverts instead of removing still reaches this fork's `main` 7 days after it appeared on upstream, and the revert follows 7 days after it appeared. Packages behave the same way: a bad version that is replaced by a newer one, but not withdrawn, is still installed once it is old enough. The job uses only `git` and `jq`: no third-party actions, and no upstream code is run. @@ -20,6 +24,14 @@ Claude Code updates a plugin only when its version changes. Without a `version` `.gitattributes` gives both files the merge driver `fork-manifest` (`fork-manifest.sh`), which `sync.sh` configures: where both sides changed the file, the result is upstream's file without the version fields. After each merge, `sync.sh` also deletes version fields that only upstream changed. Any other change this fork makes to these two files is replaced by upstream's next change to them. +## When the job stops running + +GitHub turns off scheduled workflows in a public repository after 60 days without activity. The daily push to `upstream-sync-state` may count as activity; GitHub does not say. If GitHub turns the schedule off after a quiet spell, it sends an email, the fork stays on the last upstream commit it merged, and this turns the schedule back on: + +```sh +gh workflow enable upstream-sync.yml -R TruthfulAI-research/codex-plugin-cc +``` + ## When the job fails A failed run never changes `main`, and GitHub emails the person who last changed the workflow's schedule. The run's log says why it failed and which upstream commit it tried to merge. Merge that commit, not upstream's newer head: it is the one that has been on upstream for 7 days. diff --git a/.github/upstream-sync/sync.sh b/.github/upstream-sync/sync.sh index f857eb311..b1c936e36 100755 --- a/.github/upstream-sync/sync.sh +++ b/.github/upstream-sync/sync.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# Merges upstream's main into this fork's main, but only commits first seen on upstream main at least -# 7 days ago. See README.md in this folder. +# Merges upstream's main into this fork's main, but only commits that upstream's main has had for at +# least 7 days, like a package manager's minimum release age. See README.md in this folder. # # Environment (all optional): # SYNC_DRY_RUN=1 do everything locally, push nothing, print what would happen @@ -78,43 +78,60 @@ case $status in *) die "could not list the branches of $origin_url" ;; esac +# seen.tsv is a log of upstream's head: a row whenever the head differs from the last row's commit, +# even if that commit is in an earlier row. So the last row is upstream's current head. +head=$(git rev-parse refs/remotes/upstream/main) +last_row=$(tail -n 1 "$work/seen.tsv") +if [ "${last_row%%"$tab"*}" = "$head" ]; then + echo "Upstream main is at $head, its head since ${last_row#*"$tab"}." +else + printf '%s\t%s\n' "$head" "$now" >> "$work/seen.tsv" + echo "Upstream main is at $head, a new head as of now ($now)." +fi + if grep -Evq "^[0-9a-f]{40}${tab}[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z\$" "$work/seen.tsv"; then die "seen.tsv on $state_branch has a malformed line" fi -duplicates=$(cut -f1 "$work/seen.tsv" | sort | uniq -d) -if [ -n "$duplicates" ]; then - die "seen.tsv on $state_branch lists a commit twice" +if ! awk -F "$tab" 'NR > 1 && $2 < previous { exit 1 } { previous = $2 }' "$work/seen.tsv"; then + die "seen.tsv on $state_branch is not in time order, or the clock is behind its last row" fi -head=$(git rev-parse refs/remotes/upstream/main) -head_seen=$(awk -F "$tab" -v head="$head" '$1 == head { print $2 }' "$work/seen.tsv") -if [ -n "$head_seen" ]; then - echo "Upstream main is at $head, first seen $head_seen." -else - printf '%s\t%s\n' "$head" "$now" >> "$work/seen.tsv" - echo "Upstream main is at $head, first seen now ($now)." -fi +shas=() times=() +while IFS="$tab" read -r sha time; do + shas+=("$sha") + times+=("$time") +done < "$work/seen.tsv" + +# stayed : whether the row's commit is an ancestor of every later row's commit, that is, stayed +# in upstream's history without a break since that row. A later commit that is no longer on upstream +# main is not fetched; it counts as a break. +stayed() { + local j + for ((j = $1 + 1; j < ${#shas[@]}; j++)); do + git merge-base --is-ancestor "${shas[$1]}" "${shas[$j]}" 2> /dev/null || return 1 + done +} -# The candidate: the newest commit first seen at least 7 days ago that is still on upstream main. -# Commit dates are never used: an attacker can backdate them. +# The candidate: the newest row at least 7 days old whose commit stayed in upstream's history since. +# A commit that left upstream's history and came back counts only from its new row. Commit dates +# are never used: an attacker can backdate them. candidate='' candidate_seen='' -sort -t "$tab" -k2,2r "$work/seen.tsv" > "$work/newest-first.tsv" -while IFS="$tab" read -r sha first_seen; do - if [[ $first_seen > $cutoff ]]; then continue; fi - if git cat-file -e "$sha^{commit}" 2> /dev/null && git merge-base --is-ancestor "$sha" refs/remotes/upstream/main; then - candidate=$sha candidate_seen=$first_seen +for ((i = ${#shas[@]} - 1; i >= 0; i--)); do + if [[ ${times[$i]} > $cutoff ]]; then continue; fi + if stayed "$i"; then + candidate=${shas[$i]} candidate_seen=${times[$i]} break fi - echo "Skipping $sha (first seen $first_seen): it is no longer on upstream main." -done < "$work/newest-first.tsv" + echo "Skipping ${shas[$i]} (on upstream main at ${times[$i]}): not every later head is known to contain it." +done merged=0 conflict=0 if [ -z "$candidate" ]; then - echo "No commit still on upstream main was first seen at least 7 days ago (by $cutoff): nothing to merge." + echo "No commit has been on upstream main for 7 days without a break (since $cutoff or earlier): nothing to merge." elif git merge-base --is-ancestor "$candidate" refs/remotes/origin/main; then - echo "$candidate (first seen $candidate_seen) is already in main: nothing to merge." + echo "$candidate (on upstream main since $candidate_seen) is already in main: nothing to merge." else - echo "Merging $candidate (first seen $candidate_seen). Upstream commits it brings:" + echo "Merging $candidate (on upstream main since $candidate_seen). Upstream commits it brings:" git log --oneline --no-decorate "refs/remotes/origin/main..$candidate" git checkout -q --detach refs/remotes/origin/main if git merge -q --no-ff --no-commit "$candidate"; then @@ -135,10 +152,10 @@ else echo echo "Upstream: $upstream_url" echo "Commit: $candidate" - echo "First seen on upstream main: $candidate_seen" + echo "On upstream main since: $candidate_seen" echo - echo "Merged by the upstream sync job, which merges an upstream commit only 7 days after" - echo "it first appeared on upstream main (.github/upstream-sync/README.md)." + echo "Merged by the upstream sync job, which merges an upstream commit only once it has" + echo "been on upstream main for 7 days (.github/upstream-sync/README.md)." } > "$work/message" git commit -q --no-verify -F "$work/message" merged=1 diff --git a/.github/upstream-sync/test.sh b/.github/upstream-sync/test.sh index af9445665..af5b33b58 100755 --- a/.github/upstream-sync/test.sh +++ b/.github/upstream-sync/test.sh @@ -87,6 +87,15 @@ said() { grep -qF -- "$1" "$root/out"; } succeeded() { [ "$status" = 0 ]; } equal() { [ "$1" = "$2" ]; } +# put_state : replaces seen.tsv on the state branch, as anyone with write access could. +put_state() { + local blob tree + blob=$(printf '%s\n' "$1" | git -C "$fork" hash-object -w --stdin) + tree=$(printf '100644 blob %s\tseen.tsv\n' "$blob" | git -C "$fork" mktree) + edited_state=$(git -C "$fork" commit-tree "$tree" -p upstream-sync-state -m "edited state") + git -C "$fork" update-ref refs/heads/upstream-sync-state "$edited_state" +} + echo "# a missing state branch is created" setup u0=$(git -C "$up" rev-parse HEAD) @@ -112,7 +121,8 @@ check "a run at 7 days succeeds" succeeded check "main's first parent is the old main" equal "$(git -C "$fork" rev-parse main^1)" "$main0" check "main's second parent is the upstream commit" equal "$(git -C "$fork" rev-parse main^2)" "$u1" check "the merge message names the commit" grep -qF "Commit: $u1" <(git -C "$fork" log -1 --format=%B main) -check "the merge message says when it was first seen" grep -qF "First seen on upstream main: 2026-01-02T00:00:00Z" <(git -C "$fork" log -1 --format=%B main) +check "the merge message says since when it was on upstream main" \ + grep -qF "On upstream main since: 2026-01-02T00:00:00Z" <(git -C "$fork" log -1 --format=%B main) merged=$(main_head) run_sync 2026-01-10T00:00:00Z check "a later run succeeds" succeeded @@ -132,6 +142,41 @@ check "the run succeeds" succeeded check "the commit seen 7 days ago is in main" in_main "$u1" check "the commit seen 4 days ago is not" not in_main "$u2" +echo "# a re-pushed commit counts only from its re-push" +setup +printf 'line 1\nm\n' > "$up/a.txt" +m=$(up_commit "m") +run_sync 2026-01-01T00:00:00Z +git -C "$up" reset -q --hard HEAD~1 +printf 'line 1\nx\n' > "$up/a.txt" +x=$(up_commit "x") +run_sync 2026-01-02T00:00:00Z +git -C "$up" reset -q --hard "$m" +run_sync 2026-01-04T00:00:00Z +check "seen.tsv logs the re-pushed commit again" equal "$(state_file seen.tsv | cut -f1 | tr '\n' ' ')" "$m $x $m " +run_sync 2026-01-10T23:59:59Z +check "a run 7 days after its first row succeeds" succeeded +check "it skips the first row" said "Skipping $m (on upstream main at 2026-01-01T00:00:00Z)" +check "it merges nothing" equal "$(main_head)" "$main0" +run_sync 2026-01-11T00:00:00Z +check "7 days after the re-push, the commit is merged" equal "$(git -C "$fork" rev-parse main^2)" "$m" +check "the commit that replaced it for 2 days is not in main" not in_main "$x" + +echo "# a commit reverted within the week is still merged on day 7, as with packages" +setup +run_sync 2026-01-01T00:00:00Z +printf 'line 1\nbad\n' > "$up/a.txt" +m=$(up_commit "bad") +run_sync 2026-01-02T00:00:00Z +git -C "$up" revert --no-edit HEAD > /dev/null +r=$(git -C "$up" rev-parse HEAD) +run_sync 2026-01-03T00:00:00Z +run_sync 2026-01-09T00:00:00Z +check "the run on day 7 succeeds" succeeded +check "it merges the reverted commit" equal "$(git -C "$fork" rev-parse main^2)" "$m" +run_sync 2026-01-10T00:00:00Z +check "the revert follows a day later" equal "$(git -C "$fork" rev-parse main^2)" "$r" + echo "# an upstream version bump merges without a conflict and the version stays deleted" setup printf '{\n "name": "codex",\n "version": "1.0.1",\n "description": "d2"\n}\n' > "$up/$plugin" @@ -219,14 +264,16 @@ check "the state branch is still updated" equal "$(state_file last_checked)" 202 echo "# a malformed seen.tsv fails the job and pushes nothing" setup run_sync 2026-01-01T00:00:00Z -blob=$(printf 'not a commit\t2026-01-01T00:00:00Z\n' | git -C "$fork" hash-object -w --stdin) -tree=$(printf '100644 blob %s\tseen.tsv\n' "$blob" | git -C "$fork" mktree) -bad_state=$(git -C "$fork" commit-tree "$tree" -p upstream-sync-state -m "bad state") -git -C "$fork" update-ref refs/heads/upstream-sync-state "$bad_state" -run_sync 2026-01-02T00:00:00Z -check "the run fails" not succeeded -check "it names the problem" said "malformed line" -check "the state branch is unchanged" equal "$(git -C "$fork" rev-parse upstream-sync-state)" "$bad_state" +u0=$(git -C "$up" rev-parse HEAD) +for content in "not a commit${tab}2026-01-01T00:00:00Z" \ + "$u0${tab}2026-01-03T00:00:00Z +$u0${tab}2026-01-01T00:00:00Z"; do + put_state "$content" + run_sync 2026-01-04T00:00:00Z + check "the run fails" not succeeded + check "it names the problem" said "seen.tsv on upstream-sync-state" + check "the state branch is unchanged" equal "$(git -C "$fork" rev-parse upstream-sync-state)" "$edited_state" +done if [ "$failures" -gt 0 ]; then echo "$failures check(s) failed" diff --git a/.github/workflows/upstream-sync.yml b/.github/workflows/upstream-sync.yml index 051f6850f..2fddcdf64 100644 --- a/.github/workflows/upstream-sync.yml +++ b/.github/workflows/upstream-sync.yml @@ -1,7 +1,7 @@ name: Upstream sync -# Merges upstream's main (Crazytieguy/codex-plugin-cc) into this fork's main, but only commits -# first seen on upstream at least 7 days ago. See .github/upstream-sync/README.md. +# Merges upstream's main (Crazytieguy/codex-plugin-cc) into this fork's main, but only commits that +# upstream's main has had for at least 7 days. See .github/upstream-sync/README.md. on: schedule: From f1a52c56f56c0a98a408f20af2c0e29c639875c9 Mon Sep 17 00:00:00 2001 From: johannestreutlein Date: Wed, 28 Oct 2026 18:05:22 -0400 Subject: [PATCH 4/4] upstream-sync: never merge upstream changes to the job; check GitHub's activity log - Upstream commits that touch .github/ or .gitattributes (adds, edits, mode changes, symbolic links) are never merged automatically: the run fails and asks for a hand merge. A second check compares the merge result with main. So upstream can never change the job's scripts, workflow or merge driver. The check after a failed push is gone. - Before merging, the job reads GitHub's activity log of upstream's main. A force-push or deletion of main after the candidate's row blocks the merge; if the head is still the last row's commit, it is logged again. If the log cannot be read, the run fails without merging. - The run fails when upstream's main no longer has a commit that the fork merged, and when an upstream commit has waited 30 days. - seen.tsv must end with a newline. - test.sh: the fake fork carries the real sync files and workflow, and its hook models GitHub's rule as checked on 2026-10-28 (a new commit must not change .github/workflows/ compared with its first parent). New cases cover the checks above, a clock behind the last row, and merges on top of earlier sync merges. Co-Authored-By: Claude Opus 5.5 --- .github/upstream-sync/README.md | 55 ++++--- .github/upstream-sync/sync.sh | 174 +++++++++++++++------- .github/upstream-sync/test.sh | 215 +++++++++++++++++++++++++--- .github/workflows/upstream-sync.yml | 4 +- 4 files changed, 353 insertions(+), 95 deletions(-) diff --git a/.github/upstream-sync/README.md b/.github/upstream-sync/README.md index ddad3a0d4..5ee2b52b9 100644 --- a/.github/upstream-sync/README.md +++ b/.github/upstream-sync/README.md @@ -9,14 +9,20 @@ The workflow `.github/workflows/upstream-sync.yml` runs `sync.sh` once a day. Ea 1. Fetches upstream's `main`, this fork's `main`, and the branch `upstream-sync-state`. That branch holds `seen.tsv`, a log of upstream's head with one row per change (the full commit hash and the time the job saw it, in UTC), and `last_checked`, the time of the last run. The first run creates the branch. 2. Adds a row to `seen.tsv` when upstream's head differs from the last row's commit, even if that commit is in an earlier row. 3. Picks the newest row that is at least 7 days old and whose commit has stayed in upstream's history since: it is an ancestor of every later row's commit, the last of which is upstream's current head. Commit dates play no part, since anyone can backdate them. -4. If that commit is not yet in `main`, merges it into `main` with a merge commit that names it and the time of its row. -5. Pushes `upstream-sync-state`, on every run, then `main` if it merged something. +4. Merges nothing if GitHub's activity log of upstream's `main` shows a force-push or a deletion of `main` after that row's time. +5. Merges nothing, and fails, if any upstream commit to merge touches `.github/` or `.gitattributes`, file modes and symbolic links included. The job's scripts, its workflow and its merge driver live there, so upstream can never change them. +6. Otherwise merges the commit into `main`, with a merge commit that names it and the time of its row. +7. Pushes `upstream-sync-state`, on every run, then `main` if it merged something. -So a commit that a force-push removes from upstream within the 7 days is never merged, and a commit that comes back later counts only from its new row. +## How force-pushes are handled + +The job looks at upstream once a day, so `seen.tsv` shows upstream's head only at those times. A commit that a force-push removes from upstream is never merged, and a commit that comes back counts only from its new row. A removal and restore between two runs leaves no trace in `seen.tsv`; GitHub's activity log shows it, and step 4 then holds back every row logged before it. When the head after such a force-push is the same commit as the last row, the job logs it again, so it counts from that run. + +So after any force-push of upstream's `main`, nothing is merged until a head logged after it has been on upstream for 7 days. If an upstream commit waits more than 30 days, the job fails and says why. What the rule does not cover: a bad commit that upstream reverts instead of removing still reaches this fork's `main` 7 days after it appeared on upstream, and the revert follows 7 days after it appeared. Packages behave the same way: a bad version that is replaced by a newer one, but not withdrawn, is still installed once it is old enough. -The job uses only `git` and `jq`: no third-party actions, and no upstream code is run. +The job uses `git`, `jq` and `curl` (for GitHub's activity log), and no third-party actions. It never runs upstream code: step 5 keeps upstream from changing what the job runs. ## Plugin versions @@ -26,7 +32,7 @@ Claude Code updates a plugin only when its version changes. Without a `version` ## When the job stops running -GitHub turns off scheduled workflows in a public repository after 60 days without activity. The daily push to `upstream-sync-state` may count as activity; GitHub does not say. If GitHub turns the schedule off after a quiet spell, it sends an email, the fork stays on the last upstream commit it merged, and this turns the schedule back on: +GitHub turns off scheduled workflows in a public repository after 60 days without activity. The daily push to `upstream-sync-state` may count as activity; GitHub does not say. If GitHub turns the schedule off, it sends an email, the fork stays on the last upstream commit it merged, and this turns the schedule back on: ```sh gh workflow enable upstream-sync.yml -R TruthfulAI-research/codex-plugin-cc @@ -34,30 +40,33 @@ gh workflow enable upstream-sync.yml -R TruthfulAI-research/codex-plugin-cc ## When the job fails -A failed run never changes `main`, and GitHub emails the person who last changed the workflow's schedule. The run's log says why it failed and which upstream commit it tried to merge. Merge that commit, not upstream's newer head: it is the one that has been on upstream for 7 days. - -- **Upstream changed `.github/workflows/`.** The job's token may not push workflow files. Read the workflow changes first: once merged, they run in this repository. Then run the job's script from a clone of `main` on the laptop, where git pushes with your own GitHub login, which needs the `workflow` scope. It applies the same 7-day rule: +GitHub emails the person who last changed the workflow's schedule. The run's log says why the run failed and names the upstream commit. Only the 30-day alert can come from a run that also merged something; every other failed run leaves `main` as it was. - ```sh - git clone https://github.com/TruthfulAI-research/codex-plugin-cc.git && cd codex-plugin-cc - bash .github/upstream-sync/sync.sh - ``` +- **Upstream changed `.github/` or `.gitattributes`.** Read the change first: merged workflows run in this repository, and this fork's sync files should stay as they are unless you mean to change them. Then merge by hand, as below, from a machine whose token may push workflow files (Johannes's laptop). +- **The merge conflicts.** Merge by hand, as below. +- **Upstream removed commits that this fork merged.** Upstream may have removed them because they were bad. Check them, and revert them in `main` if needed. The job fails on every run until their rows are deleted from `seen.tsv` on the branch `upstream-sync-state`; delete them once the commits are checked. +- **An upstream commit has waited 30 days.** The log says what held it back, usually force-pushes on upstream. Check upstream; the job merges again once a head has been on upstream for 7 days without a force-push. +- **GitHub's activity log could not be read.** Nothing was merged; the next run tries again. -- **The merge conflicts.** Merge by hand, keeping the version fields deleted: +To merge by hand, merge the commit that the log names, not upstream's newer head: it is the one that has been on upstream for 7 days. - ```sh - git clone https://github.com/TruthfulAI-research/codex-plugin-cc.git && cd codex-plugin-cc - git config merge.fork-manifest.driver "bash '$PWD/.github/upstream-sync/fork-manifest.sh' %B %A %P" - git fetch https://github.com/Crazytieguy/codex-plugin-cc.git main - git merge --no-ff - # resolve the conflicts, commit, then: - git push origin main - ``` +```sh +git clone https://github.com/TruthfulAI-research/codex-plugin-cc.git && cd codex-plugin-cc +git config merge.fork-manifest.driver "bash '$PWD/.github/upstream-sync/fork-manifest.sh' %B %A %P" +git fetch https://github.com/Crazytieguy/codex-plugin-cc.git main +git merge --no-ff --no-commit +# Resolve any conflicts, then delete the version fields that the merge driver did not: +for f in plugins/codex/.claude-plugin/plugin.json .claude-plugin/marketplace.json; do + bash .github/upstream-sync/fork-manifest.sh "$f" "$f.new" "$f" && mv "$f.new" "$f" && git add "$f" +done +git commit +git push origin main +``` - The next run sees that the commit is in `main` and goes on from there. +The next run sees that the commit is in `main` and goes on from there. ## Dry runs and tests -- A pull request that changes the sync files runs `test.sh` and a dry run, which merges locally and pushes nothing. +- A pull request that changes the sync files runs `test.sh` and a dry run, which merges locally and pushes nothing. The dry run merges upstream's current candidate into the current `main`, so it can fail because of an upstream conflict that has nothing to do with the pull request. - A dry run by hand: `gh workflow run upstream-sync.yml -R TruthfulAI-research/codex-plugin-cc -f dry_run=true`, or `SYNC_DRY_RUN=1 bash .github/upstream-sync/sync.sh` locally. - `bash .github/upstream-sync/test.sh` tests `sync.sh` on scratch repos (a fake upstream and a fake fork, with the clock set by `SYNC_NOW`). It needs `git` and `jq`. diff --git a/.github/upstream-sync/sync.sh b/.github/upstream-sync/sync.sh index b1c936e36..896536b22 100755 --- a/.github/upstream-sync/sync.sh +++ b/.github/upstream-sync/sync.sh @@ -6,16 +6,20 @@ # SYNC_DRY_RUN=1 do everything locally, push nothing, print what would happen # SYNC_TOKEN token for pushing to the fork over HTTPS (the job's GITHUB_TOKEN) # SYNC_NOW the current time, as 2026-10-28T17:00:00Z (tests) +# SYNC_API_TOKEN token for reading GitHub's activity log of upstream (read only) # SYNC_ORIGIN_URL the fork (default: this GitHub repository) # SYNC_UPSTREAM_URL upstream (default: Crazytieguy/codex-plugin-cc) +# SYNC_ACTIVITY_URL GitHub's activity log of upstream (tests use a file:// URL) set -euo pipefail export LC_ALL=C here=$(cd "$(dirname "$0")" && pwd) origin_url=${SYNC_ORIGIN_URL:-https://github.com/${GITHUB_REPOSITORY:-TruthfulAI-research/codex-plugin-cc}.git} upstream_url=${SYNC_UPSTREAM_URL:-https://github.com/Crazytieguy/codex-plugin-cc.git} +activity_url=${SYNC_ACTIVITY_URL:-https://api.github.com/repos/Crazytieguy/codex-plugin-cc/activity} state_branch=upstream-sync-state -min_age_seconds=$((7 * 24 * 60 * 60)) +min_age_days=7 +stall_days=30 manifests="plugins/codex/.claude-plugin/plugin.json .claude-plugin/marketplace.json" tab=$(printf '\t') time_re='^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$' @@ -36,7 +40,9 @@ esac now=${SYNC_NOW:-$(date -u +%Y-%m-%dT%H:%M:%SZ)} [[ $now =~ $time_re ]] || die "SYNC_NOW must look like 2026-10-28T17:00:00Z, not $now" # Times in this format compare correctly as strings. -cutoff=$(jq -nr --arg now "$now" --argjson age "$min_age_seconds" '$now | fromdateiso8601 - $age | todate') +days_before() { jq -nr --arg now "$now" --argjson days "$1" '$now | fromdateiso8601 - $days * 86400 | todate'; } +cutoff=$(days_before "$min_age_days") +stall_cutoff=$(days_before "$stall_days") work=$(mktemp -d "${RUNNER_TEMP:-${TMPDIR:-/tmp}}/upstream-sync.XXXXXX") trap 'rm -rf "$work"' EXIT @@ -89,6 +95,9 @@ else echo "Upstream main is at $head, a new head as of now ($now)." fi +if [ -s "$work/seen.tsv" ] && [ -n "$(tail -c 1 "$work/seen.tsv")" ]; then + die "seen.tsv on $state_branch does not end with a newline" +fi if grep -Evq "^[0-9a-f]{40}${tab}[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z\$" "$work/seen.tsv"; then die "seen.tsv on $state_branch has a malformed line" fi @@ -102,6 +111,15 @@ while IFS="$tab" read -r sha time; do times+=("$time") done < "$work/seen.tsv" +# Commits that this fork merged and that upstream's main no longer has: upstream removed them after +# the 7 days, which may mean that they were bad. +removed=$(for sha in "${shas[@]}"; do + if git merge-base --is-ancestor "$sha" refs/remotes/origin/main 2> /dev/null && + ! git merge-base --is-ancestor "$sha" refs/remotes/upstream/main 2> /dev/null; then + echo "$sha" + fi +done | sort -u) + # stayed : whether the row's commit is an ancestor of every later row's commit, that is, stayed # in upstream's history without a break since that row. A later commit that is no longer on upstream # main is not fetched; it counts as a break. @@ -125,51 +143,112 @@ for ((i = ${#shas[@]} - 1; i >= 0; i--)); do echo "Skipping ${shas[$i]} (on upstream main at ${times[$i]}): not every later head is known to contain it." done -merged=0 conflict=0 -if [ -z "$candidate" ]; then - echo "No commit has been on upstream main for 7 days without a break (since $cutoff or earlier): nothing to merge." +# rewrites_since