Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/scripts/update-marketplace.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
set -euo pipefail

usage() {
printf 'usage: %s <marketplace.json> <X.Y.Z>\n' "${0##*/}" >&2
exit 2
}

[[ $# -eq 2 ]] || usage
index=$1
version=$2

if [[ ! -f $index ]]; then
printf 'marketplace index not found: %s\n' "$index" >&2
exit 1
fi

if ! command -v jq >/dev/null; then
printf 'required command not found: jq\n' >&2
exit 1
fi

if ! [[ $version =~ ^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$ ]]; then
printf 'release version must use X.Y.Z: %s\n' "$version" >&2
exit 1
fi

count=$(jq '[.plugins[] | select(.name == "behavior-diff")] | length' "$index")
if [[ $count -ne 1 ]]; then
printf 'expected one behavior-diff marketplace entry, found %s\n' "$count" >&2
exit 1
fi

tmp=$(mktemp "${index}.tmp.XXXXXX")
trap 'rm -f "$tmp"' EXIT
jq --arg version "$version" '
(.plugins[] | select(.name == "behavior-diff") | .version) = $version |
(.plugins[] | select(.name == "behavior-diff") | .source.ref) = ("v" + $version)
' "$index" >"$tmp"

jq -e --arg version "$version" '
[.plugins[] |
select(.name == "behavior-diff" and
.version == $version and
.source.ref == ("v" + $version))] |
length == 1
' "$tmp" >/dev/null

mv "$tmp" "$index"
trap - EXIT
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: CI

on:
workflow_call:
pull_request:
push:
branches:
Expand Down Expand Up @@ -67,6 +68,7 @@ jobs:
- name: Check shell syntax
run: |
bash -n \
.github/scripts/*.sh \
bin/behavior-diff \
plugin/scripts/*.sh \
plugin/skills/behavior-diff/scripts/*.sh \
Expand All @@ -86,3 +88,6 @@ jobs:

- name: Run report contract checks
run: bash tests/live-report-contract.sh

- name: Run release workflow checks
run: bash tests/release-workflow-test.sh
87 changes: 87 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: Release

on:
release:
types: [published]

permissions:
contents: read

jobs:
ci:
if: ${{ !github.event.release.prerelease }}
uses: ./.github/workflows/ci.yml

marketplace:
if: ${{ !github.event.release.prerelease }}
needs: ci
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out released tag
uses: actions/checkout@v5
with:
ref: refs/tags/${{ github.event.release.tag_name }}
fetch-depth: 0

- name: Validate release
env:
RELEASE_TAG: ${{ github.event.release.tag_name }}
run: |
set -euo pipefail
if ! [[ $RELEASE_TAG =~ ^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$ ]]; then
printf 'stable release tag must use vX.Y.Z: %s\n' "$RELEASE_TAG" >&2
exit 1
fi

VERSION=${RELEASE_TAG#v}
CLAUDE_VERSION=$(jq -r '.version' plugin/.claude-plugin/plugin.json)
CODEX_VERSION=$(jq -r '.version' plugin/.codex-plugin/plugin.json)
if [[ $CLAUDE_VERSION != "$VERSION" || $CODEX_VERSION != "$VERSION" ]]; then
printf 'release %s does not match plugin manifests (%s, %s)\n' \
"$VERSION" "$CLAUDE_VERSION" "$CODEX_VERSION" >&2
exit 1
fi

git fetch origin \
'+refs/heads/main:refs/remotes/origin/main'
if ! git merge-base --is-ancestor HEAD origin/main; then
printf 'released commit is not on main: %s\n' "$RELEASE_TAG" >&2
exit 1
fi

printf 'VERSION=%s\n' "$VERSION" >>"$GITHUB_ENV"

- name: Publish marketplace entry
env:
MARKETPLACE_DEPLOY_KEY: ${{ secrets.MARKETPLACE_DEPLOY_KEY }}
run: |
set -euo pipefail
test -n "$MARKETPLACE_DEPLOY_KEY"
umask 077

MARKETPLACE=$RUNNER_TEMP/marketplace
SSH_DIR=$RUNNER_TEMP/marketplace-ssh
install -d -m 700 "$SSH_DIR"
trap 'rm -rf "$SSH_DIR"' EXIT
printf '%s\n' "$MARKETPLACE_DEPLOY_KEY" >"$SSH_DIR/key"
curl --fail --silent --show-error https://api.github.com/meta |
jq -r '.ssh_keys[] | "github.com " + .' >"$SSH_DIR/known_hosts"
test -s "$SSH_DIR/known_hosts"
export GIT_SSH_COMMAND="ssh -i $SSH_DIR/key -o IdentitiesOnly=yes -o IdentityAgent=none -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$SSH_DIR/known_hosts"

git clone --depth 1 git@github.com:spacedock-dev/marketplace.git "$MARKETPLACE"
INDEX=$MARKETPLACE/.claude-plugin/marketplace.json
"$GITHUB_WORKSPACE/.github/scripts/update-marketplace.sh" "$INDEX" "$VERSION"
git -C "$MARKETPLACE" diff --check
if git -C "$MARKETPLACE" diff --quiet -- .claude-plugin/marketplace.json; then
printf 'marketplace already publishes Behavior Diff %s\n' "$VERSION"
exit 0
fi

git -C "$MARKETPLACE" add -- .claude-plugin/marketplace.json
git -C "$MARKETPLACE" \
-c user.name=github-actions \
-c user.email=actions@github.com \
commit --signoff -m "behavior-diff $VERSION"
git -C "$MARKETPLACE" push origin HEAD:main
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ For the full deterministic suite, run:
bash tests/hooks-test.sh
python3 plugin/skills/behavior-diff/scripts/decisions.py --check
bash tests/live-report-contract.sh
bash tests/release-workflow-test.sh
```

For Markdown-only changes, also run `git diff --check`. Do not replace these
Expand Down
3 changes: 3 additions & 0 deletions CODING_GUIDELINES.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,13 @@ docker run --rm -v "$PWD:/mnt" -w /mnt \
mvdan/shfmt:v3.14.0 -d -i 2 -ci .
uvx ruff@0.16.5 format --check --diff .
bash -n \
.github/scripts/*.sh \
bin/behavior-diff \
plugin/scripts/*.sh \
plugin/skills/behavior-diff/scripts/*.sh \
tests/*.sh
shellcheck \
.github/scripts/*.sh \
bin/behavior-diff \
plugin/scripts/*.sh \
plugin/skills/behavior-diff/scripts/*.sh \
Expand All @@ -156,6 +158,7 @@ python3 -m py_compile \
bash tests/hooks-test.sh
python3 plugin/skills/behavior-diff/scripts/decisions.py --check
bash tests/live-report-contract.sh
bash tests/release-workflow-test.sh
git diff --check
```

Expand Down
35 changes: 21 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ Use Behavior Diff when you:

## Install

Behavior Diff supports Claude Code and Codex. The commands below are for the
public marketplace release. The marketplace entry is not live yet.
Behavior Diff supports Claude Code and Codex. The commands below install the
public marketplace release.

### Claude Code

Expand Down Expand Up @@ -97,12 +97,9 @@ In Claude Code, you can also run:
/behavior-diff
```

Behavior Diff finds the changed instruction file and prepares a neutral task.
It shows you the task and the model cost before it starts. The run starts only
after you approve it.

The standard run starts six fresh agent trials: three without the change and
three with it. Fast mode starts one trial for each side.
Behavior Diff finds the changed instruction file and uses your request as the
comparison task. Once the task is known, it runs the comparison and opens the
report.

## Read the report

Expand Down Expand Up @@ -143,17 +140,27 @@ Behavior Diff creates two copies of the same project state:
1. The **before** copy uses the committed instruction file.
2. The **after** copy adds only your uncommitted instruction change.

It gives both copies the same task and starts fresh agent sessions. Standard
mode repeats each side three times. This makes one unusual model response less
likely to control the result.
It gives both copies the same task and starts fresh agent sessions.

The runner records tool calls, commands, evidence, decisions, and final answers.
It converts those traces into a common flow format, compares the two sides, and
builds the HTML report. It does not use a model to declare a winner.

The plugin also watches edits to `CLAUDE.md`, `AGENTS.md`, and `SKILL.md`. After
you finish the current task, it can ask whether you want to run Behavior Diff.
It never starts a model run without your approval.
The plugin also watches edits to `CLAUDE.md`, `AGENTS.md`, and `SKILL.md`.
After you finish the current task, the agent can use that task to run Behavior
Diff on the instruction change.

## Release

1. Update both plugin manifests to the same `X.Y.Z` version.
2. Merge the version change to `main` and wait for CI.
3. Create a GitHub Release with tag `vX.Y.Z`, targeting `main`.
4. Publish it as a stable release, not a prerelease.
5. Confirm the Release workflow pins the marketplace entry to `vX.Y.Z`.

The release workflow rejects tags that do not match both plugin manifests or
do not point to a commit on `main`. Drafts and prereleases do not update the
stable marketplace.

### Repository layout

Expand Down
Loading