Skip to content

ci(port): keep partial work on timeout/abort, label it port-incomplete #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
LukasParke wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

ci(port): keep partial work on timeout/abort, label it port-incomplete #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 109 additions & 20 deletions .github/workflows/upstreamer-port.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ name: Upstreamer Port
#
# Opens a PR. Never pushes to main. A failed parity eval leaves
# .upstreamer/state.yaml unchanged, so the next run retries the same delta.
# A converter that times out or aborts still pushes its partial tree, labelled
# port-incomplete, so the work is inspectable rather than lost with the runner.

on:
repository_dispatch:
Expand All @@ -42,6 +44,11 @@ on:
required: false
default: false
type: boolean
timeout_minutes:
description: "Converter time limit in minutes (max 300). Raise for large upstream ranges."
required: false
default: "120"
type: string

permissions:
contents: write
Expand All @@ -55,7 +62,10 @@ concurrency:
jobs:
port:
runs-on: ubuntu-latest
timeout-minutes: 150
# Must exceed the converter limit (timeout_minutes, max 300) plus setup and
# PR steps. If the JOB times out, nothing after the port step runs and all
# partial work is lost, so the converter's own alarm must always fire first.
timeout-minutes: 330
env:
# Surfaced as env because the `secrets` context is NOT available in a
# step-level `if:` (only github/needs/strategy/matrix/job/runner/env/vars/
Expand Down Expand Up @@ -102,28 +112,49 @@ jobs:
fi
echo "ref=$REF" >> "$GITHUB_OUTPUT"

# A timed-out or aborted converter does NOT fail this step. Its exit code is
# recorded instead, so the partial tree is still committed and pushed to the
# PR below (labelled port-incomplete) rather than discarded with the runner.
# The last step fails the job afterwards.
- name: Run port
id: port
env:
# Provide these in repo settings:
# Secret OPENROUTER_API_KEY — sk-or-... key opencode uses for inference
# Variable OPENCODE_MODEL — e.g. openrouter/~anthropic/claude-opus-latest
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
OPENCODE_MODEL: ${{ vars.OPENCODE_MODEL }}
UPSTREAMER_TIMEOUT_SECONDS: 7200
TIMEOUT_MINUTES: ${{ inputs.timeout_minutes || '120' }}
run: |
set -euo pipefail
if [ -z "${OPENROUTER_API_KEY:-}" ]; then
echo "::error::OPENROUTER_API_KEY secret is not set. See .upstreamer/port.env.example."
exit 1
fi
case "$TIMEOUT_MINUTES" in
''|*[!0-9]*) echo "::error::timeout_minutes must be a whole number, got '$TIMEOUT_MINUTES'"; exit 1 ;;
esac
if [ "$TIMEOUT_MINUTES" -lt 1 ] || [ "$TIMEOUT_MINUTES" -gt 300 ]; then
echo "::error::timeout_minutes must be between 1 and 300 (the job limit is 330), got $TIMEOUT_MINUTES"
exit 1
fi
export UPSTREAMER_TIMEOUT_SECONDS=$((TIMEOUT_MINUTES * 60))
# Only pass --ref when there is actually a ref. `--ref ""` is not the
# same as omitting it: the arg parser consumes the empty value and the
# script would target an empty ref instead of defaulting to HEAD.
args=()
REF="${{ steps.target.outputs.ref }}"
[ -n "$REF" ] && args+=(--ref "$REF")
[ "${{ inputs.force }}" = "true" ] && args+=(--force)
./scripts/upstream ${args[@]+"${args[@]}"}
rc=0
./scripts/upstream ${args[@]+"${args[@]}"} || rc=$?
echo "rc=$rc" >> "$GITHUB_OUTPUT"
case "$rc" in
0) ;;
142) echo "::error::Converter hit the ${TIMEOUT_MINUTES}-minute limit. Partial work will be pushed to a port-incomplete PR. Re-run with a narrower ref or a larger timeout_minutes." ;;
4) echo "::error::Converter aborted (opencode ended on an error but exited 0). Partial work will be pushed to a port-incomplete PR." ;;
*) echo "::error::scripts/upstream exited $rc. Any partial work will be pushed to a port-incomplete PR." ;;
esac

- name: Check for changes
id: diff
Expand All @@ -135,18 +166,50 @@ jobs:
echo "No changes — upstream unchanged or port was a no-op."
fi

# State only advances when the verifier AND the parity eval passed, so an
# unchanged state file next to a changed tree means the port did not pass.
# Label the PR accordingly instead of letting it look green.
- name: Detect eval failure
# Three outcomes, labelled distinctly:
# incomplete — the converter did not finish (timeout, abort, error). The
# eval never ran to completion, so this says nothing about
# parity; the partial tree is pushed for inspection only.
# eval-failed — the converter finished but state.yaml did not advance, so
# the verifier or the parity eval did not pass.
# passed — state.yaml advanced.
- name: Classify outcome
if: steps.diff.outputs.changed == 'true'
id: gate
env:
PORT_RC: ${{ steps.port.outputs.rc }}
run: |
if git diff --quiet -- .upstreamer/state.yaml; then
echo "passed=false" >> "$GITHUB_OUTPUT"
set -euo pipefail
base="port: sync with @openrouter/agent upstream"
if [ "${PORT_RC:-1}" != "0" ]; then
# An unfinished run must never advance the port pointer, even if the
# converter happened to write state.yaml before it died.
git checkout -- .upstreamer/state.yaml
{
echo "status=incomplete"
echo "passed=false"
echo "title=$base (INCOMPLETE — converter did not finish, do not merge)"
echo "commit=port: partial sync with @openrouter/agent upstream (incomplete)"
echo "labels=upstreamer, automated, port-incomplete"
} >> "$GITHUB_OUTPUT"
echo "::warning::Converter exited $PORT_RC — pushing partial work as port-incomplete."
elif git diff --quiet -- .upstreamer/state.yaml; then
{
echo "status=eval-failed"
echo "passed=false"
echo "title=$base (EVAL FAILED — do not merge)"
echo "commit=$base"
echo "labels=upstreamer, automated, eval-failed"
} >> "$GITHUB_OUTPUT"
echo "::warning::state.yaml did not advance — parity eval did not pass. See .upstreamer/eval-report.md."
else
echo "passed=true" >> "$GITHUB_OUTPUT"
{
echo "status=passed"
echo "passed=true"
echo "title=$base"
echo "commit=$base"
echo "labels=upstreamer, automated"
} >> "$GITHUB_OUTPUT"
fi

# Mint a GitHub App installation token so the PR is created by the App
Expand Down Expand Up @@ -193,23 +256,23 @@ jobs:
token: ${{ steps.app-token.outputs.token || secrets.GITHUB_TOKEN }}
branch: upstreamer/sync
delete-branch: true
title: >-
${{ steps.gate.outputs.passed == 'true'
&& 'port: sync with @openrouter/agent upstream'
|| 'port: sync with @openrouter/agent upstream (EVAL FAILED — do not merge)' }}
commit-message: "port: sync with @openrouter/agent upstream"
labels: >-
${{ steps.gate.outputs.passed == 'true'
&& 'upstreamer, automated'
|| 'upstreamer, automated, eval-failed' }}
title: ${{ steps.gate.outputs.title }}
commit-message: ${{ steps.gate.outputs.commit }}
labels: ${{ steps.gate.outputs.labels }}
body: |
Automated Upstreamer port of `@openrouter/agent` into this repo.

- Outcome: **${{ steps.gate.outputs.status }}** (converter exit code `${{ steps.port.outputs.rc }}`)
- Run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} (logs in the `upstreamer-logs` artifact)
- Contract: `.upstreamer/upstreamer.md`
- Run log: `.upstreamer/logs/`
- Parity eval: `.upstreamer/eval-report.md`
- Parity eval passed: **${{ steps.gate.outputs.passed }}**

`port-incomplete` means the converter was cut off (time limit, abort, or
error) before it finished, so this tree is partial work and the eval
result above is not meaningful. `eval-failed` means the converter
finished but the verifier or the parity eval did not pass.

Review the diff as a port, not as a normal PR: check behavioral parity
against the TypeScript reference, not just that it compiles. If
`.upstreamer/state.yaml` did not advance, the eval did not pass and this
Expand All @@ -233,10 +296,36 @@ jobs:
echo "::warning::PORT_BOT_APP_ID / PORT_BOT_PRIVATE_KEY are not configured, so this PR was opened with GITHUB_TOKEN and will receive NO pull_request-event checks. main's required status checks cannot be satisfied, so the PR cannot merge as-is. The dispatched run below is informational only. Configure the App (see PORTING.md) or close and reopen the PR by hand to generate real checks."
gh workflow run ci.yaml --repo "$GITHUB_REPOSITORY" --ref upstreamer/sync

# create-pull-request only ever ADDS labels, so a PR reused across runs
# would keep a stale outcome label. Drop the ones that no longer apply.
- name: Clear stale outcome labels
if: >-
steps.diff.outputs.changed == 'true'
&& steps.open-pr.outputs.pull-request-number != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR: ${{ steps.open-pr.outputs.pull-request-number }}
STATUS: ${{ steps.gate.outputs.status }}
run: |
for label in eval-failed port-incomplete; do
[ "$label" = "$STATUS" ] && continue
gh pr edit "$PR" --repo "$GITHUB_REPOSITORY" --remove-label "$label" 2>/dev/null || true
done

- name: Upload logs
if: always()
uses: actions/upload-artifact@v4
with:
name: upstreamer-logs
path: .upstreamer/logs/
if-no-files-found: ignore

# The port step records a converter failure instead of failing, so that the
# partial work above still gets pushed. Fail the job here so the run is red.
- name: Fail if the converter did not finish
if: steps.port.outputs.rc != '0'
env:
PORT_RC: ${{ steps.port.outputs.rc }}
run: |
echo "::error::Converter did not finish (exit $PORT_RC). See the port-incomplete PR, if any, and the upstreamer-logs artifact."
exit 1
8 changes: 8 additions & 0 deletions PORTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ correct outcome for a failed port — it makes the next run retry the same delta
instead of skipping past the gap. The workflow labels such a PR `eval-failed` and
marks the title `do not merge`.

A run that is cut off before it finishes (the `timeout_minutes` limit, default
120, or an opencode abort) is different: the eval never completed, so it says
nothing about parity. The workflow still commits and pushes the partial tree so
the work is not lost, but labels the PR `port-incomplete`, never advances
`state.yaml`, and fails the run. For a large upstream range, dispatch the
workflow with an intermediate `ref` (one big upstream PR at a time) or a larger
`timeout_minutes` (max 300).

Never hand-edit `state.yaml` to make a run look successful.

## Running it locally
Expand Down
40 changes: 35 additions & 5 deletions scripts/upstream
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,9 +227,39 @@ prompt="$(cat "$prompt_file")"

echo "Writing opencode log to $log_file" >&2

if [ -n "${UPSTREAMER_TIMEOUT_SECONDS:-}" ]; then
perl -e 'alarm shift; exec @ARGV' "$UPSTREAMER_TIMEOUT_SECONDS" \
opencode "${model_args[@]}" "$@" run "$prompt" 2>&1 | tee "$log_file"
else
opencode "${model_args[@]}" "$@" run "$prompt" 2>&1 | tee "$log_file"
run_opencode() {
if [ -n "${UPSTREAMER_TIMEOUT_SECONDS:-}" ]; then
perl -e 'alarm shift; exec @ARGV' "$UPSTREAMER_TIMEOUT_SECONDS" \
opencode "${model_args[@]}" "$@" run "$prompt"
else
opencode "${model_args[@]}" "$@" run "$prompt"
fi
}

# Capture the converter's status instead of letting `set -e` exit on it, so the
# outcome can be reported precisely. Exit codes the workflow relies on:
# 0 converter finished (the eval may still have failed; state.yaml says)
# 142 UPSTREAMER_TIMEOUT_SECONDS elapsed (SIGALRM); the tree holds partial work
# 4 opencode exited 0 but the run ended on an error, e.g. "Tool execution
# aborted" (seen 2026-10-05); the tree holds partial work
# * any other opencode failure, passed through
rc=0
run_opencode "$@" 2>&1 | tee "$log_file" || rc="${PIPESTATUS[0]}"

if [ "$rc" -eq 142 ]; then
echo "ERROR: converter timed out after ${UPSTREAMER_TIMEOUT_SECONDS}s; the port is INCOMPLETE." >&2
exit 142
elif [ "$rc" -ne 0 ]; then
echo "ERROR: opencode exited $rc; the port is INCOMPLETE." >&2
exit "$rc"
fi

# opencode exits 0 even when the session dies on an aborted or failed tool call.
# Mid-run tool errors the agent recovers from (a failed edit it retries) are
# normal, so only a run whose LAST output line is an error counts as aborted.
last_line="$(sed -e 's/\x1b\[[0-9;]*m//g' "$log_file" | awk 'NF { line = $0 } END { print line }')"
case "$last_line" in
Error:*)
echo "ERROR: opencode exited 0 but the run ended on an error ($last_line); the port is INCOMPLETE." >&2
exit 4 ;;
esac
Loading