Repository navigation
Conversation
Add built-in harness_profile.v1 files codex, codex-review, copilot and copilot-review, so repo run and repo reach can compare guidance arms under codex exec, codex exec review --commit HEAD, copilot -p and Copilot's /review agent, not only Claude Code. - trace: codex-json and copilot-json convert the CLIs' JSON event streams to events.jsonl (cli_traces.py); Codex read events come from shell commands that only print files. - native_prompt: a profile may run the harness's own prompt; reach refuses it. - answer_format: codex-review appends a findings block parsed from Codex's review text (answers.py), so review graders read it unchanged. - bin_companions: helper files mounted beside the executable (Codex's codex-code-mode-host, without which no tool call runs in the container). - repo reach now writes the profile's config_files, as repo run does. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #116 +/- ##
==========================================
+ Coverage 91.64% 91.65% +0.01%
==========================================
Files 222 226 +4
Lines 27409 27990 +581
Branches 4508 4635 +127
==========================================
+ Hits 25118 25654 +536
- Misses 1620 1645 +25
- Partials 671 691 +20
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟡 Changes recommended
Trace conversion can distort evaluation results, and reach configuration produces invalid JSON on Windows.
4 open issues
What changed in this PR
Adds Codex and Copilot CLI harnesses so repository evaluations can compare guidance across agents, including native review modes.
Changes:
- Adds four harness profiles and optional profile settings.
- Converts CLI traces and native review answers into evaluation artifacts.
- Applies profile configuration during reach probes, with tests and documentation.
| File | Description |
|---|---|
| dokimi/tests/test_repo_reach.py | Tests reach configuration generation. |
| dokimi/tests/test_repo_cli_harnesses.py | Tests profiles, converters, and runner integration. |
| dokimi/dokimi/schemas/harness_profile.v1.json | Adds optional harness capabilities. |
| dokimi/dokimi/repos/runner.py | Mounts companion binaries and converts answers. |
| dokimi/dokimi/repos/reach.py | Writes configuration and rejects native prompts. |
| dokimi/dokimi/repos/profiles/copilot.yaml | Adds Copilot execution profile. |
| dokimi/dokimi/repos/profiles/copilot-review.yaml | Adds Copilot review profile. |
| dokimi/dokimi/repos/profiles/codex.yaml | Adds Codex execution profile. |
| dokimi/dokimi/repos/profiles/codex-review.yaml | Adds Codex native review profile. |
| dokimi/dokimi/repos/harness_profile.py | Parses and validates new settings. |
| dokimi/dokimi/repos/events.py | Dispatches new trace formats. |
| dokimi/dokimi/repos/cli_traces.py | Normalizes Codex and Copilot events. |
| dokimi/dokimi/repos/answers.py | Converts native review findings. |
| dokimi/docs/repo-evaluation-contract.md | Documents profiles and evaluation behavior. |
| dokimi/docs/architecture.md | Describes new modules and harnesses. |
| dokimi/changelog.d/20261019_050000_nikolay.a.petrov_codex_copilot_profiles.md | Records harness additions and reach fix. |
Review effort: Balanced
…d, cd commands, and incomplete commands On Windows, str(Path) contains backslashes that produce invalid JSON escape sequences when inserted into configuration files. Use Path.as_posix() for both config_dir and support_dir to ensure forward slashes. shell_reads() now splits only on unquoted shell operators (&&, ||, ;, newline), preserving quoted arguments like sed -n '1p;2p'. Previously, it split all semicolons regardless of quoting context, breaking multi-range sed commands. Track directory changes from cd commands in shell scripts so subsequent file reads are resolved to the correct path. For example, 'cd sub && cat AGENTS.md' now correctly reports reading 'sub/AGENTS.md'. Preserve incomplete commands (item.started without item.completed) in Codex JSON traces. When an agent times out mid-command, record the command with rc: null per contract §4.1, instead of losing it entirely. Correlate started and completed items by command text to update rc on completion. Add regression tests for all three fixes. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Create tests/__init__.py and normalize test module imports to use explicit 'tests.' prefix. This resolves the mypy error where artifact_fixtures was found under two module names. All imports of test fixtures now consistently use the 'tests.' prefix (repo_eval_fixture, artifact_fixtures, test_process). Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Fix remaining imports from conftest, test_novelty_analyze, and test_repo_reach to use explicit 'tests.' prefix for consistency now that tests is a proper package. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Fix remaining import from test_novelty_gate in test_novelty_items.py to use explicit 'tests.' prefix for consistency. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
There was a problem hiding this comment.
🟡 Changes recommended
Overlapping identical Codex commands produce incorrect trace events used by graders.
7 open issues
Review effort: Balanced
…ay uv.lock Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
Review at 27911d6. The profiles, schema widening and mounting logic hold up, but the trace converters have several wrong-result cases, and the Codex findings parser is fragile. Four things are worth fixing before merge; the inline comments have the inputs and outputs.
What I ran, on a Linux review host at 27911d6:
make ci-local: 19 passed, 0 failed, 4 skipped (docker/semgrep/jobs that do not run on a PR); changed-line coverage gate passes.- Reproductions for the code findings, with no model or network calls.
- A mutation run over the new tests: 36 of 83 mutants of
cli_traces.pypass the suite; two of the severity-mapping mutants inanswers.pypass too.
I agree with Copilot that Codex commands are matched by text, not item.id; it still holds (two overlapping make build items give rc [None, 0, 1]).
Checked, no issues
- Schema: the
harness_profile.v1.jsonchange only widens thetraceenum and adds optional keys, so no version bump is needed;schema_compatpasses against the base;bin_companionsnames reject..and/. - Docs: flags, keys, section numbers and the profile table match the code; the changelog fragment is present; no
tests/__init__.pyoruv.lockremains after the reverts. - Mounts and reach: a missing companion fails with a clear error, a symlinked
--binfinds the companion beside the real file, andrepo reachrefusesnative_promptprofiles before--dry-run.
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
Review at ba0b4a3. The fixes since 1f8d127 hold: the brace-group cd, the _LONG_VALUE table, the token sum and the severity mapping all give the right result on the cases from my threads. Three points are left, two worth fixing before merge and one nit; the inline comments have the details.
What I ran, on a Linux review host at ba0b4a3:
make ci-local: 26 passed, 0 failed, 4 skipped (docker, semgrep and jobs that do not run on a PR); 6,481 tests passed, 1 skipped; changed-line coverage 94% (490 lines).- Reproductions for the code findings, with no model or network calls.
- A mutation run over the code and tests changed since 1f8d127 (5 mutants; 1 passes
test_repo_cli_harnesses.py).
Checked, no issues
- Brace groups:
{ cd x; } & cat f,{ cd x; } 2>&1 | cat; cat fand{ cd x; cat f; } & cat ggivef,fandx/f,g. - Long options: every
_LONG_VALUEentry run as<cmd> <option> 5 agives['a']. - Tests:
failed = not result.okfails the exit-0 failed-review test; droppingscenario.cwdfrom theconvert()call inreach.pyfails the newcwd=subtest; replacing the token sum with an assignment (input or output) failstest_codex_usage_is_summed_and_a_missing_or_zero_usage_is_not_a_measured_zero. - Docs: the contract table,
architecture.mdandcopilot-review.yamlagree thatcopilot-reviewis a prompt variant; the schema change only widens,schema_compatpasses, and the changelog fragment is present.
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
I pushed a commit per nit, up to f038189, which fix the nit from my review:
f038189:dokimi/changelog.d/20261019_050000_nikolay.a.petrov_codex_copilot_profiles.md:3: replaced "Copilot's/reviewagent" with "copilot -pwith a/review-prefixed prompt".
make ci-local at f038189: 26 passed, 0 failed, 4 skipped.
Not approving yet. Still open: 2 risk finding(s) from my review.
…l.parent Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
@exai-bot /review |
exai-bot
left a comment
exai-bot
left a comment
There was a problem hiding this comment.
Note
AI-generated review of 31719ab3f2cc, requested by @vchamarthi, using the default review skill. It can be wrong: verify a finding before acting on it. React with a thumbs up or down on a finding to tell the bot whether it helped.
Model us.anthropic.claude-sonnet-5-5, effort default, 19 of 94 turns, 1.8 of 48 minutes, 794k in / 12k out tokens.
This PR adds Codex and Copilot CLI harness profiles, including a native-review variant for each. It also adds two JSON trace converters, an answer_format step that appends a findings block for Codex review text, and bin_companions mounting. repo reach now writes a profile's config_files. The change is careful: format drift in review text suppresses the findings block rather than reading as a clean review, and a missing companion binary stops the cell before any agent runs. I found one minor issue, in how shell activity is read as evidence, and one nit. I could not run anything.
2 findings are on the changed lines, as comments below.
For a human reviewer
- The PR describes
copilot-reviewas Copilot's native review mode. The profile only sends/review <task prompt>as plain text under-pand does not setnative_prompt. Check that the docs and the comparison framing say so. - The
Tests (py3.13)check failed. The author attributes this to unrelated flaky tests under xdist. Confirm with a re-run.
Not verified
I ran nothing. The real Codex and Copilot output shapes, the codex exec review behaviour, the shell-parser quoting and heredoc corner cases, and the reach table all rest on the author's live runs and tests. I also read only the first 20,000 characters of tests/test_repo_cli_harnesses.py.
| def unwrap_shell(command: str) -> str: | ||
| """The script of ``/usr/bin/bash -lc '<script>'`` (how Codex reports a command), else ``command``.""" | ||
| m = _BASH_WRAP.match(command.strip()) | ||
| if not m: |
There was a problem hiding this comment.
Minor (plausible, correctness): Shell-derived read events miss piped/grep reads and count failed reads
- Problem:
shell_readsrecords a read only for a barecat/head/tail/nl/sed -nwith no pipe or redirect.cat AGENTS.md | head -50,grep -n . AGENTS.md,rg x AGENTS.md,python -c ...andawkproduce noreadevent. - Problem:
codex_jsonalso emits the derivedreadevents atitem.started, before the exit code is known. A failedcat missing.mdtherefore still counts as a read. - Why it matters:
repo reachclassifytellsfetched(read by a tool) fromloaded(auto-loaded) usingtranscript.reads(). A Codex or Copilot run that fetched a subtree AGENTS.md through a pipe orgrepis classedloaded, which is the distinction the probe exists to make. Codex'snonetool mode still leaves shell access, per the profile's own comments, so those reads are possible in reach scenarios. - Fix: document the limit for these profiles next to the reach matrix. Optionally skip derived reads when the completed command has a non-zero
exit_code.
Suggested fix:
Defer derived read events to item.completed and skip them when exit_code != 0. For reach, consider treating any shell command that names a probed path as a read.
There was a problem hiding this comment.
Fixed in bc66cd0. Verified: a derived read was emitted at item.started, so a failed 'cat missing.md' counted. Derived reads are now dropped when the command completes with a non-zero exit_code (test_codex_failed_command_does_not_count_as_a_read fails on the old code). Pipe/grep/rg reads remain unseen on purpose; the limit is now documented next to the fetched class in the reach section of repo-evaluation-contract.md.
| import re | ||
| from collections.abc import Callable | ||
| from typing import Any | ||
|
|
There was a problem hiding this comment.
Nit (plausible, maintainability): Answer-format list is repeated in the schema and in code
ANSWER_FORMATS, _CONVERTERS and the answer_format enum in harness_profile.v1.json each list the formats. Adding a format to one but not the others fails late, after the agent has run, as a ValueError that run_cell records as a host-error failure. A small test that the three lists agree would catch this.
There was a problem hiding this comment.
Added in bc66cd0: test_answer_format_lists_agree asserts ANSWER_FORMATS, _CONVERTERS and the schema enum match, so drift fails in CI instead of after an agent run.
…sts agree Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
On the exai-bot review question about |
…l files cat sub/AGENTS.md sub/CLAUDE.md prints AGENTS.md and exits 1; dropping both reads made reach class a fetched file as loaded. Only a one-part command naming one file is dropped. Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
|
Re-ran at bc66cd0, on Linux, the original inputs of every earlier thread, plus real codex-cli 0.163 and Copilot CLI 1.0.100 against a local stub, and the full suite with ruff, mypy and bandit. Still open:
Pushed:
Left for the author: the comment wording above. |
|
On the codex.yaml / codex-review.yaml wording (xaleryb): fixed in c1c400e. Both profile comments and the contract's Codex routing paragraph now say that |
…ith env_key Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
Review at c1c400e. Both open threads are fixed, and the new _failed_read logic gives the right result on the cases I tried. Two points are left: one test gap worth closing before merge, and one stale doc sentence.
What I ran, on a Linux review host at c1c400e:
make ci-local: 25 passed, 1 failed, 6 skipped; 6,811 tests passed, 1 skipped; changed-line coverage 94% (502 lines). The one failure ispip-audit(virtualenv 21.14.7, CVE-2026-113722, fix in 21.14.9). The PR does not touch the lock files, so it comes from the base, not from this PR;mainfixed it in #152.- Reproductions for the code findings, with no model or network calls.
- A mutation run over the code and tests changed since f038189 (
_failed_read, theharness_mountscompanion lookup, theenv_keydoc test, the format-list test).
Findings that do not anchor inline:
- Nit:
docs/preparing-a-repository.md:207-210still says onlyclaude-codeis built in. The sentence reads "Only theclaude-codeprofile is built in; for Codex or Copilot, pass aharness_profile.v1file" and "tool-use counts ... today meansclaude-stream-json". The PR adds built-incodex,codex-review,copilotandcopilot-reviewprofiles and thecodex-json/copilot-jsontraces; the contract andarchitecture.mdsay so. A repository owner following this page writes a profile that already exists. Suggestion: say thatclaude-code,codex,codex-review,copilotandcopilot-revieware built in, and that the convertible traces areclaude-stream-json,codex-jsonandcopilot-json. (outside this diff)
Checked, no issues
- Failed reads:
cat awith rc 1 drops its read on both the started and the no-started path;cat a missing,cd x && cat f,cat a bandcat a; cat akeep theirs. Droppinglen(reads) == 1, dropping the one-part check, or removing either branch's check each failstest_codex_failed_command_does_not_count_as_a_reador..._keeps_the_reads_it_may_have_made. - Companion lookup: replacing
real.parentwithopts.harness_bin.parentinharness_mountsfailstest_harness_mounts_takes_the_companion_from_beside_the_link_target. - Profile docs:
env_key = "CODEX_API_KEY"is in both Codex profiles and in the contract; deleting it from any of the three fails the new test. The comment that nothing readsendpoint_envis right (grepfinds only the parser). - Format lists: extending
ANSWER_FORMATSor the schema enum alone failstest_answer_format_lists_agree. - Schema and merge: the schema change only widens;
schema_compatagainst the base passes;ruffandmypyare clean onrepos/; the changelog fragment is present and does not clash with main's fragments.
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
I pushed a commit per nit, up to a22d109, which fix the nit from my review:
a22d109:dokimi/docs/preparing-a-repository.md:207: lists the built-inclaude-code,codex,codex-review,copilotandcopilot-reviewprofiles and theclaude-stream-json,codex-jsonandcopilot-jsontraces.
make ci-local at a22d109: 25 passed, 1 failed (pip-audit, from the base), 6 skipped.
Not approving yet. Still open: the risk from my review.
…s read Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…ers and parse_profile Behavior-preserving; the new functions are back under the complexity cap and parse_profile is back at its baseline. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
The refactor and the new test hold, and the open thread is closed. Two test gaps are worth closing before merge, and there is one nit.
What I ran, on a Linux review host at 0100221:
make ci-local: 28 passed, 1 failed, 7 skipped; 7,345 tests passed, 1 skipped; changed-line coverage 95% (555 lines). The one failure ispip-audit(gitpython 3.2.0, CVE-2026-114145, fix in 3.2.1). The PR does not touch the lock files, so it comes from the base.- A differential run of
adfe925and0100221againsta22d109: 132,893 inputs (random Codex and Copilot streams, generated shell scripts throughshell_reads,_split_scriptand_sed_files, andparse_profileinputs) give identical transcripts, events andProfileErrormessages. The refactor changes no behaviour. - A mutation run over the code touched by
adfe925(55 mutants; 23 passtest_repo_cli_harnesses.py,test_repo_reach.pyandtest_repo_reach_edges.py). The two that matter are inline.
Findings that do not anchor inline:
- Nit:
docs/preparing-a-repository.md:270still says onlyclaude-codeis built in. The checklist item reads "dokimi repo reach(onlyclaude-codeis built in; others need aharness_profile.v1file)". The PR now shipscodex,codex-review,copilotandcopilot-review, and line 208 says so. Lines 193-195 say "no loading behavior was measured" for Codex and Copilot, while the profiles'guidance.notesrecord measurements. Suggestion: change the checklist item to "dokimi repo reach(claude-code,codex,copilotandcopilot-revieware built in;codex-reviewruns its own prompt and is refused; others need aharness_profile.v1file)". (outside this diff)
Checked, no issues
- Thread on the unknown exit code:
rc not in (None, 0)changed torc != 0fails both parametrizations oftest_codex_command_with_an_unknown_exit_code_keeps_its_read(2 failed). - Refactor:
_Splitter,_Walk,_SedScan, theby_idpop and the CopilotstartNone check behave as before.ruffis clean onrepos/;mypyreports nothing underrepos/. - Profiles and schema: all five built-in profiles load and build argv for
none,readandall; the schema change only widens;schema_compatpasses; the changelog fragment is present; no new doc page, so no nav change.
| event["output"], event["output_truncated"] = clip_output(str(item["aggregated_output"])) | ||
| derived = event_reads.get(id(event), []) | ||
| if _failed_read(str(event["text"]), derived, event["rc"]): # drop the read events derived from it | ||
| dropped = {id(e) for e in derived} |
There was a problem hiding this comment.
No test pins that a failed read drops only its own read events.
_codex_command_finished removes the reads derived from the failed command (dropped = {id(e) for e in derived}). Mutation check: changing the line to dropped = {id(e) for e in out.events if e["kind"] == "read"} removes every read in the trace, and the three files test_repo_cli_harnesses.py, test_repo_reach.py and test_repo_reach_edges.py stay green (225 passed).
Case it lets through: the agent runs cat AGENTS.md (exit 0), then cat missing.md (exit 1). With the mutant the trace has no read of AGENTS.md, so repo reach classes the file loaded instead of fetched. The failed-read tests in this file (test_codex_failed_command_does_not_count_as_a_read and its neighbours) never put a successful read before the failing command.
Test to add: an item.started / item.completed pair for cat AGENTS.md with exit_code 0, then one for cat missing.md with exit_code 1; assert the reads are ["AGENTS.md"].
There was a problem hiding this comment.
Added in d479a0f: test_codex_failed_command_drops_only_its_own_reads (test_repo_cli_harnesses.py). cat AGENTS.md exits 0, then cat missing.md exits 1; the reads stay ["AGENTS.md"] and the rcs are [0, 1]. It fails with the dropped set widened to every read event.
| continue | ||
| shell = data.get("shellExecution") or {} | ||
| code = shell.get("exitCode") if isinstance(shell, dict) else None | ||
| event["rc"] = code if isinstance(code, int) else (None if data.get("success") is None |
There was a problem hiding this comment.
The success fallback for a Copilot command's exit code is not tested.
When shellExecution.exitCode is absent, rc comes from data["success"]. Every tool.execution_complete in test_repo_cli_harnesses.py carries shellExecution.exitCode (lines 351-352), so the fallback never runs. Mutation check: swapping the branches (1 if data.get("success") else 0) and, separately, returning 0 for a missing success both leave the three files green (225 passed). The first inverts the result: a successful command is recorded as rc: 1, a failed one as rc: 0.
Test to add: one tool.execution_complete with "success": true and no shellExecution, one with "success": false, and one with neither, asserting rc 0, 1 and None.
Related, at _CopilotRun.apply: rec.get("exitCode") not in (0, None) changed to != 0 also survives. This is the Copilot twin of the None case that test_codex_command_with_an_unknown_exit_code_keeps_its_read now pins for Codex. A result record without exitCode would set is_error and no test fails. Add a {"type": "result"} record with no exitCode and assert is_error is false.
There was a problem hiding this comment.
Added in d479a0f: test_copilot_command_exit_code_falls_back_to_success (success true gives rc 0, success false gives rc 1, neither gives None) and test_copilot_result_without_an_exit_code_is_not_an_error (a result record with no exitCode leaves is_error false). They fail with the swapped success branches and with exitCode != 0 respectively.
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
|
Re-checked at 0100221 on Linux: the original inputs of every earlier point (ours and every open thread) now pass, including real codex-cli 0.163 and Copilot CLI 1.0.100 against a local stub; adfe925 is behaviour-preserving on a 60k-input differential run against 9667bb4; the full suite, ruff, mypy and bandit are clean at the head and at the head merged into main. |
antonwolfy
left a comment
antonwolfy
left a comment
There was a problem hiding this comment.
I pushed a commit per nit, up to 25dd17d, which fix the nit from my review:
25dd17d:dokimi/docs/preparing-a-repository.md:270: the checklist item now listsclaude-code,codex,copilotandcopilot-reviewas built in and sayscodex-reviewis refused.
make ci-local at 25dd17d: 28 passed, 1 failed, 7 skipped.
Not approving yet. Still open: the two risks from my review.
|
Review-body items from 1f8d127 and b4457f9: contract §9 now names bin_companions beside the mounted executable (d479a0f); token accumulation over several turn.completed is pinned by test_codex_usage_is_summed_and_a_missing_or_zero_usage_is_not_a_measured_zero (earlier commit); the duplicated SUPPORT_HOST and the _PRINTERS spacing are already fixed at the head (reach.py imports SUPPORT_HOST from runner.py). |
… fallback; name bin_companions in contract §9 Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Summary
repo runandrepo reachhad one built-in harness, Claude Code. This adds the Codex CLI and the Copilot CLI, including each CLI's own review mode, so the same task can be compared across guidance arms × harness × model.harness_profile.v1is widened in place with three optional keys:Things that only show up when running the real CLIs:
codex review --commitrefuses a prompt.codex-code-mode-hostbeside its binary, so a single-file mount leaves it with no tools.--available-toolslist leaves every tool on, sononenames a tool that doesn't exist.--binhas to name the native binary.Not here: hosted PR reviewers (Copilot code review,
@codex review, CodeRabbit). They are not CLIs and need a separate PR-replay adapter.Evidence
Before:
repo run --harness codexfails withno harness profile 'codex' (built-in: claude-code). A profile with a Codex provider inconfig_filesprobed viarepo reachhitwss://api.openai.comwith a 401, because reach never wrote the file.After: live runs on oneDAL
review_seeded_fc(forkdev/agent-evalsc4e2667f5, imageagent-benchmark/onedal:a28e8a6c33a7, gpt-6-luna through a LiteLLM proxy,guidance:raw, 1 rep):The first codex-review run, without
bin_companions, answered "the repository command tool failed to start" and got recall 0.0.Live
repo reachon oneDAL main 7f6d9f86 (3 scenarios, gpt-6-luna):Both CLIs load AGENTS.md from the git root down to cwd, and reading a file in a subtree does not pull in that subtree's AGENTS.md. In
read=Codex skipped the read, because the reach ask says "use the Read tool" and Codex has no tool by that name. Rewording the ask is left for a follow-up: it would change Claude's recorded ask too.Tests:
tests/test_repo_cli_harnesses.py(29 tests) covers:shell_reads, including pipes, redirects andsed -i;codex_review_findings;repo runcell that grades a native review answer;bin_companionsmounted, or a stop before any agent runs.test_repo_reach.pyadds a test that a profile'sconfig_filesreach the probe.make ci-local: every row passes excepttest. There, every PR-touched test passes, but 1–3 unrelated tests fail at random, a different one each run:test_processPopen ResourceWarning,test_container_agent"AF_UNIX path too long". This happens under 224 xdist workers on a shared box.pytest -n 32gives 5208 passed, 2 skipped. Those two files alone: 149 passed.Merge Danger
Door: two-way
New profiles and optional schema keys only. Profiles that already exist are left as they were.
schema_compatpasses against the base.Blast Radius:
repo reachEvery reach run now writes the profile's
config_files. Forclaude-codethat adds the no-background hooksettings.jsonto each scenario's config dir. Reach scenarios run with toolsnone/read, so the Bash hook never fires. Onlyrepo run/repo reachread the new keys.🤖 Generated with Claude Code