Skip to content

fix(review): isolate review.profile from consensus prompts#1391

Closed
JSONbored wants to merge 3 commits into
mainfrom
codex/propose-fix-for-review.profile-logic-bug
Closed

fix(review): isolate review.profile from consensus prompts#1391
JSONbored wants to merge 3 commits into
mainfrom
codex/propose-fix-for-review.profile-logic-bug

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • A recent change threaded .gittensory.yml review.profile into the single system prompt used for both advisory write-ups and block-mode consensus reviewers, allowing a maintainer presentation knob to influence gate-producing consensus outputs.
  • review.profile is intended to be PRESENTATION ONLY and must never affect the consensus-defect path that the gate consumes.

Description

  • Split system-prompt construction by adding an options.includeProfile flag to buildSystemPrompt so callers can opt out of appending the review.profile suffix.
  • Compute two prompts in runGittensoryAiReview: advisorySystem (includes review.profile) and consensusSystem (explicitly excludes the profile), and route advisory/BYOK calls to advisorySystem and block-mode Workers-AI consensus calls to consensusSystem.
  • Adjust neuron estimation to account for the separate advisory/consensus prompts so grounding/path context is still billed without letting review.profile change consensus accounting.
  • Add a regression unit test that proves an assertive profile affects the advisory prompt but not the two block-mode consensus prompts.

Testing

  • Ran npx vitest run test/unit/ai-review.test.ts --reporter=verbose, which passed (1 file, 41 tests, all green) including the new regression test.
  • Ran npm run typecheck, which completed successfully with no type errors.
  • Ran git diff --check to ensure no whitespace/conflict markers; it passed.

Codex Task

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jun 25, 2026
@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jun 26, 2026
@JSONbored JSONbored added the gittensor:bug Gittensor-scored bug fix - worth 0.5x multiplier. label Jun 26, 2026
review.profile is a presentation-only tone knob and must never influence
the consensus-defect path the gate consumes. Split buildSystemPrompt with
an options.includeProfile flag and compute two prompts in
runGittensoryAiReview: advisorySystem (with profile) and consensusSystem
(profile-free). Route the advisory/BYOK leg to advisorySystem and the
block-mode Workers-AI consensus reviewers to consensusSystem; when a
profile diverges the prompts, do not reuse the profile-tainted advisory
review as a consensus opinion (run a fresh profile-free leg instead).
Bill the neuron estimate against each effective prompt. Add a regression
test proving an assertive profile reaches the advisory prompt but neither
block-mode consensus prompt.
@JSONbored JSONbored force-pushed the codex/propose-fix-for-review.profile-logic-bug branch from c428610 to fe77d61 Compare June 26, 2026 21:01
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Jun 26, 2026
@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.38%. Comparing base (9e1c351) to head (c80f326).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1391   +/-   ##
=======================================
  Coverage   95.38%   95.38%           
=======================================
  Files         201      201           
  Lines       21598    21603    +5     
  Branches     7807     7810    +3     
=======================================
+ Hits        20601    20606    +5     
  Misses        416      416           
  Partials      581      581           
Files with missing lines Coverage Δ
src/services/ai-review.ts 98.58% <100.00%> (+0.02%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb

gittensory-orb Bot commented Jun 27, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review — held for maintainer review

2 files · 1 AI reviewers · no blockers · readiness 48/100 · CI green · blocked

⏸️ Held for maintainer review

Signal Result Evidence
Code review ✅ No blockers 1 reviewers, synthesized
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ⚠️ 3 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Review load ❌ 8/20 Readiness component derived from cached public PR metadata and labels; size label size:M.
Validation evidence ❌ 5/25 Cached preflight status is hold.
Open PR queue ❌ 3/10 48 open PR(s), 9 likely reviewable, 39 unlinked.
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 81 PR(s), 261 issue(s).
Gate result ✅ Passing No configured blocker found.
Nits — 2 non-blocking
  • Repository config was not parsed
  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 81 PR(s), 261 issue(s).
  • Related work: Titles/paths share 6 meaningful terms. (PR #1392)
  • Related work: Titles/paths share 6 meaningful terms. (PR #1396)
  • Related work: Titles/paths share 6 meaningful terms. (PR #1398)
  • Additional title-only matches omitted; title-only overlap does not block.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Review top overlaps.
  • Add scope summary.
  • Fix blocker.
  • Expect slower review.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Review load = cached public PR metadata such as size labels, changed paths, and preflight status.
  • Open PR queue = repo-wide review pressure; it is not a PR quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@JSONbored

Copy link
Copy Markdown
Owner Author

Closing — this guards a path the prompt already prevents. The assertive profile raises extra items as nits (ai-review.ts:51,55: "defensive or speculative hardening … is a NIT, not a blocker"), and the consensus gate only fires when both reviewers populate blockers[] (ai-review.ts:672, consensusDefectOf returns null otherwise). So review.profile can't leak into the consensus verdict. The fix also abandons the #1347 advisory-leg reuse (it pays for an extra free Workers-AI consensus call whenever a profile is set), and the regression test fabricates a model that promotes a nit into blockers[], which the prompt explicitly forbids. Net cost, no real protection.

@JSONbored JSONbored closed this Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aardvark codex gittensor:bug Gittensor-scored bug fix - worth 0.5x multiplier. gittensor Gittensor contributor context size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant