Repository navigation
Add clause-dash advisory, judge calibration and frozen holdout cases - #55
Conversation
…e and frozen holdout cases - laconic-check: advisory for clause dashes in 2 or more sentences of a paragraph (never fails lint), calibrated on the Style Manual (2 advisories); stock phrases 'innovative', 'practical solutions' and timeless openers. - judge.py --calibrate with 10 hand-labelled responses; 10 of 10 agreed on 1 November 2026 (claude-opus-5-5). - Prose skill: fiction keeps figurative language; social post venue and regression case. - README limits now say the cases guided rule wording; 10 frozen holdout cases.
|
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 36 |
| Duplication | 0 |
AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Pull Request Overview
The calibration workflow can still abort unexpectedly when the judge returns no rows, because it indexes the first result without handling an empty response. This should be addressed before merging.
The advisory-only CLI exit-code contract and mirrored skill consistency lack automated tests, leaving two important acceptance areas unverified. The judging entry point also has elevated complexity, which increases maintenance risk but is not itself blocking.
About this PR
- Add automated coverage for the advisory-only CLI exit-code contract and for consistency between the canonical and mirrored skill files.
1 comment outside of the diff
skills/laconic-prose/SKILL.md
line 1🟡 MEDIUM RISK
Suggestion: Add the required skill metadata, including an author, to the canonical skill file and synchronize the.cursor/skills/mirror so the metadata is identical in both copies.
Test suggestions
- Detect clause dashes across two sentences, including em dashes, en dashes, spaced hyphens and double hyphens, while ignoring paired asides, ranges, compounds, list markers, tables and code.
- Confirm clause-dash findings are advisory and do not produce a failing lint exit code.
- Detect the added stock phrases and avoid false positives for legitimate words, protected text and unrelated forms.
- Run calibration fixtures independently, calculate agreement, report mismatches and reject an empty fixture set.
- Validate the ten holdout cases and detect any modification through the pinned digest.
- Verify the new skill guidance is synchronised between canonical and mirrored files.
- Enforce the Style Manual corpus ceilings for normal findings and clause-dash advisories.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Confirm clause-dash findings are advisory and do not produce a failing lint exit code.
2. Verify the new skill guidance is synchronised between canonical and mirrored files.
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| prompt = build_judge_prompt(cases[fixture["case_id"]], {"fixture": fixture["response"]}, labels, rubric) | ||
| rows, spent = judge(prompt, (fixture["case_id"], fixture["name"]), labels) | ||
| cost += float(spent or 0) | ||
| if rows[0]["blocker"] != fixture["blocker"]: |
There was a problem hiding this comment.
🟡 MEDIUM RISK
Handle an empty result explicitly before indexing rows[0], either by recording a calibration error or raising a descriptive ValueError.
TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.
| return parser | ||
|
|
||
|
|
||
| def main(argv: Optional[list[str]] = None) -> int: |
There was a problem hiding this comment.
🟡 MEDIUM RISK
Suggestion: Consider extracting runner setup and calibration handling into focused helpers so main primarily parses arguments, dispatches the selected mode, and reports errors.
See Issue in Codacy
See Complexity in Codacy
TIP React 👍 if this review is right, or reply with what's wrong and how it could be better to help improve the reviewer.



Summary
Adds a clause-dash advisory and new stock phrases to
laconic-check, a calibration mode for the blocker judge, 10 frozen holdout cases and 3 skill-text additions.Skill text: a fiction line in 'Write literally', a social post venue with a regression row, and a limits sentence in the README. Each is mirrored into
.cursor/skills/.Evidence
judge.py --calibrateagreed with all 10 labelled responses (5 blockers).cargo test(56), clippy-D warnings,cargo fmt --check, Python unittest (168),run_evals.py validate,claude plugin validate .andgit diff --checkpass after rebasing on 24ce59c.Merge danger
Door: two-way. Blast radius: lint output
The new rule is advisory, so
lintexit codes do not change; output gains one finding type.Unverified
check_context_compat.tsdid not run (bun not installed locally); this change does not touch its inputs.