Skip to content

Prioritize causal regression findings in mypy_primer analysis - #11763

Merged
Rich Chiodo (rchiodo) merged 5 commits into
mainfrom
rchiodo-mypy-primer-analysis
Sep 15, 2026
Merged

Rich Chiodo (rchiodo) merged 5 commits into
mainfrom
rchiodo-mypy-primer-analysis

Conversation

@rchiodo

@rchiodo Rich Chiodo (rchiodo) commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Replace the diagnostic-count comment table with ranked regression investigations: baseline evidence, changed behavior, downstream impact, causal explanation, separate PR attribution, and the remaining uncertainty or next check. Counts remain in the full artifact; less-concerning assessments are collapsed.
  • Extract generic warning signals from the complete primer evidence: bare Any/Unknown assertion results, other assertion failures, disappeared checking diagnostics, and increased gradual-type occurrences in changed details. These are leads, not confirmed regressions or proof of causation.
  • Keep recorded signals in the regression-review section even if the model labels them an improvement or a stub issue. Type erasures rank first. A likely-pr attribution requires a line-pinned source citation at the analyzed head.
  • Use GPT-5.6 Sol with the approved 200-credit analysis budget, retaining disabled token steering, the existing CLI pin, and independent threat detection. The 30-turn and 15-minute agent limits are unchanged.
  • Submit one project per tool call to respect the framework's 10240-byte string-input limit without compressing the entire investigation into one call. Assemble and validate all projects before rendering or publication; preserve the combined 1000000-character bound and compatibility with historical single-submission reports.
  • Require inspection of representative call sites and declarations, distinguish assertion expectations from observed baseline types, and prohibit treating PR intent or tests expecting Any as proof that lost checking is harmless.

Motivation

The first manual preview of #11601 classified 268 new pandas-stubs assertion failures and two scipy-stubs failures as high-confidence exposed typing issues, despite acknowledging that it had not inspected the affected overload declarations. The recorded results received bare Any instead of expected container or shape-specialized types. This needed a regression investigation, not a diagnostic inventory or an assumption that intentional conformance changes cannot cause compatibility regressions.

The new report contract separates the observed change, its consequences, and attribution. The deterministic warning layer prevents the same kind of benign model judgment from silently burying these signals. It is not an exhaustive detector and does not prove that every flagged difference is caused by the PR.

Validation

  • 127 focused analysis and MCP tests passed for the per-project submission and Sol change, including exact rendered reports, warning extraction and bounds, classification-resistant ranking, pinned attribution evidence, arbitrary project-name safety, and retained publication/preview safeguards. The generated-workflow configuration test also passed after the budget-only increase to 200.
  • New per-project tests cover an 11-project report larger than the single-call transport limit, complete publication, rejection of missing/duplicate/malformed/uncited entries, historical report compatibility, and the exact aggregate size boundary using 100 individually transport-sized submissions.
  • Reprocessed all eight saved Apply overload materialization to nested Any #11601 shards: detected the 268 pandas-stubs erasures, two scipy-stubs erasures, pip's increased Unknown return details, and removed-check leads.
  • Replayed the old model judgments with the new fields explicitly marked uninvestigated for the offline format/policy check: pandas-stubs and scipy-stubs became the first regression-review entries rather than benign table rows. This was not a new AI analysis.
  • Compilation with gh-aw v0.88.7 validated; repeated generation of the per-project change was byte-identical. Action-resolution pins are unchanged. Targeted lint, formatting, and whitespace checks passed.
  • The generated detection, manual-preview, and publisher jobs remain structurally unchanged. SymPy-only reporting policy and all freshness, completeness, rendering, and publication gates remain intact.
  • Review follow-up 4b991f5e4 removes three partial escapers from test expectations in favor of explicit fixture values. Complete output assertions remain intact.

Test policy classification: D — harness/configuration. No checker or typeshed behavior changed; test expectation changes cover the intentional report contract and presentation changes.

Live previews

The four comparison runs below used the historical #11601 primer run 31429223864, attempt 1, including all 11 projects.

Successful 200-credit Sol preview

Run 35001550116, at commit 9ef49f680, completed successfully end to end and used 120.14492 analysis credits out of 200. Agent execution, report validation, threat detection, and the preview job all succeeded. The mypy-primer-analysis-report artifact contains the actual comment.md and full-report.md; no local recovery was needed.

All 11 projects are included. The comment leads with pandas-stubs and scipy-stubs as high-confidence possible regressions, linking their bare-Any results to exact-head implementation and realistic PR test controls. Core and pip retain regression concerns but explicitly uncertain PR attribution. Exact downstream revisions and reproductions remain unverified, and baseline assertion acceptance is not an independent baseline reveal.

The publisher was intentionally skipped because this was a manual preview. No PR comment was posted.

Earlier runs and the submission fix

Terra run 34993340047 used 34.69602 analysis credits. Terra flagged the pandas-stubs and scipy-stubs erasures, but completion validation rejected Spark's likely-pr attribution because its citation used the wrong commit. Threat detection passed; preview and publication were skipped.

Terra run 34994808196 used 29.07228 analysis credits. Spark's citation used the correct commit, but Core was classified as possible-regression with no evidence, so validation failed again.

Further investigation found a transport problem in that second run: the MCP tool rejected the initial report for exceeding its 10240-byte input limit. Terra compressed the report and dropped citations, including Core's only citation. The pinned compiler does not expose a per-custom-input limit override, so commit 9d15743dd changes to per-project submissions rather than weakening that guard. Replaying the logged pre-compression payload through the unmodified pinned length validator preserves all 11 projects, each at most 2443 bytes. This fixes the transport problem, not unsupported causal claims: the original report still fails a separate Prefect citation check.

The initial Sol preview 34998888439, at commit 9d15743dd, submitted all 11 projects, with the largest submission 3163 bytes. The complete normalized output passed the repository validator locally, and threat detection passed. Its implementation/test citations were more useful than the earlier reports, but it recorded 101.39692 credits against the former 100-credit cap, followed by HTTP 403 on the next provider request. The agent exited unsuccessfully after submission, so its GitHub preview job was skipped and the report was recovered locally. The separately approved increase to 200 in 9ef49f680 allowed the subsequent run above to finish normally. These were model-plus-submission changes, not a model-only comparison.

Use GPT-5.6 Terra with the approved 100-credit budget. Require baseline, changed behavior, impact and pinned PR attribution; keep independently extracted regression warnings prominent even when model assessments are benign. Classification D: workflow harness/configuration; no checker, typeshed or type-precision expectation changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member

🔒 Automated review in progress — Bill Schnurr (@bschnurr) is auto-reviewing this PR.

@github-actions

This comment has been minimized.

Classification D: harness issue. Use explicit Markdown fixture values instead of partial escaping helpers. Preserve exact report assertions and all checker type expectations; no typeshed change applies.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member

Result: ⚠️ needs-more-tests

Verification details

Verification: Isolated verification observed failures whose relationship to this PR could not be determined: Targeted Prettier check; this review is not fully verified.

Summary: The two targeted Jest suites passed: 105 analysis tests and 16 MCP tests, matching the PR&#x27;s stated 121 focused tests. TypeScript compilation also passed. A targeted Prettier check failed on both changed Markdown files; whether this is introduced by the PR could not be established without baseline metadata. Coverage is strong, but the new fork-repository attribution acceptance path lacks a direct test.

Test runs: 5 passed, 1 failed

  • ❌ Failed | relationship unknown | Targeted Prettier check | pnpm exec prettier -c ../../build/ci/mypyPrimerAnalysis.ts src/tests/mypyPrimerAnalysis.test.ts ../../.github/workflows/mypy-primer-analysis.md ../../docs/mypy-primer-analysis.md
  • ✅ Passed | mypy_primer analysis Jest suite | node ./node_modules/jest/bin/jest.js src/tests/mypyPrimerAnalysis.test.ts --runInBand
  • ✅ Passed | mypy_primer native MCP preflight Jest suite | node ./node_modules/jest/bin/jest.js src/tests/mypyPrimerMcp.test.ts --runInBand
  • ✅ Passed | Dependency and test discovery | printf 'sandbox_profile=%s\n' "${AUTOMATION_SANDBOX_PROFILE:-}"; git status --short; git diff --name-status HEAD^ HEAD; test -d packages/pyright-internal/node_modules && echo 'node_modules=present' || echo 'node_modules=missing'; node -e "const p=require('./packages/pyright-internal/package.json'); console.log(JSON.stringify({scripts:p.scripts},null,2))"; find packages/pyright-internal/src/tests -maxdepth 1 -type f -name 'mypyPrimerAnalysis' -print
  • ✅ Passed | Offline dependency bootstrap | pnpm install --offline --frozen-lockfile
  • ✅ Passed | pyright-internal TypeScript compilation | pnpm exec tsc --noEmit
❌ Targeted Prettier check diagnostic output
Checking formatting...
[warn] ../../.github/workflows/mypy-primer-analysis.md
[warn] ../../docs/mypy-primer-analysis.md
[warn] Code style issues found in 2 files. Forgot to run Prettier?

@bschnurr Bill Schnurr (bschnurr) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@bschnurr Bill Schnurr (bschnurr) added the review-auto:approved Automated review: no blocking findings (approval posted). label Sep 15, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Classification D: harness/configuration. Avoid the pinned MCP tool's 10 KB per-call limit by collecting one project per submission. Preserve complete coverage, citation validation, historical reports, and the aggregate payload bound. Keep the 100-credit budget. No checker type precision or typeshed changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Classification D: harness/configuration. Increase the explicitly approved Sol analysis cap and update generated configuration, budget assertions, and documentation. Preserve model selection, report validation, and all checker type expectations; no typeshed changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member

Result: 🔴 could-not-verify

Verification details

Verification: The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

Summary: Verification could not be completed because the execution environment was unavailable before test discovery or targeted execution. The PR adds substantial focused Jest coverage for regression-signal extraction, report ranking/rendering, exact-head citation validation, per-project submission assembly, publication completeness, malformed payload rejection, historical compatibility, and aggregate size limits. No test result is claimed as passed.

Test runs: 2 not run

  • ⚠️ Not run | Dependency and targeted-test discovery | printf 'profile=%s\n' "${AUTOMATION_SANDBOX_PROFILE:-}"; test -d node_modules && echo root-node_modules-present || echo root-node_modules-missing; test -d packages/pyright-internal/node_modules && echo package-node_modules-present || echo package-node_modules-missing; node -p "require('./packages/pyright-internal/package.json').scripts"; test -f packages/pyright-internal/node_modules/jest/bin/jest.js && echo jest-present || echo jest-missing
  • ⚠️ Not run | mypyPrimerAnalysis focused Jest suite | node ./node_modules/jest/bin/jest.js src/tests/mypyPrimerAnalysis.test.ts --runInBand
⚠️ Dependency and targeted-test discovery diagnostic output
run_shell execution environment was unavailable; command did not run.
⚠️ mypyPrimerAnalysis focused Jest suite diagnostic output
Not attempted because dependency/test discovery could not execute.

@github-actions

This comment has been minimized.

Classification D: harness coverage. Exercise exact-head citations from the fork and upstream repository, and reject mismatched commits, missing line anchors, and unrelated repository paths. Preserve complete report assertions and all existing type precision; no checker or typeshed changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@rchiodo

Copy link
Copy Markdown
Collaborator Author

Addressed the verification feedback in 8411be2.

  • Added direct tests for exact-head citations from a fork and from upstream for a fork PR. The tests also reject a different commit, a missing line anchor, an unrelated owner, and a repository-name prefix lookalike. Full rendered-output assertions remain intact.
  • Reran both focused Jest suites on the current code: 117 analysis tests + 16 MCP tests = 133 passed. TypeScript compilation (tsc --noEmit), targeted ESLint, and whitespace checks also passed locally.
  • Reproduced the two Markdown Prettier warnings. The review command ran from packages/pyright-internal, so it did not load the root .prettierignore. That existing file excludes *.md on both the base branch and this branch. The same command passes with --ignore-path ../../.prettierignore; I left the repository's formatting policy and Markdown unchanged.

These are local verification results; the later review runner's unavailable execution environment is a separate issue, not a failing test. The new feedback was in top-level verification summaries, and there are no unresolved inline review threads.

@github-actions

Copy link
Copy Markdown
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

sympy (https://github.com/sympy/sympy)
-   .../projects/sympy/sympy/solvers/bivariate.py:135:15 - error: Operator "-" not supported for "None" (reportOptionalOperand)
-   .../projects/sympy/sympy/solvers/bivariate.py:139:17 - error: Operator "-" not supported for type "Basic | Unknown" (reportOperatorIssue)
-   .../projects/sympy/sympy/solvers/bivariate.py:144:23 - error: Operator "-" not supported for "None" (reportOptionalOperand)
-   .../projects/sympy/sympy/solvers/deutils.py:234:14 - error: Operator "not in" not supported for types "str" and "Unknown | int"
-     Operator "not in" not supported for types "str" and "int" (reportOperatorIssue)
-   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:176:44 - error: Cannot access attribute "expand" for class "Basic"
-     Attribute "expand" is unknown (reportAttributeAccessIssue)
-   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:423:38 - error: Argument of type "Unknown | Expr | Literal[0]" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:423:38 - error: Argument of type "int | Expr" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
-     Type "Unknown | Expr | Literal[0]" is not assignable to type "Expr"
+     Type "int | Expr" is not assignable to type "Expr"
-       "Literal[0]" is not assignable to "Expr" (reportArgumentType)
+       "int" is not assignable to "Expr" (reportArgumentType)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:504:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:505:19 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:506:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:569:42 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+     Operator "*" not supported for types "int" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:569:50 - error: Operator "*" not supported for types "Expr" and "Unknown | Basic"
+     Operator "*" not supported for types "Expr" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:725:42 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:735:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:736:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:737:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:814:47 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:841:26 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:842:22 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:36 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+     Operator "*" not supported for types "int" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:42 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:51 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+     Operator "*" not supported for types "int" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:57 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:66 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:845:53 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:854:18 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:856:22 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:865:22 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:865:36 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:867:27 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:868:27 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:869:27 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:870:27 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:871:27 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:872:27 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:876:24 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:877:24 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:878:24 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+     Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:879:24 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:880:24 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:881:24 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+     Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+   .../projects/sympy/sympy/solvers/diophantine/diophantine.py:888:37 - error: Operator "*" not supported for types "int" and "Unknown | None"
+     Operator "*" not supported for types "int" and "None" (reportOperatorIssue)

... (truncated 644 lines) ...

@github-actions

Copy link
Copy Markdown
Contributor

mypy_primer analysis

Advisory AI analysis of primer run 35003751736, attempt 1
for commit 8411be2d18b768329583e396072fd9d28c62447b. This is not an approval or proof of correctness.

Only SymPy changed. These differences are treated as non-blocking primer noise; no other project changed.

Recorded changes: 251 added / 147 removed diagnostic headers; 181 added / 146 removed detail lines.

Full evidence and limitations: download mypy-primer-analysis-report from this analysis run.
The original raw primer comment is unchanged. Treat uncertainty and possible regressions as requests for human review.

@bschnurr

Copy link
Copy Markdown
Member

Result: ✅ verified

Verification details

Verification: Isolated verification observed failures that were not classified as caused by this PR: Dependency and focused-test discovery.

Summary: Initial Jest discovery could not run because dependencies were absent; the frozen offline install then succeeded. The focused analysis suite passed 117/117 tests, and the MCP preflight suite passed 16/16 tests. These cover regression signals, report ranking/rendering, attribution validation, per-project assembly/publication, workflow configuration, and MCP safeguards. All relevant tests passed after setup, so confidence is high.

Test runs: 3 passed, 1 failed

  • ❌ Failed | unrelated to this PR | Dependency and focused-test discovery | printf 'profile=%s\n' "${AUTOMATION_SANDBOX_PROFILE:-unset}"; node --version; pnpm --version; test -f ./node_modules/jest/bin/jest.js && echo 'jest-runner=present' || echo 'jest-runner=missing'; test -f ../../pnpm-lock.yaml && echo 'lockfile=present' || echo 'lockfile=missing'; node ./node_modules/jest/bin/jest.js --listTests mypyPrimerAnalysis.test.ts
  • ✅ Passed | mypy_primer analysis | node ./node_modules/jest/bin/jest.js src/tests/mypyPrimerAnalysis.test.ts --runInBand --forceExit
  • ✅ Passed | mypy_primer native MCP preflight | node ./node_modules/jest/bin/jest.js src/tests/mypyPrimerMcp.test.ts --runInBand --forceExit
  • ✅ Passed | Offline dependency bootstrap | pnpm install --offline --frozen-lockfile
❌ Dependency and focused-test discovery diagnostic output
profile=typescript
v24.15.0
11.21.0
jest-runner=missing
lockfile=present
Error: Cannot find module '/workspace/packages/pyright-internal/node_modules/jest/bin/jest.js'

@rchiodo
Rich Chiodo (rchiodo) merged commit 3f5800a into main Sep 15, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants