[triceo]

As a result of many experiments, I decided to keep the self-hosted runner, as I couldn't get this to stabilize on Github infrastructure. I got it down to +/- 6 % in the end, but that's still too much.

[Copilot]

🟡 Changes recommended

Unresolved runner security and benchmark correctness issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Stabilizes score-director benchmarks through batched execution, adjusted JMH settings, pinned benchmark builds, and self-hosted CI execution.

Changes:

• Batch 100 benchmark cycles per JMH invocation.
• Adjust benchmark checkout and build references.
• Run score-director benchmarks on self-hosted infrastructure.

File summaries

File	Summary
src/main/java/ai/timefold/solver/benchmarks/micro/scoredirector/Main.java	Updates JMH configuration; tiered compilation control needs to be retained.
src/main/java/ai/timefold/solver/benchmarks/micro/scoredirector/jmh/AbstractBenchmark.java	Adds batched invocation execution; documentation should accurately describe the lifecycle.
.github/workflows/performance_score_director.yml	Uses self-hosted execution and adjusts benchmark refs; runner isolation and consistent refs require changes.
.github/workflows/performance_move_provider.yml	Avoids unintended benchmark branch switching.

Review details

Suppressed comments (2)

.github/workflows/performance_score_director.yml:98

• This pin is later conditionally overwritten by the "Switch to correct Benchmarks branch" step: when the selected baseline ref exists in the benchmark repository (for example, a release tag), build_baseline compiles that ref, while the SUT can retain ${{ github.sha }} for main. The two jars can therefore contain different Main/AbstractBenchmark implementations, so the comparison still conflates harness changes with solver changes. Remove the later benchmark checkout or make one explicit benchmark ref apply to both builds.

          # Same commit for both baseline and SUT builds, so a harness-only change never confounds
          # the solver comparison. Pinning this to a fixed branch name meant a harness fix on any
          # other branch (incl. this one) was silently never exercised by this workflow; use the
          # commit that dispatched this run instead, whichever branch that happens to be.
          ref: ${{ github.sha }}

src/main/java/ai/timefold/solver/benchmarks/micro/scoredirector/jmh/AbstractBenchmark.java:23

• These are not independent draw-execute-score cycles: AbstractProblem.runInvocation() skips calculateScore() for temporary moves and deliberately keeps every tenth move, so the score-director state carries across iterations. Please describe this as batching the per-invocation lifecycle instead, otherwise the new documentation misstates what the benchmark measures.

     * Batches this many independent draw-execute-score cycles into one JMH invocation,

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.