Skip to content

Add review.sh pre-push review entry point - #12874

Draft
dougqh wants to merge 2 commits into
masterfrom
dougqh/review-entry-point
Draft

dougqh wants to merge 2 commits into
masterfrom
dougqh/review-entry-point

Conversation

@dougqh

@dougqh dougqh commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Adds review.sh, a pre-push review entry point, as part of APMLP-2033 (tracked under APMLP-1964). The script lives in tooling/review.sh, with a symlink at the repo root so contributors can run ./review.sh.

It does three things:

  • Picks the diff: it reviews the branch changes since the merge-base with upstream master. --base <ref> overrides that, and is also normalized to a merge-base.
  • Runs the harness's code review headlessly: it uses claude -p or codex exec, with a read-only tool set so the run can't stop on a permission prompt or edit files. The review applies the AGENTS.md Review Guidelines and Critical constraints.
  • Chains the repo extension /perf-review: --no-extensions skips it.

Like the rest of the plan, it is advisory: findings are printed, the script exits 0 after a review, and nothing is enforced. --out <file> saves the output for the eval comparison against a human baseline.

AGENTS.md's Review Guidelines and .claude/CLAUDE.md now point at ./review.sh.

Motivation

The 9/21 "Agent Code Review in Datadog SDKs" decision asks each tracer repo for a review.sh entry point, with review expectations kept in AGENTS.md. This builds on the existing perf-review skill.

Additional Notes

This is a draft while the interface is agreed across tracer repos. Open points:

  • Shared interface with other tracer repos: the flags and the output format, which matter for comparing results across languages.
  • Prompt vs built-in command: the code-review step currently sends a plain-language prompt rather than calling the harness's slash command (e.g. /review), so it works the same on both harnesses. Should it call the native command instead?
  • Codex: only the Claude path was run (about 45s on this branch's own diff, --no-extensions). The codex exec --sandbox read-only path is untested.
  • /techdebt: it is still listed in AGENTS.md but not chained by default.
  • Pre-push hook: an opt-in .githooks/pre-push is not included yet.

Follow-ups planned for the epic:

  • move recurring Java review rules into AGENTS.md and the docs it links
  • send checks that can be deterministic lints to static analysis (APMLP-1979)
  • build an eval set against human review
  • name a maintenance owner

🤖 Generated with Claude Code

dougqh and others added 2 commits October 14, 2026 11:58
Runs the harness's code review over the branch changes against the
AGENTS.md Review Guidelines, then /perf-review. Advisory only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh dougqh added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM labels Oct 14, 2026
@datadog-datadog-prod-us1-2

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.85 s [+0.0%; +1.6%] (maybe worse)
startup:insecure-bank:tracing:Agent 12.87 s 12.91 s [-1.0%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 17.07 s 16.98 s [-0.4%; +1.4%] (no difference)
startup:petclinic:iast:Agent 16.83 s 17.06 s [-2.2%; -0.5%] (maybe better)
startup:petclinic:profiling:Agent 16.58 s 16.64 s [-1.5%; +0.7%] (no difference)
startup:petclinic:sca:Agent 17.10 s 16.87 s [+0.5%; +2.2%] (maybe worse)
startup:petclinic:tracing:Agent 16.05 s 16.21 s [-1.8%; -0.1%] (maybe better)

Commit: c5d40219 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh
dougqh requested review from andrewlock and bric3 October 14, 2026 16:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant