Skip to content

Configure shared and release S3 build cache namespaces - #13125

Open
sarahchen6 wants to merge 3 commits into
bdu/use-s3-gradle-build-cachefrom
sarahchen6/s3-cache-path
Open

sarahchen6 wants to merge 3 commits into
bdu/use-s3-gradle-build-cachefrom
sarahchen6/s3-cache-path

Conversation

@sarahchen6

@sarahchen6 sarahchen6 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Make the full S3 cache prefix configurable, share caches across non-tag (e.g. PR, nightly, etc.) pipelines, and ensure release caches are isolated. Also prevent cached agent JARs from having stale dependency version stamps and validate artifacts against a non-cached build.

Motivation

Reduce repeated CI work while keeping release caches isolated and verifying that cached outputs match a fresh build. This should also reduce CI runtime. See detailed results in #12604

Additional Notes

Tested with the following pipeline runs:

Run Pipeline Duration
Initial cache-enabled 148987382 46m41s
Cache-disabled 149012709 51m14s
Warm cache 149038257 29m09s
Latest master 149030299 49m43s

Contributor Checklist

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-1897, https://datadoghq.atlassian.net/browse/APMLP-1899

@sarahchen6 sarahchen6 added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Oct 29, 2026
@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.68% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ebb43c0 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.93 s 13.93 s [-0.8%; +0.7%] (no difference)
startup:insecure-bank:tracing:Agent 12.95 s 12.95 s [-0.7%; +0.6%] (no difference)
startup:petclinic:appsec:Agent 17.71 s 17.49 s [+0.2%; +2.3%] (maybe worse)
startup:petclinic:iast:Agent 17.42 s 17.70 s [-2.6%; -0.7%] (maybe better)
startup:petclinic:profiling:Agent 16.71 s 17.52 s [-8.9%; -0.4%] (maybe better)
startup:petclinic:sca:Agent 17.03 s 17.54 s [-7.2%; +1.4%] (no difference)
startup:petclinic:tracing:Agent 16.74 s 16.80 s [-1.0%; +0.4%] (no difference)

Commit: ebb43c0b · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@sarahchen6
sarahchen6 force-pushed the sarahchen6/s3-cache-path branch from 69c1312 to 2feecba Compare October 30, 2026 15:42
@sarahchen6
sarahchen6 requested a review from bric3 October 30, 2026 19:00
@sarahchen6
sarahchen6 marked this pull request as ready for review October 30, 2026 19:00
@sarahchen6
sarahchen6 requested review from a team as code owners October 30, 2026 19:00
@sarahchen6
sarahchen6 requested review from erikayasuda and removed request for a team October 30, 2026 19:00
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-30T19:06:35.404669Z ebb43c0 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-30T19:09:52.725448Z ebb43c0 Draft marked ready

Security findings

Finding details are still loading. Check the individual review comments.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebb43c0b6c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .gitlab-ci.yml Resolved

@datadog-prod-us1-6 datadog-prod-us1-6 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static review supports per-pipeline release-cache isolation and artifact validation that rebuilds without cache reuse. End-to-end cache behavior and artifact equivalence remain unverified without a full Gradle build or GitLab pipeline.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit ebb43c0

}
includeShadowJar(traceShadowJar, 'trace', includedJarFileTree)

[sharedShadowJar, traceShadowJar].each { shadowJarTask ->

@sarahchen6 sarahchen6 Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, Gradle can reuse a cached agent JAR with version stamps from an older commit (failing validate_build). This makes Gradle notice when those stamps change and rebuild the JAR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant