diff --git a/AGENTS.md b/AGENTS.md index 065d5f9b77c..cea154a77ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -45,6 +45,12 @@ docs/ Developer documentation (see below) **When working on a topic above, read the linked file first** — they are the source of truth maintained by humans. +## Gradle changes + +Follow the [Gradle guide](docs/how_to_work_with_gradle.md), including its +[required skill](docs/how_to_work_with_gradle.md#gradle-skills). Before editing files under +`buildSrc/` or `build-logic/`, also read that folder's `AGENTS.md`. + ## Build quick reference ```shell diff --git a/build-logic/AGENTS.md b/build-logic/AGENTS.md new file mode 100644 index 00000000000..6df2b453f6e --- /dev/null +++ b/build-logic/AGENTS.md @@ -0,0 +1,25 @@ +# Working in build-logic + +Use `gradle-best-practices` before changing Gradle code here. If it is missing or incomplete, +follow the [skill setup](../docs/how_to_work_with_gradle.md#gradle-skills) before editing. + +This included build holds project conventions and the smoke-test, tag-registry and +testcontainers plugins. Its settings import `../gradle/libs.versions.toml` and configure +repository proxies separately from the main build. + +The smoke-test plugin runs nested builds with older Gradle versions. Its tests use Java 21 +for that reason; keep that launcher when changing the tests or toolchain configuration. + +## Tests + +The plugin tests use JUnit 5 and Kotlin. Reuse the module's TestKit setup, including its +repository proxy handling and environment isolation. + +Run the tests for the module you changed, for example: + +```shell +./gradlew -p build-logic :smoke-test:test +``` + +For changes across modules, `./gradlew :build-logic:check` runs the included build's checks +from the repository root. diff --git a/build-logic/CLAUDE.md b/build-logic/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/build-logic/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/buildSrc/AGENTS.md b/buildSrc/AGENTS.md new file mode 100644 index 00000000000..4d5896a04be --- /dev/null +++ b/buildSrc/AGENTS.md @@ -0,0 +1,23 @@ +# Working in buildSrc + +Use `gradle-best-practices` before changing Gradle code here. If it is missing or incomplete, +follow the [skill setup](../docs/how_to_work_with_gradle.md#gradle-skills) before editing. + +For new conventions, follow the [build logic placement advice](../docs/how_to_work_with_gradle.md#build-logic-location). + +The Java 8 target in `build.gradle.kts` is needed by build-time instrumentation and Muzzle +workers. A newer Gradle daemon or compiler JVM does not remove that requirement. + +JMH uses the consuming build's plugin classpath. Read the comments in +[its convention](src/main/kotlin/dd-trace-java.jmh-conventions.gradle.kts) before changing +that integration or adding JMH to `implementation`. + +## Tests + +The plugin tests use JUnit 5 and Kotlin. Reuse +[GradleFixture](src/test/kotlin/datadog/gradle/plugin/GradleFixture.kt) for TestKit tests. +CLI tests need `-PrunBuildSrcTests`; see [the test commands](../docs/how_to_work_with_gradle.md#running-build-logic-tests). +Keep the `:test` selector when filtering so the filter does not reach unrelated subprojects. + +Changes to `call-site-instrumentation-plugin` or `modifiable-config-agent` also need their +own subproject tests. Run `./gradlew -p buildSrc spotlessCheck` for formatting. diff --git a/buildSrc/CLAUDE.md b/buildSrc/CLAUDE.md new file mode 100644 index 00000000000..43c994c2d36 --- /dev/null +++ b/buildSrc/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/docs/how_to_work_with_gradle.md b/docs/how_to_work_with_gradle.md index b0f8ef43d9a..0e8f8645999 100644 --- a/docs/how_to_work_with_gradle.md +++ b/docs/how_to_work_with_gradle.md @@ -1,10 +1,17 @@ # How to Work with Gradle -This guide covers the fundamentals of working with Gradle in this project. Understanding these concepts will help you navigate the build system and contribute effectively. +This guide covers Gradle best practices, how we apply them in this repository, and the +constraints that currently require exceptions. Understanding these concepts will help +you navigate the build system and contribute effectively. + +The guide describes shared practices and repository constraints. Details specific to one +plugin or task belong beside its source or in local documentation. ## Quick Reference **Core Concepts:** + +- [Gradle skills](#gradle-skills) - Installing and using the required skill for agent-assisted changes - [Gradle Files](#what-are-gradle-files) — Groovy vs Kotlin DSL - [Build Lifecycle](#gradle-build-lifecycle) — Initialization → Configuration → Execution phases - [Tasks](#gradle-tasks) — Lifecycle tasks, inputs/outputs, lazy configuration @@ -25,16 +32,30 @@ This guide covers the fundamentals of working with Gradle in this project. Under > [!TIP] > First time with Gradle? Start with [Gradle Files](#what-are-gradle-files), [Build Lifecycle](#gradle-build-lifecycle), and [Tasks](#gradle-tasks). +## Gradle skills + +Gradle publishes a [best-practices skill](https://github.com/gradle/gradle-skills) for reviewing +build scripts and plugin code. In this repository, agent-assisted Gradle code changes require +an audit of the affected code and its consuming modules with this skill. The requested task +defines the scope of fixes; unrelated findings are reported separately. + +The [upstream installation instructions](https://github.com/gradle/gradle-skills#installation) +cover supported agents. A complete installation includes the skill's reference files. +Without them, only read-only investigation is available until installation is complete. + +The skill's catalog may describe newer or nightly APIs. Its recommendations need to match +our Gradle wrapper version and the JVMs that run the affected code. + ## What are Gradle Files? -Gradle builds are defined through a set of build scripts. These scripts can be written in two Domain Specific Languages (DSLs): **Groovy DSL** and **Kotlin DSL**. +Gradle builds are defined through a set of build scripts. These scripts can be written in two Domain Specific Languages (DSLs): **Groovy DSL** (`*.gradle`) and **Kotlin DSL** (`*.gradle.kts`) . ### Groovy DSL The original Gradle DSL uses Groovy syntax. Files use the `.gradle` extension. Avoid Groovy DSL if you can, small project, however since the project still rely -heaviliy on _script plugins_ written in Groovy, this is not always straighforward to use Kotlin DSL. +heavily on _script plugins_ written in Groovy, this is not always straightforward to use Kotlin DSL. ```Gradle plugins { @@ -93,19 +114,39 @@ Gradle executes builds in distinct phases. Understanding this lifecycle is essen ### 1. Initialization Phase -Gradle determines which projects are part of the build. It executes: +Gradle runs init scripts, then evaluates the settings script to determine which projects belong +to the build. -- **`init.gradle`** (or scripts in `~/.gradle/init.d/`): Global initialization scripts that run before any project is evaluated -- **`settings.gradle.kts`**: Defines the main repository project structure and discovers subprojects +- **[Init scripts](https://docs.gradle.org/current/userguide/init_scripts.html)** can configure + a build without changing its build scripts. They can be passed with `--init-script` or loaded + from the Gradle user home's `init.d/` directory (`~/.gradle/init.d/` by default). -```Gradle Kotlin DSL -// settings.gradle.kts -rootProject.name = "my-project" + For example, an init script can enable offline dependency resolution: + + ```Gradle Kotlin DSL + // offline.init.gradle.kts + gradle.startParameter.isOffline = true + ``` + + ```bash + ./gradlew --init-script offline.init.gradle.kts build + ``` -include("module-a") -include("module-b") -include("module-c:submodule") -``` + This invocation uses cached dependencies and fails if a required dependency is missing. + + +- **`settings.gradle.kts`** defines the root project and includes subprojects. + + The settings script defines the project structure: + + ```Gradle Kotlin DSL + // settings.gradle.kts + rootProject.name = "my-project" + + include(":module-a") + include(":module-b") + include(":module-c:submodule") + ``` ### 2. Configuration Phase @@ -161,19 +202,60 @@ Only tasks required to complete the requested goal are executed. In a well-organized Gradle project, build logic lives in specific places: -| Location | Purpose | -|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `settings.gradle.kts` | Project structure, repository settings, plugin management | -| `build.gradle.kts` | Project-specific build configuration | -| `buildSrc/` | Build logic automatically included by Gradle; contains convention plugins and shared configuration. It's possible to use different location(s) but it requires explicit declaration(s). | -| `gradle/` | Version catalogs and wrapper files and script plugins | +| Location | Purpose | +|-----------------------|----------------------------------------------------------------| +| `settings.gradle.kts` | Project structure, repositories and plugin management | +| `build.gradle.kts` | Configuration for one project | +| `buildSrc/` | Legacy build logic, automatically available to the main build | +| `build-logic/` | Build logic included explicitly through plugin management | +| `gradle/` | Version catalogs, wrapper files and remaining *script plugins* | + +`buildSrc` and included builds have their own settings and dependencies. Repositories and +version catalogs from the main build are not inherited, so each build has its own declarations +or explicit imports. + +Gradle [recommends an included `build-logic` build](https://docs.gradle.org/current/userguide/best_practices_structuring_builds.html#favor_composite_builds) +for shared conventions. Its plugins have explicit dependencies and are added only to the +projects that use them. +This limits classpath coupling and can reduce build script recompilation when build logic +changes. Plugins can also be built and tested independently. + +`buildSrc` is the legacy location for our build logic and remains supported by Gradle. + +Project build scripts and applied *script plugins* load `buildSrc` classes through a common +parent classloader. We are replacing script plugins with convention plugins, using `buildSrc` +as an intermediate step for conventions applied across all modules through existing scripts. +Those scripts still need access to shared classes through that classloader, which currently +prevents those classes from moving into `build-logic`. Independent conventions applied on +demand can go directly into `build-logic`: explicit plugin requests provide their classpath +to the consuming projects. > [!CAUTION] -> Script plugins are not recommended. The best practice for developing our build logic in plugins is -> to create _convention plugins_ or _binary plugins_. +> *Script plugins* are not recommended. Shared build logic belongs in _convention plugins_ +> or _binary plugins_. ### Running Build Logic Tests +Unit tests cover helpers, while Gradle's TestKit covers behaviour that depends on Gradle. +Simple declarative conventions may not need dedicated tests. Larger plugins with conditional +behaviour benefit from functional tests covering: + +- Plugin application, including different application orders for interacting plugins +- Configuration roles and dependency resolution +- Task wiring, outcomes and generated files +- Behaviour in a consuming module + +Existing fixtures provide the setup for these tests. Isolated Projects support is not currently +required, but avoiding cross-project state reduces future migration work. +See Gradle's [plugin testing guide](https://docs.gradle.org/current/userguide/testing_gradle_plugins.html). + +Repeated runs exercise different behaviour: + +- `--rerun` on a task, or `--rerun-tasks` for all selected tasks, forces execution again. +- Two identical runs with `--configuration-cache` show whether the second reuses the entry. +- Unchanged inputs leave an incremental task `UP-TO-DATE`; changed inputs update its output. +- Restoring outputs from a populated build cache into a clean fixture produces `FROM-CACHE`. + Running `buildSrc/` tests within IntelliJ IDEA works without special treatment because the build detects the injected `idea.active` system property. @@ -253,6 +335,11 @@ In custom tasks, use annotations to declare inputs and outputs: More annotations are documented [here](https://docs.gradle.org/current/userguide/implementing_custom_tasks.html#annotating_inputs_and_outputs). +Each task owns its outputs, and path sensitivity reflects which parts of an input path affect +the result. A custom task class can opt into build caching with `@CacheableTask` when its +outputs are reproducible from its declared inputs. `@DisableCachingByDefault` records why +caching is unsuitable otherwise. + ##### In custom tasks declared with DSL For ad-hoc tasks or when you can't use annotations, declare inputs and outputs programmatically via the `inputs` and `outputs` properties: @@ -333,6 +420,21 @@ When running Gradle tasks, you'll see status labels indicating what happened dur | `NO-SOURCE` | The task had no input files to process | +### Task Dependencies + +Gradle infers a task dependency when a producing task's output provider is connected to a +consuming task's input. The declared inputs and outputs also support up-to-date checks; +see [Using Providers](#using-providers). + +An explicit `dependsOn` schedules a prerequisite even when no artifact flows between tasks. +Lifecycle tasks such as `check` and `build` use it to collect other tasks. + +`mustRunAfter` and `shouldRunAfter` only order tasks already selected to run. They do not +schedule a prerequisite. `finalizedBy` schedules follow-up work such as cleanup, +including when that task fails. See [Controlling Task Execution](https://docs.gradle.org/current/userguide/controlling_task_execution.html). + +Avoid usage of `TaskDependency` interface. + ### Tasks tips - **Prefer `Sync` over `Copy`**: `Sync` mirrors the source to the destination by removing files that no longer exist @@ -1131,14 +1233,39 @@ common configuration patterns and can be applied like any other plugin. > Convention plugins promote consistency across modules. Instead of copy-pasting configuration, > define it once and apply it everywhere. +### Plugin Boundaries + +Plugins retain separate responsibilities even within `buildSrc`. Plugin IDs, documented +extensions, properties, task paths and public Gradle APIs let them interact without depending +on each other's implementation details. Shared code is useful when it represents knowledge +that must change together across plugins. + +See Gradle's [plugin development guide](https://docs.gradle.org/current/userguide/implementing_gradle_plugins_binary.html). + +### Dependencies on Main Projects + +Build tooling remains independent of production code. The tasks it registers can consume +production artifacts through declared inputs. Convention plugins can also declare +[project dependencies](https://docs.gradle.org/current/userguide/multi_project_builds.html) +for the projects they configure. + +Concrete project paths usually belong in the consuming build, keeping reusable plugins +independent of the repository layout. Exceptions have a documented reason. + +[`buildSrc` is compiled before the main build scripts are evaluated](https://docs.gradle.org/current/userguide/sharing_build_logic_between_subprojects.html). +Gradle schedules tasks for project dependencies as part of the main build. + ### Project Convention Plugins Files ending in `.gradle.kts` placed in `buildSrc/src/main/kotlin/` target `Project` and can configure tasks, dependencies, and extensions. The `buildSrc/` directory is automatically included by Gradle before the main build. +New convention plugins belong in `build-logic`. During the script-plugin migration, conventions +that need shared `buildSrc` classes still need to land in `buildSrc`; see [Build Logic Location](#build-logic-location). + > [!IMPORTANT] -> Prefer build logic that is independent of concrete project paths. Referring to repository projects -> from `build-logic` or `buildSrc` couples the plugin to the project layout; keep such dependencies -> deliberate and limited to conventions that need them. +> Hard-coded project paths such as `:dd-trace-api` tie a plugin to the repository layout, in other words it creates coupling. +> They belong in the consuming build script unless the dependency is part of the convention; +> see [Dependencies on Main Projects](#dependencies-on-main-projects). In this project, convention plugins use the `dd-trace-java.` prefix. For example, `dd-trace-java.configure-tests.gradle.kts` configures test tasks across all subprojects: @@ -1246,7 +1373,7 @@ Script plugins are standalone `.gradle` or `.gradle.kts` files that can be appli apply(from = "$rootDir/gradle/some-script.gradle") ``` -As warned, don't wrtite new ones, use convention plugins instead ! +As warned, don't write new ones, use convention plugins instead ! - **No type safety**: When written in Groovy DSL, you lose IDE support and compile-time checking - **Mixed DSL confusion**: Projects often end up with a mix of Groovy and Kotlin scripts @@ -1259,43 +1386,54 @@ maintainability and performance. For module build files, use a `dd-trace-java.mo ## Gradle Lazy API -Each time Gradle is invoked it project **must** go through the Gradle **Configuration phase**, in which it -evaluates all build scripts of the participating projects. Any expensive actions in this phase will be run -_every single time_. +Gradle configures participating projects before executing tasks, unless it can reuse the +*configuration cache*. Configuration work affects even short commands such as `help`. +With roughly 630 projects and 33,000 tasks in dd-trace-java, unnecessary task creation and +configuration can add substantial overhead. -This means inefficient configuration directly impacts developer experience by slowing down all builds -— regardless of which tasks actually execute. While time savings per individual task may seem modest, -they compound quickly: **dd-trace-java has ~630 projects and ~33,000 tasks**. At that scale, even small -inefficiencies add up significantly. - -The solution is Gradle's **lazy API**: make task creation and configuration as lazy as possible, so Gradle only realizes and configures objects it actually needs to execute. +Gradle's lazy APIs defer task creation, task configuration and value calculation until needed. ### Why Lazy Configuration Matters -When you use eager APIs, values are computed immediately during configuration—even if the task never runs. Lazy APIs defer this work to execution time, and Gradle can automatically track dependencies between producers and consumers. +The following methods `register`, `named` and `configureEach` avoid creating and +configuring unused tasks. Task configuration still belongs to the configuration phase. +The providers defer value calculation until they are queried, which can happen during +configuration or execution. Providers backed by task outputs also carry producer +dependencies. ### Eager vs Lazy API Comparison -| Eager (Don't ❌) | Lazy (Prefer ✅) | Notes | -|---------------------------------|----------------------------------------|------------------------------------------------------------------------| -| `configurations.getByName("x")` | `configurations.named("x")` | Returns a `NamedDomainObjectProvider` instead of resolving immediately | -| `tasks.getByName("x")` | `tasks.named("x")` | Avoids triggering task creation/configuration | -| `tasks.findByName("x")` | `tasks.named("x")` | Returns `null` if not found, but still realizes the task | -| `tasks.findByPath(":x")` | `tasks.named("x")` on target project | Realizes task eagerly; use project reference with `named()` instead | -| `tasks.create("x")` | `tasks.register("x")` | Task is only created when needed | -| `property.set(someValue)` | `property.set(provider { someValue })` | Defers computation of the value | -| `collection.all { }` | `collection.configureEach { }` | Configures lazily as elements are realized | -| `collection.forEach { }` | `collection.configureEach { }` | Avoids forcing realization of all elements | -| `file(path).exists()` | Use task inputs/outputs | Let Gradle track file dependencies | -| `exec { }.exitValue` | See Exec pattern below | Avoid running processes at configuration time | +| Eager (Don't ❌) | Lazy (Prefer ✅) | Notes | +| ------------------------------- | ------------------------------------------- | -------------------------------------------------------------------------------------- | +| `configurations.getByName("x")` | `configurations.named("x")` | Defers realization of the configuration object, not dependency resolution | +| `tasks.getByName("x")` | `tasks.named("x")` | Avoids creating and configuring the task during lookup | +| `tasks.findByName("x")` | `tasks.named("x")` | `findByName` returns `null` if absent; `named` throws instead | +| `tasks.findByPath(":x")` | `tasks.named("x")` in the owning project | `findByPath` returns `null` if absent; `named` takes a local name and throws if absent | +| `tasks.create("x")` | `tasks.register("x")` | Registers the task without creating or configuring it | +| `property.set(computeValue())` | `property.set(provider { computeValue() })` | Defers the computation; wrapping an already computed value adds no laziness | +| `collection.all { }` | `collection.configureEach { }` | Runs configuration actions only for realized domain objects | +| `collection.forEach { }` | `collection.configureEach { }` | Applies to configuration actions, rather than collection transformations | +| `file(path).exists()` | Declared task inputs/outputs | Tracks file dependencies; configuration-time existence checks may still be necessary | +| `configuration.files` | `inputs.files(configuration)` | Passes the file collection through and preserves producer dependencies | +| `exec { }.exitValue` | `providers.exec { ... }` | `exec` runs immediately; `providers.exec` defers execution until its output is queried | +| `ProcessBuilder(...).start()` | `providers.exec { ... }` | Exposes process output lazily; see the example under Using Providers | > [!IMPORTANT] -> Any function that iterates over a Gradle collection (`forEach`, `map`, `filter`, `all`, `any`, -> `find`, `first`, etc.) will **eagerly realize all elements**. This defeats lazy configuration. -> Always use `configureEach`, or use `named`/`withType` to get lazy providers. +> Iterating a task or configuration container can realize its elements. +> **Any function that iterates over a Gradle collection** (`forEach`, `map`, `filter`, `all`, `any`, `find`, `first`, etc.) will **eagerly realize all elements**, +> defeating any other lazy API usage. +> +> Instead prefer, +> * `configureEach` registers actions for realized elements; +> * `withType` creates a filtered live collection. +> * Provider transformations such as `map` and `flatMap` remain lazy until queried. > [!WARNING] -> **Groovy DSL pitfall**: The shorthand syntax `name { }` is **eager** for both tasks and configurations. It calls `getByName()` under the hood, which realizes the element and its dependencies immediately. +> **Groovy DSL pitfall**: Dynamic task and configuration blocks (shorthand syntax +> `theTaskName { }`, or `theConfigurationName { }`) look up their target immediately +> (calling under the hood `getByName("theTaskName")`, or `getByName("theConfigurationName")`), realizing the object. +> +> Note that just looking up a configuration by name does not resolve its dependencies. > > ```Gradle > // ❌ Eager - realizes the task immediately @@ -1310,9 +1448,11 @@ When you use eager APIs, values are computed immediately during configuration— > ``` > > ```Gradle -> // ❌ Eager - resolves the configuration immediately -> runtimeClasspath { -> exclude group: 'org.slf4j' +> // ❌ Eager - realizes the configuration object +> configurations { +> runtimeClasspath { +> exclude group: 'org.slf4j' +> } > } > > // ✅ Lazy - configures only when needed @@ -1331,20 +1471,30 @@ tasks.create("processData") { // ✅ Lazy - task is created only when needed tasks.register("processData") { - // configuration runs only when this task is in the execution graph + // Task configuration normally runs when this task is needed for the execution graph or eager lookup. } ``` +Configuration actions may run immediately, later or never. Mutating another task from one +of these actions makes the result depend on realization order. Dependencies therefore belong +in the consuming task's own configuration action. + ### Configuration Access +`getByName` realizes the configuration object. Its dependencies remain unresolved until Gradle +needs their files. + ```Gradle Kotlin DSL -// ❌ Eager - resolves configuration immediately +// ❌ Eager - realizes the configuration object val runtimeClasspath = configurations.getByName("runtimeClasspath") // ✅ Lazy - returns a provider val runtimeClasspath = configurations.named("runtimeClasspath") ``` +Queries such as `.files`, `.singleFile` or `.resolve()` trigger dependency resolution. +Task inputs can accept a configuration as a file collection without resolving it in build logic. + ### Walking Collections ```Gradle Kotlin DSL @@ -1355,8 +1505,8 @@ tasks.all { } } -// ❌ Eager - forces all tasks to be created -tasks.withType() { +// ❌ Eager - realizes all JavaCompile tasks +tasks.withType { options.encoding = "UTF-8" } @@ -1368,74 +1518,232 @@ tasks.withType().configureEach { ### Using Providers +A provider represents a value that Gradle can query when needed. Properties and file +collections that accept providers retain this connection without reading the value immediately. +`map` transforms a value; `flatMap` connects to another provider, such as a task's output property. + +With the Java plugin applied, these examples show eager and lazy access to its JAR: + ```Gradle Kotlin DSL -// ❌ Eager - version is read immediately -tasks.register("myJar") { - archiveVersion.set(project.version.toString()) +// ❌ Eager - reads the archive path while configuring this task +tasks.register("stageJarEager") { + from(tasks.named("jar").get().archiveFile.get()) + into(layout.buildDirectory.dir("staged-eager")) } -// ✅ Lazy - version is read when the jar task runs -tasks.register("myJar") { - archiveVersion.set(project.provider { project.version.toString() }) +// ✅ Lazy - preserves the output provider and its dependency on the jar task +tasks.register("stageJar") { + from(tasks.named("jar").flatMap { it.archiveFile }) + into(layout.buildDirectory.dir("staged")) } ``` +The lazy version also carries the producer dependency, so Gradle runs `jar` before `stageJar`. +The eager version passes only a file path and loses that dependency information. + > [!TIP] -> In Gradle Kotlin DSL (and Groovy DSL) you can use `=` instead of `set(...)`, e.g. -> `archiveVersion = project.version.toString()` +> Kotlin DSL supports `=` for Gradle lazy properties, for example `archiveFileName = "app.jar"`. +> This has the same effect as `archiveFileName.set("app.jar")`; both forms also accept providers. -### Lazy JVM Arguments and System Properties +Provider support depends on the receiving API. Some APIs with `Any` or `Object` parameters +convert a provider to a string instead of reading its value. JVM arguments are one example; +see [Lazy JVM Arguments and System Properties](#lazy-jvm-arguments-and-system-properties). -A common pitfall is reading task outputs or other values during configuration. Here's how to pass values lazily to `Test` or `JavaExec` tasks using `CommandLineArgumentProvider`: +Command output can also be exposed as a provider: -```Gradle -// ❌ Eager - task output is resolved at configuration time -tasks.withType(Test).configureEach { - def fooShadowJarTask = tasks.named('fooShadowJar', ShadowJar) - def barShadowJarTask = tasks.named('barShadowJarTask', ShadowJar) - dependsOn fooShadowJarTask, barShadowJarTask - - // This resolves the archive path immediately during configuration! - systemProperty "smoketest.foo.path", fooShadowJarTask.get().archiveFile.get() - environment "BAR_PATH", barShadowJarTask.get().archiveFile.get() +```Gradle Kotlin DSL +val gitVersion = providers.exec { + commandLine("git", "--version") +}.standardOutput.asText.map { it.trim() } +``` + +The command runs when its output is queried. A configuration-time query makes that output +a configuration-cache input. + +> [!TIP] +> Existing `afterEvaluate` blocks can often be refactored away using providers, +> lazy task configuration and plugin callbacks. + +#### Custom Value Sources + +When a provider is queried during configuration, Gradle tracks the external data it reads +to decide whether the configuration cache can be reused. A custom +[`ValueSource`](https://docs.gradle.org/current/javadoc/org/gradle/api/provider/ValueSource.html) +lets Gradle track the computed result instead. This is useful when only part of the data +affects configuration. + +For example, reading the `version` entry from a `release.properties` file that also contains +other release metadata: + +```Gradle Kotlin DSL +import java.util.Properties + +val releaseFile = layout.projectDirectory.file("release.properties") + +// ❌ Any file content change invalidates the configuration cache +val releaseVersionProvider = providers.fileContents(releaseFile).asText.map { + Properties().apply { load(it.reader()) }.getProperty("version") } -// ✅ Lazy - use CommandLineArgumentProvider to defer resolution -tasks.withType(Test).configureEach { - def fooShadowJarTask = tasks.named('fooShadowJar', ShadowJar) - def barShadowJarTask = tasks.named('barShadowJarTask', ShadowJar) - dependsOn fooShadowJarTask, barShadowJarTask - - jvmArgumentProviders.add(new CommandLineArgumentProvider() { - @Override - Iterable asArguments() { - // This is only called at execution time - return fooShadowJarTask.map { ["-Dsmoketest.foo.path=${it.archiveFile.get()}"] }.get() +// ✅ Only a change to the returned version invalidates the configuration cache +abstract class ReleaseVersionValueSource : + ValueSource { + interface Parameters : ValueSourceParameters { + val releaseFile: RegularFileProperty + } + + override fun obtain(): String = + parameters.releaseFile.get().asFile.reader().use { + Properties().apply { load(it) }.getProperty("version") } - }) - - // Workaround: environment() calls toString() at execution time - environment("BAR_PATH", new Object() { - @Override - String toString() { - return barShadowJarTask.get().archiveFile.get().asFile.absolutePath +} + +val releaseVersion = providers.of(ReleaseVersionValueSource::class) { + parameters.releaseFile.set(releaseFile) +} +``` + +When queried during configuration, `obtain()` is rerun on subsequent builds to check its +result. Changes to other entries leave the returned version unchanged and allow cache reuse. +Built-in providers remain sufficient for simple environment, file and command reads. + +A value source is also a good fit to run a CLI through `ExecOperations`: + +```Gradle Kotlin DSL +import java.io.ByteArrayOutputStream + +abstract class GitVersionValueSource @Inject constructor( + private val execOperations: ExecOperations +) : ValueSource { + override fun obtain(): String { + val output = ByteArrayOutputStream() + execOperations.exec { + commandLine("git", "--version") + standardOutput = output } - }) + return output.toString().trim() + } } + +val gitVersion = providers.of(GitVersionValueSource::class) {} ``` -> [!TIP] -> `CommandLineArgumentProvider` is the recommended way to pass lazily-computed JVM arguments. -> It's configuration-cache compatible and properly tracks inputs for up-to-date checks. However, -> for older APIs like `environment()` that don't accept providers, use the `toString()` wrapper -> trick: pass an anonymous object whose `toString()` method computes the value—it will only be -> called at execution time. +For a simple command, `providers.exec` is shorter. A custom source is useful when combining +command output with other external data. The command runs again when Gradle checks cache +reuse, so slow commands add overhead even when the cached configuration is reused. + +### Lazy JVM Arguments and System Properties + +A common issue is passing arguments immediately by reading task outputs of other tasks or computing values during configuration. + +Instead, a `CommandLineArgumentProvider` supplies JVM arguments from tracked task inputs. +It is applicable to `Test`, `JavaExec`, `Exec`, and any others. +For example, this can be used to provide the output of `Jar` tasks (in the below example +`fooJar` and `barJar`) are needed to pass a system property and an environment var. + +```Gradle Kotlin DSL +// ❌ Eager - reads producer task properties while configuring Test +tasks.withType().configureEach { + val fooJarTask = tasks.named("fooJar") + val barJarTask = tasks.named("barJar") + dependsOn(fooJarTask, barJarTask) + + systemProperty("smoketest.foo.path", fooJarTask.get().archiveFile.get().asFile.absolutePath) + environment("BAR_PATH", barJarTask.get().archiveFile.get().asFile.absolutePath) +} + +abstract class SmokeTestJvmArguments : CommandLineArgumentProvider { + @get:InputFile + @get:PathSensitive(PathSensitivity.NONE) + abstract val fooJar: RegularFileProperty + + override fun asArguments(): Iterable = + listOf("-Dsmoketest.foo.path=${fooJar.get().asFile.absolutePath}") +} + +// ✅ Lazy - tracks the artifacts and reads their paths when needed +tasks.withType().configureEach { + val fooJar = tasks.named("fooJar").flatMap { it.archiveFile } + val barJar = tasks.named("barJar").flatMap { it.archiveFile } + val arguments = objects.newInstance() + arguments.fooJar = fooJar + jvmArgumentProviders.add(arguments) + + // Workaround: environment() currently do not accept providers + inputs.file(barJar).withPropertyName("barJar").withPathSensitivity(PathSensitivity.NONE) + doFirst { + environment("BAR_PATH", barJar.get().asFile.absolutePath) + } +} +``` + +`@InputFile` on `SmokeTestJvmArguments.fooJar` makes the JAR's fingerprint of the +test task's up-to-date checks. Setting that property from the `fooJar` task's output also +schedules `fooJar` before the tests. `inputs.file(barJar)` provides the same tracking and +dependency for `barJar`; `doFirst` sets `BAR_PATH` before the test process starts. + +> [!CAUTION] +> Deferred evaluation alone does not guarantee input tracking or configuration-cache compatibility. +> Argument providers and task actions follow the same +> [configuration-cache requirements](https://docs.gradle.org/current/userguide/configuration_cache_requirements.html). +> +> * An incorrect input on an argument provider coud track path **string** without tracking the file's contents: +> +> ```Gradle Kotlin DSL +> // ❌ Tracks the path string, so changes to the JAR at that path are missed +> abstract class PathOnlyJvmArguments : CommandLineArgumentProvider { +> @get:Input +> abstract val fooJarPath: Property +> +> override fun asArguments(): Iterable = +> listOf("-Dsmoketest.foo.path=${fooJarPath.get()}") +> } +> ``` +> +> The earlier `SmokeTestJvmArguments` properly uses `@InputFile` on a `RegularFileProperty` to track the JAR's contents. +> +> * This assignment is deferred, but does not track the JAR's contents or schedule its producer: +> +> ```Gradle Kotlin DSL +> // ❌ Deferred assignment without a declared file input +> tasks.withType().configureEach { +> val barJar = tasks.named("barJar").flatMap { it.archiveFile } +> doFirst { +> environment("BAR_PATH", barJar.get().asFile.absolutePath) +> } +> } +> ``` +> +> The `inputs.file(barJar)` declaration in the lazy example above provides both. +> +> * Capturing another task in an action also prevents configuration-cache storage: +> +> ```Gradle Kotlin DSL +> // ❌ Deferred lookup that retains the producing task +> tasks.withType().configureEach { +> val fooJarTask = tasks.named("fooJar").get() +> dependsOn(fooJarTask) +> doFirst { +> systemProperty("smoketest.foo.path", fooJarTask.archiveFile.get().asFile.absolutePath) +> } +> } +> ``` +> +> `dependsOn` orders the tasks, but the captured `Jar` task cannot be stored in the +> configuration cache. The lazy example reads the declared `fooJar` file property instead. + +### Replacing `afterEvaluate` + +`afterEvaluate` runs after a project's build script, during configuration. Its callbacks +run in registration order, which makes interactions between plugins fragile. Existing +blocks can often be eliminated with provider wiring, lazy task configuration or +`pluginManager.withPlugin`, depending on what they are waiting for. ### Benefits of Lazy Configuration 1. **Faster configuration**: Only necessary work is performed 2. **Automatic dependency tracking**: Gradle knows which tasks produce values that others consume -3. **Configuration cache compatibility**: Lazy providers are serializable and can be cached +3. **Configuration cache compatibility**: Explicit inputs and compatible provider implementations support configuration cache reuse 4. **Correct ordering**: Task dependencies are inferred from provider relationships ## Gradle Daemon JVM