Skip to content

feat(provider-tck): source the spec artifacts from the open-feature/spec submodule #1838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
aepfli wants to merge 1 commit into from
Draft

feat(provider-tck): source the spec artifacts from the open-feature/spec submodule #1838

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ jobs:
steps:
- name: Checkout Repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
# No submodules: this module's feature files, flags and control-API spec are in-repo.
with:
# The feature files, canonical flag set and control-API document are copied in from
# the open-feature/spec submodule at generate-resources; without it there is no suite.
submodules: recursive

- name: Set up JDK 21
uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,6 @@
[submodule "tools/flagd-api-testkit/test-harness"]
path = tools/flagd-api-testkit/test-harness
url = https://github.com/open-feature/test-harness.git
[submodule "tools/provider-tck/spec"]
path = tools/provider-tck/spec
url = https://github.com/open-feature/spec.git
7 changes: 7 additions & 0 deletions tools/provider-tck/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Copied from the `spec` submodule at build time (mvn generate-resources).
# Do not edit these files directly — they are the language-agnostic definition of the
# provider contract and live in open-feature/spec, under
# specification/assets/provider-tck/ (Appendix F).
src/main/resources/features/
src/main/resources/flags/
src/main/resources/openapi/
37 changes: 27 additions & 10 deletions tools/provider-tck/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,19 +427,36 @@ Three steps are new:
| `When the resolved value is remembered` / `Then the resolved details value should have changed` | the control API only requires that `/change` changes `changing-flag`'s value, not which value it changes to; asserting a delta keeps the scenario vendor-neutral |
| `Then no exception should have been thrown` | makes the "never throws" half of the error contract explicit rather than implicit in a step failure |

## Where these artifacts should live
## Where these artifacts come from

The feature files, the control API spec and the canonical flag set are **not Java artifacts**. They
are language-agnostic definitions of the provider contract that every language's TCK must agree on
byte for byte, and that backend vendors implement in whatever language their testbed is written in.
The feature files, the canonical flag set and the control API document are **not Java artifacts**.
They are language-agnostic definitions of the provider contract that every language's TCK must agree
on byte for byte, and that backend vendors implement in whatever language their testbed is written
in.

They belong in the OpenFeature [spec repository](https://github.com/open-feature/spec), with this
module as their Java delivery vehicle. The three travel together by necessity: a feature file that
evaluates `boolean-flag` is meaningless without the flag definition, and a disconnect scenario is
meaningless without the endpoint that produces the disconnect.
They live in the OpenFeature [spec repository](https://github.com/open-feature/spec) as
[Appendix F: Provider Conformance](https://github.com/open-feature/spec/blob/main/specification/appendix-f-provider-conformance.md),
under `specification/assets/provider-tck/`. This module is their Java delivery vehicle: the `spec`
git submodule is updated at `initialize`, the three directories are copied into
`src/main/resources/` at `generate-resources`, and from there they are packaged into the release
JAR. Consumers see no difference — the features stay on the classpath and need no submodule of their
own.

They live here for now only because the PoC had to start somewhere. Moving them changes nothing for
consumers — the features stay on the classpath and stay inside the JAR.
The three travel together by necessity: a feature file that evaluates `boolean-flag` is meaningless
without the flag definition, and a disconnect scenario is meaningless without the control endpoint
that produces the disconnect.

> **Do not edit `src/main/resources/features/`, `flags/` or `openapi/`.** They are generated and
> git-ignored. Changes belong in `open-feature/spec` and arrive here by bumping the submodule.

Building this module therefore needs the submodule:

```bash
git submodule update --init tools/provider-tck/spec
```

Maven does this itself at `initialize`, so a plain `mvn verify` works from a fresh clone; the
explicit command is only useful when working offline or inspecting the sources by hand.

## Known gaps

Expand Down
149 changes: 120 additions & 29 deletions tools/provider-tck/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,39 +45,32 @@
</developers>

<!--
NOTE ON THE SOURCE OF TRUTH FOR THE SPEC ARTIFACTS
=================================================
Three of the resources in src/main/resources are NOT Java artifacts. They
are language-agnostic definitions of the provider contract, and every
language's TCK must agree on them byte for byte or "conformance" means
nothing:

features/ the canonical Gherkin — the test cases themselves
openapi/control-api.yaml what a backend under test must expose
flags/canonical-flags.json the flag set those test cases assume
SOURCE OF TRUTH FOR THE SPEC ARTIFACTS
======================================
Three of the resources under src/main/resources are NOT Java artifacts.
They are language-agnostic definitions of the provider contract, and
every language's TCK must agree on them byte for byte or "conformance"
means nothing:

They currently live in this module and are packaged into the release JAR,
so consumers load them from the classpath and need no git submodule.

Long term all three belong in the OpenFeature *spec* repository
(open-feature/spec), with this module acting purely as their Java
delivery vehicle. The three travel together by necessity: a feature file
that evaluates `boolean-flag` is meaningless without the flag definition,
and a disconnect scenario is meaningless without the control endpoint
that produces the disconnect.
features/ the canonical Gherkin — the test cases
flags/canonical-flags.json the flag set those test cases assume
openapi/control-api.yaml what a backend under test must expose

When that move happens the packaging mechanism does not change, only the
origin of the files. Wire it exactly like tools/flagd-api-testkit does
today:
They live in the OpenFeature spec repository, as Appendix F
(specification/assets/provider-tck/), and are copied in from the `spec`
git submodule at build time — see the two plugins below. The three
travel together by necessity: a feature file that evaluates
`boolean-flag` is meaningless without the flag definition, and a
disconnect scenario is meaningless without the control endpoint that
produces the disconnect.

1. git submodule add https://github.com/open-feature/spec.git spec
2. exec-maven-plugin @ initialize, running a git submodule update
3. maven-resources-plugin @ generate-resources -> copy
${basedir}/spec/provider-tck/{gherkin,flags,openapi}/
into ${basedir}/src/main/resources/{features,flags,openapi}/
DO NOT EDIT THE COPIES under src/main/resources. They are generated and
git-ignored; changes belong in open-feature/spec and arrive here by
bumping the submodule.

Consumers see no difference: @SelectClasspathResource("features") keeps
working and the JAR keeps shipping all three.
Consumers are unaffected: the copies are packaged into the release JAR,
@SelectClasspathResource("features") keeps working, and nobody needs a
submodule of their own.
-->

<dependencies>
Expand Down Expand Up @@ -199,4 +192,102 @@
</dependency>
</dependencies>

<build>
<plugins>
<!--
Phase 1 – keep the spec submodule up to date. Registered with:
git submodule add https://github.com/open-feature/spec.git tools/provider-tck/spec
-->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.6.3</version>
<executions>
<execution>
<id>update-spec-submodule</id>
<phase>initialize</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>git</executable>
<arguments>
<argument>submodule</argument>
<argument>update</argument>
<argument>--init</argument>
<argument>spec</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>

<!--
Phase 2 – copy the three spec artifacts into src/main/resources so
they are packaged into the release JAR. Consumers load them from the
classpath and need no submodule.
-->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.5.0</version>
<executions>
<execution>
<id>copy-provider-tck-gherkin</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/src/main/resources/features/</outputDirectory>
<resources>
<resource>
<directory>${basedir}/spec/specification/assets/provider-tck/gherkin/</directory>
<includes>
<include>**/*.feature</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>copy-provider-tck-flags</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/src/main/resources/flags/</outputDirectory>
<resources>
<resource>
<directory>${basedir}/spec/specification/assets/provider-tck/flags/</directory>
<includes>
<include>**/*.json</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>copy-provider-tck-openapi</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${basedir}/src/main/resources/openapi/</outputDirectory>
<resources>
<resource>
<directory>${basedir}/spec/specification/assets/provider-tck/openapi/</directory>
<includes>
<include>**/*.yaml</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
1 change: 1 addition & 0 deletions tools/provider-tck/spec
Open in desktop
Submodule spec added at 42b047
80 changes: 0 additions & 80 deletions tools/provider-tck/src/main/resources/features/errors.feature
View file
Open in desktop

This file was deleted.

59 changes: 0 additions & 59 deletions tools/provider-tck/src/main/resources/features/evaluation.feature
View file
Open in desktop

This file was deleted.

Loading
Loading