diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index de0eaf2e4..ad34cb8c8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -25,7 +25,10 @@ jobs: steps: - name: Checkout Repository uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5 - # No submodules: this module's feature files, flags and control-API spec are in-repo. + with: + # The feature files, canonical flag set and control-API document are copied in from + # the open-feature/spec submodule at generate-resources; without it there is no suite. + submodules: recursive - name: Set up JDK 21 uses: actions/setup-java@dded0888837ed1f317902acf8a20df0ad188d165 # v5 diff --git a/.gitmodules b/.gitmodules index 106b0d69f..f2ef1779c 100644 --- a/.gitmodules +++ b/.gitmodules @@ -16,3 +16,6 @@ [submodule "tools/flagd-api-testkit/test-harness"] path = tools/flagd-api-testkit/test-harness url = https://github.com/open-feature/test-harness.git +[submodule "tools/provider-tck/spec"] + path = tools/provider-tck/spec + url = https://github.com/open-feature/spec.git diff --git a/tools/provider-tck/.gitignore b/tools/provider-tck/.gitignore new file mode 100644 index 000000000..4148e2b63 --- /dev/null +++ b/tools/provider-tck/.gitignore @@ -0,0 +1,7 @@ +# Copied from the `spec` submodule at build time (mvn generate-resources). +# Do not edit these files directly — they are the language-agnostic definition of the +# provider contract and live in open-feature/spec, under +# specification/assets/provider-tck/ (Appendix F). +src/main/resources/features/ +src/main/resources/flags/ +src/main/resources/openapi/ diff --git a/tools/provider-tck/README.md b/tools/provider-tck/README.md index 85e252682..e31de0491 100644 --- a/tools/provider-tck/README.md +++ b/tools/provider-tck/README.md @@ -427,19 +427,36 @@ Three steps are new: | `When the resolved value is remembered` / `Then the resolved details value should have changed` | the control API only requires that `/change` changes `changing-flag`'s value, not which value it changes to; asserting a delta keeps the scenario vendor-neutral | | `Then no exception should have been thrown` | makes the "never throws" half of the error contract explicit rather than implicit in a step failure | -## Where these artifacts should live +## Where these artifacts come from -The feature files, the control API spec and the canonical flag set are **not Java artifacts**. They -are language-agnostic definitions of the provider contract that every language's TCK must agree on -byte for byte, and that backend vendors implement in whatever language their testbed is written in. +The feature files, the canonical flag set and the control API document are **not Java artifacts**. +They are language-agnostic definitions of the provider contract that every language's TCK must agree +on byte for byte, and that backend vendors implement in whatever language their testbed is written +in. -They belong in the OpenFeature [spec repository](https://github.com/open-feature/spec), with this -module as their Java delivery vehicle. The three travel together by necessity: a feature file that -evaluates `boolean-flag` is meaningless without the flag definition, and a disconnect scenario is -meaningless without the endpoint that produces the disconnect. +They live in the OpenFeature [spec repository](https://github.com/open-feature/spec) as +[Appendix F: Provider Conformance](https://github.com/open-feature/spec/blob/main/specification/appendix-f-provider-conformance.md), +under `specification/assets/provider-tck/`. This module is their Java delivery vehicle: the `spec` +git submodule is updated at `initialize`, the three directories are copied into +`src/main/resources/` at `generate-resources`, and from there they are packaged into the release +JAR. Consumers see no difference — the features stay on the classpath and need no submodule of their +own. -They live here for now only because the PoC had to start somewhere. Moving them changes nothing for -consumers — the features stay on the classpath and stay inside the JAR. +The three travel together by necessity: a feature file that evaluates `boolean-flag` is meaningless +without the flag definition, and a disconnect scenario is meaningless without the control endpoint +that produces the disconnect. + +> **Do not edit `src/main/resources/features/`, `flags/` or `openapi/`.** They are generated and +> git-ignored. Changes belong in `open-feature/spec` and arrive here by bumping the submodule. + +Building this module therefore needs the submodule: + +```bash +git submodule update --init tools/provider-tck/spec +``` + +Maven does this itself at `initialize`, so a plain `mvn verify` works from a fresh clone; the +explicit command is only useful when working offline or inspecting the sources by hand. ## Known gaps diff --git a/tools/provider-tck/pom.xml b/tools/provider-tck/pom.xml index ee5d7ce9e..5d350dd6b 100644 --- a/tools/provider-tck/pom.xml +++ b/tools/provider-tck/pom.xml @@ -45,39 +45,32 @@ @@ -199,4 +192,102 @@ + + + + + org.codehaus.mojo + exec-maven-plugin + 3.6.3 + + + update-spec-submodule + initialize + + exec + + + git + + submodule + update + --init + spec + + + + + + + + + maven-resources-plugin + 3.5.0 + + + copy-provider-tck-gherkin + generate-resources + + copy-resources + + + ${basedir}/src/main/resources/features/ + + + ${basedir}/spec/specification/assets/provider-tck/gherkin/ + + **/*.feature + + + + + + + copy-provider-tck-flags + generate-resources + + copy-resources + + + ${basedir}/src/main/resources/flags/ + + + ${basedir}/spec/specification/assets/provider-tck/flags/ + + **/*.json + + + + + + + copy-provider-tck-openapi + generate-resources + + copy-resources + + + ${basedir}/src/main/resources/openapi/ + + + ${basedir}/spec/specification/assets/provider-tck/openapi/ + + **/*.yaml + + + + + + + + + + diff --git a/tools/provider-tck/spec b/tools/provider-tck/spec new file mode 160000 index 000000000..42b047b9f --- /dev/null +++ b/tools/provider-tck/spec @@ -0,0 +1 @@ +Subproject commit 42b047b9fb1aa37458ec4a3dff2340b188710aff diff --git a/tools/provider-tck/src/main/resources/features/errors.feature b/tools/provider-tck/src/main/resources/features/errors.feature deleted file mode 100644 index 0346df3da..000000000 --- a/tools/provider-tck/src/main/resources/features/errors.feature +++ /dev/null @@ -1,80 +0,0 @@ -Feature: Provider error handling - - # Every scenario here asserts the same three-part contract, because all three parts matter and - # providers routinely get one of them wrong: - # - # 1. the code default is returned — an application must keep working, - # 2. the correct error code is reported — an application must be able to tell what went wrong, - # 3. nothing is thrown — an unhandled exception from a flag evaluation is never acceptable. - # - # Requires the backend to be seeded with the canonical flag set — see flags/canonical-flags.json. - - Background: - Given a stable provider - - Scenario Outline: Requesting the wrong type returns the code default - # The full non-numeric mismatch matrix. Numeric coercion is a separate question and is covered - # by the @strict-numeric-typing scenarios below, because "is 0.5 an integer?" has a defensible - # wrong answer whereas "is a string a boolean?" does not. - Given a -flag with key "" and a default value "" - When the flag was evaluated with details - Then the resolved details value should be "" - And the reason should be "ERROR" - And the error-code should be "TYPE_MISMATCH" - And no exception should have been thrown - - Examples: a string flag requested as something else - | key | requested | default | - | string-flag | Boolean | false | - | string-flag | Integer | 1 | - | string-flag | Float | 0.1 | - | wrong-flag | Boolean | false | - - Examples: a boolean flag requested as something else - | key | requested | default | - | boolean-flag | String | fallback | - | boolean-flag | Integer | 1 | - | boolean-flag | Float | 0.1 | - - Examples: a numeric flag requested as a non-numeric type - | key | requested | default | - | integer-flag | Boolean | false | - | integer-flag | String | fallback | - | float-flag | Boolean | false | - | float-flag | String | fallback | - - @object - Scenario Outline: Requesting a structured flag as a scalar returns the code default - Given a -flag with key "object-flag" and a default value "" - When the flag was evaluated with details - Then the resolved details value should be "" - And the reason should be "ERROR" - And the error-code should be "TYPE_MISMATCH" - And no exception should have been thrown - - Examples: - | requested | default | - | Boolean | false | - | String | fallback | - | Integer | 1 | - | Float | 0.1 | - - @strict-numeric-typing - Scenario: A float flag is not silently narrowed to an integer - # 'float-flag' resolves to 0.5. Narrowing that to an integer would lose information - # silently, so it must be reported as a type mismatch rather than rounded. - Given a Integer-flag with key "float-flag" and a default value "1" - When the flag was evaluated with details - Then the resolved details value should be "1" - And the reason should be "ERROR" - And the error-code should be "TYPE_MISMATCH" - And no exception should have been thrown - - Scenario: An unknown flag key returns the code default - # 'missing-flag' is deliberately absent from the canonical flag set. - Given a String-flag with key "missing-flag" and a default value "fallback" - When the flag was evaluated with details - Then the resolved details value should be "fallback" - And the reason should be "ERROR" - And the error-code should be "FLAG_NOT_FOUND" - And no exception should have been thrown diff --git a/tools/provider-tck/src/main/resources/features/evaluation.feature b/tools/provider-tck/src/main/resources/features/evaluation.feature deleted file mode 100644 index e89f174a5..000000000 --- a/tools/provider-tck/src/main/resources/features/evaluation.feature +++ /dev/null @@ -1,59 +0,0 @@ -Feature: Provider flag evaluation - - # Verifies that a provider maps backend responses onto typed resolution details correctly. - # - # This does NOT test the backend's evaluation logic. Every flag in the canonical set resolves - # to its default variant with no targeting involved, so what is under test is purely the - # provider's mapping of a backend response to a value, a variant and a reason. - # - # Requires the backend to be seeded with the canonical flag set — see flags/canonical-flags.json. - - Background: - Given a stable provider - - Scenario Outline: Resolve values with variant and reason - Given a -flag with key "" and a default value "" - When the flag was evaluated with details - Then the resolved details value should be "" - And the variant should be "" - And the reason should be "" - And the error-code should be "" - And no exception should have been thrown - - Examples: - | key | type | default | value | variant | reason | - | boolean-flag | Boolean | false | true | on | STATIC | - | string-flag | String | bye | hi | greeting | STATIC | - | integer-flag | Integer | 1 | 10 | ten | STATIC | - | float-flag | Float | 0.1 | 0.5 | half | STATIC | - - Scenario: An integer flag resolves as an integer - # Paired with the float scenario below and with the narrowing scenario in errors.feature. - # Together they pin down that the two numeric types stay distinct rather than both being - # funnelled through one numeric representation. - Given a Integer-flag with key "integer-flag" and a default value "1" - When the flag was evaluated with details - Then the resolved details value should be "10" - And the error-code should be "" - And no exception should have been thrown - - Scenario: A float flag resolves as a float - Given a Float-flag with key "float-flag" and a default value "0.1" - When the flag was evaluated with details - Then the resolved details value should be "0.5" - And the error-code should be "" - And no exception should have been thrown - - @object - Scenario: Resolve a structured value - Given a Object-flag with key "object-flag" and a default value "{}" - When the flag was evaluated with details - Then the variant should be "template" - And the reason should be "STATIC" - And the error-code should be "" - And no exception should have been thrown - And the resolved object value should contain - | key | type | value | - | showImages | Boolean | true | - | title | String | Check out these pics! | - | imagesPerPage | Integer | 100 | diff --git a/tools/provider-tck/src/main/resources/features/events.feature b/tools/provider-tck/src/main/resources/features/events.feature deleted file mode 100644 index 00e7e5ef6..000000000 --- a/tools/provider-tck/src/main/resources/features/events.feature +++ /dev/null @@ -1,42 +0,0 @@ -@events -Feature: Provider events - - # Verifies that a provider notices changes in its backend and both signals them and acts on - # them. Signalling alone is not enough: a configuration-change event that is not followed by - # a changed evaluation result is a lie, so each scenario asserts the event AND the behaviour. - # - # Outages here are simulated inside the running stack via the control API. No container is - # ever stopped or restarted — see the invariant in openapi/control-api.yaml. - - Background: - Given a stable provider - - @configuration-change - Scenario: A configuration change is signalled and applied - Given a String-flag with key "changing-flag" and a default value "unset" - And a change event handler - When the flag was evaluated with details - And the resolved value is remembered - And the flag was modified - Then the change event handler should have been executed - And the flag should be part of the event payload - When the flag was evaluated with details - Then the resolved details value should have changed - And no exception should have been thrown - - @stale - Scenario: Losing the backend makes the provider stale, regaining it makes it ready again - Given a ready event handler - And a stale event handler - When a ready event was fired - And the connection is lost - Then the stale event handler should have been executed - And the client should be in stale state - When the connection is restored - Then the ready event handler should have been executed - And the client should be in ready state - - # Deliberately NOT covered here: whether a stale provider keeps serving last-known values - # during the outage. That is caching behaviour, which depends on whether the provider holds a - # local copy of the ruleset, and it belongs behind the @caching capability once those - # scenarios are written. See the "Known gaps" section of the README. diff --git a/tools/provider-tck/src/main/resources/features/lifecycle.feature b/tools/provider-tck/src/main/resources/features/lifecycle.feature deleted file mode 100644 index 256164106..000000000 --- a/tools/provider-tck/src/main/resources/features/lifecycle.feature +++ /dev/null @@ -1,33 +0,0 @@ -@events -Feature: Provider lifecycle - - # Verifies the two terminal outcomes of provider initialisation: reaching READY against a - # healthy backend, and settling into ERROR against one that cannot be reached. - # - # The failure case matters more than it looks. A provider that blocks forever, or throws out - # of provider registration, takes the host application down with it — so the requirement is - # not merely that initialisation fails, but that it fails observably and promptly. - - Scenario: A provider reaching its backend becomes ready - Given a stable provider - And a ready event handler - Then the ready event handler should have been executed - And the client should be in ready state - - @unavailable - Scenario: A provider that cannot reach its backend reports an error - Given a unavailable provider - And a error event handler - Then the error event handler should have been executed within 10000ms - And the client should be in error state - - @unavailable - Scenario: A provider that cannot reach its backend still returns code defaults - Given a unavailable provider - And a error event handler - And a Boolean-flag with key "boolean-flag" and a default value "false" - Then the error event handler should have been executed within 10000ms - When the flag was evaluated with details - Then the resolved details value should be "false" - And the reason should be "ERROR" - And no exception should have been thrown diff --git a/tools/provider-tck/src/main/resources/flags/canonical-flags.json b/tools/provider-tck/src/main/resources/flags/canonical-flags.json deleted file mode 100644 index 343b3ae52..000000000 --- a/tools/provider-tck/src/main/resources/flags/canonical-flags.json +++ /dev/null @@ -1,82 +0,0 @@ -{ - "$comment": [ - "The canonical flag set the TCK's feature files assume. A backend under test MUST serve an", - "equivalent set under the configuration named 'default'.", - "", - "Expressed in the flagd flag-definition format because that is the only widely implemented", - "vendor-neutral format today. The format is not what matters — the keys, types, variant", - "names and resolved values are. Seed them however your backend seeds flags.", - "", - "Two things are load-bearing and easy to get wrong:", - " * 'missing-flag' MUST NOT exist. Its absence is what the FLAG_NOT_FOUND scenario tests.", - " * No flag here has targeting rules. Every scenario expects reason STATIC, because the TCK", - " tests the provider's mapping of a response, not the backend's evaluation logic." - ], - "flags": { - "boolean-flag": { - "state": "ENABLED", - "variants": { - "on": true, - "off": false - }, - "defaultVariant": "on" - }, - "string-flag": { - "state": "ENABLED", - "variants": { - "greeting": "hi", - "parting": "bye" - }, - "defaultVariant": "greeting" - }, - "integer-flag": { - "state": "ENABLED", - "variants": { - "one": 1, - "ten": 10 - }, - "defaultVariant": "ten" - }, - "float-flag": { - "state": "ENABLED", - "variants": { - "tenth": 0.1, - "half": 0.5 - }, - "defaultVariant": "half" - }, - "object-flag": { - "state": "ENABLED", - "variants": { - "empty": {}, - "template": { - "showImages": true, - "title": "Check out these pics!", - "imagesPerPage": 100 - } - }, - "defaultVariant": "template" - }, - "wrong-flag": { - "$comment": "A string flag, evaluated as a boolean by the TYPE_MISMATCH scenario.", - "state": "ENABLED", - "variants": { - "one": "uno", - "two": "dos" - }, - "defaultVariant": "one" - }, - "changing-flag": { - "$comment": [ - "The flag POST /change mutates. The TCK asserts only that its resolved value differs", - "after the change, so which of the two variants you start from does not matter." - ], - "state": "ENABLED", - "variants": { - "foo": "foo", - "bar": "bar" - }, - "defaultVariant": "foo" - } - } -} diff --git a/tools/provider-tck/src/main/resources/openapi/control-api.yaml b/tools/provider-tck/src/main/resources/openapi/control-api.yaml deleted file mode 100644 index fd9bc7000..000000000 --- a/tools/provider-tck/src/main/resources/openapi/control-api.yaml +++ /dev/null @@ -1,368 +0,0 @@ -openapi: 3.0.3 - -info: - title: OpenFeature Provider TCK — Backend Control API - version: 0.0.1 - description: | - The control API that a **backend under test** must expose so the OpenFeature - Provider TCK can drive it. - - The TCK verifies the *provider contract*: how a provider maps backend - responses to typed resolution details, lifecycle states and events. To do - that it must be able to put the backend into specific states on demand — - running, unreachable, reconfigured. This document standardises how. - - This specification is derived from the control endpoints already implemented - by [`flagd-testbed`](https://github.com/open-feature/flagd-testbed)'s - "launchpad" server, which is the reference implementation. - - ## Where this document should live - - This file currently ships inside the Java `provider-tck` artifact, but it is - not a Java artifact: it is a language-agnostic contract that every language's - TCK must implement identically, and that backend vendors implement in - whatever language their testbed is written in (Go, for flagd). - - It therefore belongs in the OpenFeature **spec** repository - (`open-feature/spec`), alongside the canonical Gherkin feature files and the - canonical flag set. Those three artifacts are a single unit — a feature file - that evaluates `boolean-flag` is meaningless without the flag definition, and - a disconnect scenario is meaningless without the endpoint that produces the - disconnect. Splitting them across repositories would let them drift. - - Each language's TCK then vendors the spec repo (git submodule or equivalent) - and packages these files into its own distribution format, so that adopting a - TCK never requires a consumer to check out a submodule of their own. - - ## Conformance language - - The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT and MAY are to be - interpreted as described in RFC 2119. - - Each operation below is tagged **REQUIRED** or **OPTIONAL**. A backend that - implements every REQUIRED operation can run the full TCK. OPTIONAL operations - have a defined fallback that the TCK applies automatically, so omitting them - costs nothing but precision. - - --- - - ## Normative requirement 1 — the no-container-restart invariant - - > **Container lifecycle operations MUST NOT be used to simulate backend - > unavailability. Backend unavailability MUST be simulated from inside the - > running stack.** - - The TCK starts the vendor's Docker Compose stack **once per test suite** and - reads the dynamically mapped host ports. Testcontainers cannot reliably - preserve mapped ports across a container stop/start in all language - bindings — a restarted container generally comes back on a *different* host - port, which silently invalidates every provider instance already pointed at - the old one. Any TCK implementation in any language hits this, so the - constraint is part of the contract rather than a Java detail. - - Therefore an implementation of `/stop`, `/restart` or any other outage - simulation MUST achieve the outage by one of: - - * killing or suspending the backend **process** inside its container - (the reference behaviour — this is what flagd-testbed does); - * a proxy in the stack refusing or blackholing connections - (e.g. a toxiproxy toxic, an envoy `direct_response`); - * an in-container firewall or socket-level block. - - An implementation MUST NOT `docker stop`, `docker kill`, `docker rm` or - recreate any container in the stack while the suite is running. The stack is - brought up before the first scenario and torn down after the last one, and - the mapped ports MUST remain stable for that entire window. - - --- - - ## Normative requirement 2 — flag state semantics across outages - - Outage simulation and flag-state seeding are orthogonal, and the TCK relies - on that separation for scenario isolation: - - * `POST /start` **MUST** (re)seed flag state to the baseline defined by the - named configuration. Any mutation previously applied by `POST /change` - MUST be discarded. This is what makes `/start` usable as a reset. - * `POST /restart` and a `POST /stop` followed by a `POST /start` **of the - same configuration** MUST leave the backend serving the same baseline - flag state it served before the outage. An outage MUST NOT be observable - as a change in flag *values* — only as a change in *availability*. - * `POST /change` mutations persist until the next `/start` or `/reset`. - - --- - - ## Normative requirement 3 — compose stack conventions - - The backend under test is delivered as a **Docker Compose stack**, not a - single image, so vendors can compose proxies, edge services or several - containers. The TCK only relies on these conventions: - - * One service — by default named `backend`, overridable by the provider - author — exposes the control API on container-internal port `8080` - (also overridable). - * The same stack exposes whatever port(s) the provider connects to. - * **All external ports are dynamically mapped.** A stack MUST NOT pin host - ports; the TCK discovers them after startup and hands them to the - provider factory. - * The stack MAY contain any number of additional services. - - --- - - ## Known gap — evaluation context passthrough - - There is currently no operation for asserting that an evaluation context sent - by the provider actually reached the backend intact. Verifying that requires - an echo mechanism (e.g. `GET /last-evaluation` returning the most recent - request the backend received). Until such an operation exists, context - passthrough is out of scope for the TCK. - - license: - name: Apache 2.0 - url: https://www.apache.org/licenses/LICENSE-2.0 - -servers: - - url: http://{host}:{port} - description: | - Resolved at runtime from the Compose stack. `host` is the Docker host and - `port` is the dynamically mapped host port for the control service's - internal port 8080. - variables: - host: - default: localhost - port: - default: "8080" - -tags: - - name: lifecycle - description: Start and stop the backend process. - - name: availability - description: Simulate outages without touching containers. - - name: flags - description: Seed and mutate flag configuration. - - name: health - description: Readiness of the control API itself. - -paths: - - /start: - post: - tags: [lifecycle] - operationId: start - summary: "[REQUIRED] Start the backend and seed flags to a named baseline" - description: | - Starts the backend process using the named configuration and seeds flag - state to that configuration's baseline. - - MUST be idempotent in the sense that calling it while the backend is - already running is not an error: the implementation restarts the process - (or otherwise ensures it is running) with the requested configuration. - - Because this operation resets flag state, the TCK uses it as its default - scenario-isolation mechanism when `/reset` is not implemented. - - The set of valid configuration names is vendor-defined. Every - implementation MUST support the name `default`, which MUST serve the - canonical flag set the TCK's feature files assume. - - Reference implementation: flagd-testbed launches the `flagd` binary with - the config file of that name from `launchpad/configs` and rewrites - `/flags/allFlags.json`. - parameters: - - name: config - in: query - required: false - description: | - Name of the configuration to start with. Defaults to `default`. - schema: - type: string - default: default - example: default - responses: - "200": - description: Backend started and flag state seeded. - "400": - description: Unknown configuration name. - content: - application/json: - schema: - $ref: "#/components/schemas/Error" - - /stop: - post: - tags: [availability] - operationId: stop - summary: "[REQUIRED] Make the backend unreachable" - description: | - Makes the backend unreachable to the provider, simulating an outage. - - **MUST NOT stop the container.** See normative requirement 1. The - reference implementation kills the flagd process while its container - keeps running. - - The backend stays unreachable until a subsequent `POST /start`. Calling - `/stop` when the backend is already stopped MUST succeed. - - The TCK uses this to drive providers into `STALE` and `ERROR` states and - to assert `PROVIDER_STALE` / `PROVIDER_ERROR` events. - responses: - "200": - description: Backend is now unreachable; container still running. - - /restart: - post: - tags: [availability] - operationId: restart - summary: "[REQUIRED] Simulate an outage of a bounded duration" - description: | - Makes the backend unreachable, waits `seconds`, then starts it again with - the configuration currently in effect. - - Flag state MUST be preserved across the outage — see normative - requirement 2. This is what distinguishes `/restart` from - `/stop` + `/start`: the former is an availability event, the latter is - also a reset. - - This operation MAY return as soon as the outage has begun rather than - blocking for the full duration; the TCK does not rely on the response - being delayed. It awaits provider events instead. - - The TCK uses this for the disconnect/reconnect scenarios: `STALE` → - `PROVIDER_STALE`, then back to `READY` → `PROVIDER_READY`. - parameters: - - name: seconds - in: query - required: false - description: | - How long the backend stays unreachable. Defaults to 5. - - Providers differ enormously in how fast they notice an outage — - a streaming provider may see it in milliseconds while a polling - provider needs up to a full poll interval. Feature files therefore - parameterise this value and provider authors tune the matching - await timeouts. - schema: - type: integer - format: int32 - minimum: 0 - default: 5 - example: 5 - responses: - "200": - description: Outage started (and, for blocking implementations, ended). - - /change: - post: - tags: [flags] - operationId: change - summary: "[REQUIRED] Mutate flag configuration so the provider observes a change" - description: | - Mutates the flag configuration such that a conforming provider observes a - configuration change and, on re-evaluation, resolves a **different value** - for the affected flag. - - The implementation MUST: - - * change the resolved value of the flag with key `changing-flag`; - * do so without restarting the backend process, so that a provider sees - a configuration-change signal rather than a reconnect; - * make the change durable until the next `/start` or `/reset`. - - The implementation SHOULD toggle between exactly two known values so that - repeated calls are meaningful and the test remains deterministic - regardless of how many times it has run against the same stack. The - reference implementation toggles `changing-flag`'s `defaultVariant` - between `foo` and `bar`. - - The TCK uses this to assert `PROVIDER_CONFIGURATION_CHANGED`, that the - changed flag key appears in the event payload, and that a subsequent - evaluation returns the new value. - responses: - "200": - description: Flag configuration mutated. - - /reset: - post: - tags: [flags] - operationId: reset - summary: "[OPTIONAL] Restore the seeded baseline without an outage" - description: | - Restores flag state to the baseline of the configuration currently in - effect, discarding any mutation applied by `/change`, **without** making - the backend unreachable at any point. - - This is the preferred scenario-isolation primitive: unlike `/start` it - causes no availability blip, so it cannot inject spurious lifecycle - events into the next scenario. - - **Scope.** This operation resets flag state only. It MUST NOT be - expected to start a backend that is currently stopped — that is what - `/start` is for. A TCK therefore uses `/reset` only when the backend is - known to be running, and `/start` otherwise. The reference client tracks - this: `/stop` and `/restart` mark the backend as possibly-unreachable, so - the scenario that follows either of them is prepared with `/start`. - - **Fallback when not implemented.** A backend that does not implement this - operation MUST respond `404` or `501`. The TCK then falls back to - `POST /start?config={defaultConfig}`, which resets flag state at the cost - of a process restart. The fallback is detected once per suite and cached. - - Implementing `/reset` is RECOMMENDED for providers whose reconnect - behaviour makes the `/start` blip hard to distinguish from a real event. - responses: - "200": - description: Flag state restored to the baseline. - "404": - description: Not implemented; the TCK falls back to `/start`. - "501": - description: Not implemented; the TCK falls back to `/start`. - - /healthz: - get: - tags: [health] - operationId: health - summary: "[OPTIONAL] Readiness of the control API" - description: | - Reports whether the control API is ready to accept commands. - - **Fallback when not implemented.** Readiness defaults to "the control - port accepts a TCP connection", which the TCK establishes with a - Testcontainers listening-port wait strategy before the first scenario. A - `404` here is therefore not a failure, and the reference implementation - does not serve this path. - - Note this reports the health of the **control API**, not of the backend. - The backend is deliberately unhealthy during outage scenarios while the - control API must stay reachable — otherwise the TCK could not end the - outage. - responses: - "200": - description: Control API ready. - content: - application/json: - schema: - $ref: "#/components/schemas/Health" - "404": - description: Not implemented; readiness falls back to a TCP port check. - "503": - description: Control API not ready yet. - -components: - schemas: - - Health: - type: object - properties: - status: - type: string - enum: [ok] - description: Present and equal to `ok` when the control API is ready. - required: [status] - - Error: - type: object - properties: - message: - type: string - description: Human-readable explanation. Never interpreted by the TCK. - required: [message]