From c2558bbae8500fd9349a18b84d119931bdebe327 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Sat, 12 Sep 2026 12:15:42 +0200 Subject: [PATCH 01/11] test(flagsmith): run the provider conformance suite against the testbed Experimental adoption. 52 scenarios: 20 pass, 12 fail, 20 skipped. Eight of the twelve failures are 'expected STATIC but was null' -- this provider never populates the resolution reason. The values are all correct; only the reason is missing. 2.2.5 makes it a SHOULD, so null is arguably permitted, but the Go and Python Flagsmith providers both populate it against the identical backend. Two more are float resolution. Flagsmith stores floats as strings, because feature_state_value is natively boolean, integer or string only. Go's provider parses the string back; this one type-checks and falls back to the code default, so GetFloatValue never works against Flagsmith. The last two are shared with every other language: float-flag and object-flag requested as a String succeed, because on this backend both really are strings. @large-integers is withheld deliberately: Java's accessor is a 32-bit Integer, so 2^53-1 cannot be asked for. Same reason Java withholds it for flagd, and it accounts for the extra skip against Go's 19. The Jackson pin is a workaround for a TCK-introduced conflict, not a provider defect: provider-tck exports jackson-databind 2.22.1 while flagsmith-java-client pins jackson-annotations 2.15.2, and the client's ObjectMapper then dies on JsonSerializeAs. Verified by removing the TCK dependency, after which the provider's own 35 tests pass. Signed-off-by: Simon Schrottner --- providers/flagsmith/pom.xml | 35 ++++++ .../flagsmith/FlagsmithProviderTckTest.java | 119 ++++++++++++++++++ .../resources/flagsmith-testbed-compose.yaml | 14 +++ 3 files changed, 168 insertions(+) create mode 100644 providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java create mode 100644 providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml diff --git a/providers/flagsmith/pom.xml b/providers/flagsmith/pom.xml index f73f689d9e..92fad51124 100644 --- a/providers/flagsmith/pom.xml +++ b/providers/flagsmith/pom.xml @@ -25,7 +25,42 @@ + + + [0.0.1,) + + + + + com.fasterxml.jackson.core + jackson-databind + 2.15.2 + + + dev.openfeature.contrib.tools + provider-tck + ${provider-tck.version} + test + + + org.testcontainers + testcontainers + 2.0.4 + test + com.flagsmith diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java new file mode 100644 index 0000000000..8bcd5a9619 --- /dev/null +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -0,0 +1,119 @@ +package dev.openfeature.contrib.providers.flagsmith; + +import dev.openfeature.contrib.tools.providertck.BackendEndpoint; +import dev.openfeature.contrib.tools.providertck.Capability; +import dev.openfeature.contrib.tools.providertck.ContainerizedProviderTckTest; +import dev.openfeature.contrib.tools.providertck.KnownDeviation; +import dev.openfeature.sdk.FeatureProvider; +import java.io.File; +import java.util.Arrays; +import java.util.EnumSet; +import java.util.List; +import java.util.Set; + +/** + * The OpenFeature Provider Conformance Suite against the Flagsmith Java provider. + * + *

An experiment rather than a finished adoption. The point is comparison: the Go adoption + * (go-sdk-contrib#959) reports 31 pass / 2 fail / 19 skip, and the Python provider -- which lives + * in the Flagsmith organisation rather than in a contrib repo -- reports 28 / 5 / 19 against the + * same container. Three providers written by different people against the same backend API is + * exactly the situation the cross-language suite exists for, and the Go and Python results already + * disagree. + * + *

The backend is the same container both of those use: + * aepfli/flagsmith-tck-testbed, the + * Flagsmith Edge Proxy with a launchpad implementing the control API. Nothing about it is + * language-specific -- it is a container with an HTTP control API, which is the point of the + * control API existing. + */ +public class FlagsmithProviderTckTest extends ContainerizedProviderTckTest { + + /** + * Fixed by the testbed. The control API has no way to communicate connection parameters -- + * {@code POST /start} returns a bare 200 with no body -- so every adoption hardcodes these, + * exactly as a flagd adoption hardcodes a port. + */ + private static final String SERVER_SIDE_KEY = "ser.provider-tck-server-key"; + + private static final int PROXY_PORT = 8000; + + @Override + public File composeFile() { + return new File("src/test/resources/flagsmith-testbed-compose.yaml"); + } + + @Override + public List backendPorts() { + return Arrays.asList(PROXY_PORT); + } + + @Override + public FeatureProvider createProvider(BackendEndpoint endpoint) { + // The Flagsmith SDK appends its own path segments, so baseUri is the API root with a + // trailing slash: remote evaluation requests "flags/" beneath it. + String baseUri = String.format( + "http://%s:%d/api/v1/", endpoint.host(), endpoint.port(PROXY_PORT)); + + FlagsmithProviderOptions options = FlagsmithProviderOptions.builder() + .apiKey(SERVER_SIDE_KEY) + .baseUri(baseUri) + // usingBooleanConfigValue is the setting the three languages disagree about. + // + // Java and Go both default it to false, meaning a boolean flag resolves from + // Flagsmith's feature_state_value. The Python provider defaults the OPPOSITE way: + // it reads the `enabled` state and treats feature_state_value as opt-in. + // + // Set explicitly here, because the canonical set models booleans as values -- every + // flag is seeded enabled, so reading `enabled` would resolve boolean-zero-flag to + // true, which is exactly what the falsy-value scenario catches. + .usingBooleanConfigValue(true) + .build(); + return new FlagsmithProvider(options); + } + + @Override + public FeatureProvider createUnavailableProvider() { + // Pointed at a closed port on localhost, never at the backend under test -- that has to + // stay up, and simulated outages belong to the control API. + FlagsmithProviderOptions options = FlagsmithProviderOptions.builder() + .apiKey(SERVER_SIDE_KEY) + .baseUri("http://localhost:9999/api/v1/") + .usingBooleanConfigValue(true) + .build(); + return new FlagsmithProvider(options); + } + + /** + * Predictions, to be corrected by the run. + * + *

{@code VARIANTS} is withheld for the reason the Go adoption established: Flagsmith has no + * variant concept for a plain feature, the evaluation response carries no variant key, and no + * seeding can produce one. That is permitted rather than defective -- 2.2.4 makes populating + * the variant a SHOULD -- so it carries no deviation entry. + * + *

{@code LARGE_INTEGERS} is withheld here and declared in Go, and the difference is real + * rather than an oversight: Java's integer accessor is a 32-bit {@code Integer}, so 2^53-1 + * cannot be asked for at all. This is the same reason Java withholds it for flagd. + * + *

The lifecycle and event capabilities are withheld pending the run. Go's provider + * implements no {@code StateHandler} whatsoever; whether Java's does is the first thing this + * run answers, and declaring them afterwards is the correct follow-up. Withholding a capability + * a provider genuinely has is the expensive mistake, because it makes the suite blind to it. + */ + @Override + public Set capabilities() { + return EnumSet.of(Capability.OBJECT, Capability.TARGETING); + } + + @Override + public List knownDeviations() { + return Arrays.asList(KnownDeviation.untracked( + Capability.NUMERIC_COERCION, + "Withheld pending the run, and recorded as a prediction rather than a measurement. " + + "Flagsmith stores floats and objects as strings because feature_state_value is " + + "natively boolean, integer or string only. Go compensates by parsing the string in " + + "its Float accessor and Python does not, so Python cannot read a Flagsmith float at " + + "all. Which of the two Java resembles is what this run is for.")); + } +} diff --git a/providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml b/providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml new file mode 100644 index 0000000000..a154fff322 --- /dev/null +++ b/providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml @@ -0,0 +1,14 @@ +# The Flagsmith provider-TCK backend: the Flagsmith Edge Proxy with a launchpad implementing the +# control API as its upstream. https://github.com/aepfli/flagsmith-tck-testbed +# +# The service is named "backend" because that is ContainerizedProviderTckTest's default. +# +# Ports are never pinned. The TCK reads the dynamically mapped host ports after the stack is up, +# and POST /stop deliberately never restarts the container precisely so those mappings survive an +# outage scenario. +services: + backend: + image: ${FLAGSMITH_TESTBED_IMAGE:-ghcr.io/aepfli/flagsmith-tck-testbed:latest} + ports: + - 8000 # Edge Proxy -- what the provider under test talks to + - 8080 # launchpad -- control API From 77220c342b83ee49aea71b3ee09282c49f6109c0 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Sat, 12 Sep 2026 15:02:57 +0200 Subject: [PATCH 02/11] test(flagsmith): declare @disabled-flags after rebasing Flagsmith's native model is `enabled` plus a value, so the canonical set's four disabled-* flags map straight onto it. The testbed grew the flags in the same pass. 56 scenarios: 24 pass, 12 fail, 20 skip. The four new scenarios pass; the twelve failures are unchanged. Signed-off-by: Simon Schrottner --- .../providers/flagsmith/FlagsmithProviderTckTest.java | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 8bcd5a9619..21ef3cd1aa 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -96,6 +96,9 @@ public FeatureProvider createUnavailableProvider() { * rather than an oversight: Java's integer accessor is a 32-bit {@code Integer}, so 2^53-1 * cannot be asked for at all. This is the same reason Java withholds it for flagd. * + *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a + * value, so the canonical set's four disabled-* flags map straight onto it. + * *

The lifecycle and event capabilities are withheld pending the run. Go's provider * implements no {@code StateHandler} whatsoever; whether Java's does is the first thing this * run answers, and declaring them afterwards is the correct follow-up. Withholding a capability @@ -103,7 +106,7 @@ public FeatureProvider createUnavailableProvider() { */ @Override public Set capabilities() { - return EnumSet.of(Capability.OBJECT, Capability.TARGETING); + return EnumSet.of(Capability.OBJECT, Capability.TARGETING, Capability.DISABLED_FLAGS); } @Override From 414bbd0280e370933e8319424570c4aa9ccb15a7 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Sat, 12 Sep 2026 21:22:28 +0200 Subject: [PATCH 03/11] test(flagsmith): follow the TCK rename to tools/tck The tool moved from tools/provider-tck to tools/tck and its package from dev.openfeature.contrib.tools.providertck to dev.openfeature.contrib.tools.tck. Dependency and imports follow. The explicit testcontainers dependency is now load-bearing rather than redundant: the TCK made its own provided and optional, so an adopter declares it. The Jackson pin stays, and was re-tested rather than assumed -- removing it still fails with ClassNotFoundException on JsonSerializeAs, so the TCK still exports databind 2.22.1 against the client's annotations 2.15.2. The comment is corrected: it is compile scope and has to be, because the provider's own main source imports com.fasterxml.jackson.databind. Saying 'test scope only' was wrong. 56 scenarios: 24 pass, 12 fail, 20 skip. Unchanged by the rebase. Signed-off-by: Simon Schrottner --- providers/flagsmith/pom.xml | 18 ++++++++++-------- .../flagsmith/FlagsmithProviderTckTest.java | 8 ++++---- 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/providers/flagsmith/pom.xml b/providers/flagsmith/pom.xml index 92fad51124..54dd3aa112 100644 --- a/providers/flagsmith/pom.xml +++ b/providers/flagsmith/pom.xml @@ -31,19 +31,21 @@ - + Aligned DOWN to the client's 2.15.2 rather than up, because jackson-annotations has no + 2.22.1 release at all (it tracks its own version line), so aligning upward needs the + Jackson BOM. Compile scope, not test: the provider's own main source imports + com.fasterxml.jackson.databind. --> com.fasterxml.jackson.core jackson-databind @@ -51,7 +53,7 @@ dev.openfeature.contrib.tools - provider-tck + tck ${provider-tck.version} test diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 21ef3cd1aa..6fdbb3be35 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -1,9 +1,9 @@ package dev.openfeature.contrib.providers.flagsmith; -import dev.openfeature.contrib.tools.providertck.BackendEndpoint; -import dev.openfeature.contrib.tools.providertck.Capability; -import dev.openfeature.contrib.tools.providertck.ContainerizedProviderTckTest; -import dev.openfeature.contrib.tools.providertck.KnownDeviation; +import dev.openfeature.contrib.tools.tck.BackendEndpoint; +import dev.openfeature.contrib.tools.tck.Capability; +import dev.openfeature.contrib.tools.tck.ContainerizedProviderTckTest; +import dev.openfeature.contrib.tools.tck.KnownDeviation; import dev.openfeature.sdk.FeatureProvider; import java.io.File; import java.util.Arrays; From 2b87171f28f837243d6d3db51269fbc997f7f8ab Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Sun, 13 Sep 2026 00:38:56 +0200 Subject: [PATCH 04/11] test(flagsmith): pin the testbed image, and follow the compose path convention The compose file moves to src/test/resources/tck/docker-compose.yaml, the path the TCK javadoc uses, and the image is pinned to 0.1.0 rather than :latest. Pinning is the substantive half. Four language adoptions pull this image, and a mutable tag lets a push to the testbed change four pull requests' results with no diff anywhere to explain it. The tag stays overridable through FLAGSMITH_TESTBED_IMAGE. 56 scenarios: 24 pass, 12 fail, 20 skip. Unchanged. Signed-off-by: Simon Schrottner --- .../providers/flagsmith/FlagsmithProviderTckTest.java | 2 +- .../docker-compose.yaml} | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) rename providers/flagsmith/src/test/resources/{flagsmith-testbed-compose.yaml => tck/docker-compose.yaml} (76%) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 6fdbb3be35..66f52fe2e9 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -40,7 +40,7 @@ public class FlagsmithProviderTckTest extends ContainerizedProviderTckTest { @Override public File composeFile() { - return new File("src/test/resources/flagsmith-testbed-compose.yaml"); + return new File("src/test/resources/tck/docker-compose.yaml"); } @Override diff --git a/providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml b/providers/flagsmith/src/test/resources/tck/docker-compose.yaml similarity index 76% rename from providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml rename to providers/flagsmith/src/test/resources/tck/docker-compose.yaml index a154fff322..534a65ef2e 100644 --- a/providers/flagsmith/src/test/resources/flagsmith-testbed-compose.yaml +++ b/providers/flagsmith/src/test/resources/tck/docker-compose.yaml @@ -3,12 +3,15 @@ # # The service is named "backend" because that is ContainerizedProviderTckTest's default. # +# The image is PINNED, not :latest. Four language adoptions pull this image, and a mutable tag lets +# a push to the testbed change four pull requests' results with no diff anywhere to explain it. +# # Ports are never pinned. The TCK reads the dynamically mapped host ports after the stack is up, # and POST /stop deliberately never restarts the container precisely so those mappings survive an # outage scenario. services: backend: - image: ${FLAGSMITH_TESTBED_IMAGE:-ghcr.io/aepfli/flagsmith-tck-testbed:latest} + image: ${FLAGSMITH_TESTBED_IMAGE:-ghcr.io/aepfli/flagsmith-tck-testbed:0.1.0} ports: - 8000 # Edge Proxy -- what the provider under test talks to - 8080 # launchpad -- control API From 2bd418f88d034af1d8197a51da10436a7b63f70a Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Sun, 13 Sep 2026 18:17:49 +0200 Subject: [PATCH 05/11] test(flagsmith): withhold @standard-reasons, with the reason recorded The base added @standard-reasons, and it is the capability that turned eight of this provider's twelve failures into skips: it never populates the resolution reason, so every evaluation returns null and not one of the reasons the capability claims is reported. The resolved values were correct throughout -- only the reason was missing. Withheld with a deviation rather than simply skipped, because 2.2.5 makes the reason a SHOULD and null is arguably permitted. What makes it a defect worth recording is that the Go Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the identical backend, so this is a gap rather than a considered choice. The numeric-coercion deviation is now measured rather than predicted: this provider behaves like Python, not like Go. float-flag resolves to the caller default, so reading a float back as a float does not work against Flagsmith at all. 65 scenarios: 31 pass, 5 fail, 29 skip -- up from 24/12/20. The five remaining are two unreadable floats, the shared type-system pair, and object-flag not resolving as a structure. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 66f52fe2e9..b137047c92 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -96,6 +96,10 @@ public FeatureProvider createUnavailableProvider() { * rather than an oversight: Java's integer accessor is a 32-bit {@code Integer}, so 2^53-1 * cannot be asked for at all. This is the same reason Java withholds it for flagd. * + *

{@code STANDARD_REASONS} is withheld, and it is a defect rather than a permitted absence + * -- see the deviation below. It is the capability that turned eight failures into skips here, + * which is the point of it being a claim a provider opts into rather than an exemption. + * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. * @@ -111,7 +115,18 @@ public Set capabilities() { @Override public List knownDeviations() { - return Arrays.asList(KnownDeviation.untracked( + return Arrays.asList( + KnownDeviation.untracked( + Capability.STANDARD_REASONS, + "This provider never populates the resolution reason: every evaluation returns null, so " + + "not one of the reasons this capability claims is reported. The resolved values are " + + "correct throughout -- only the reason is missing. 2.2.5 makes the reason a SHOULD, " + + "so null is arguably permitted, which is exactly why this is a withheld claim rather " + + "than a failure. What makes it worth recording as a defect is that the Go Flagsmith " + + "provider reports STATIC, DISABLED and TARGETING_MATCH against the identical backend, " + + "so it is a gap rather than a considered choice. Withholding turns eight failures " + + "into skips carrying this reason."), + KnownDeviation.untracked( Capability.NUMERIC_COERCION, "Withheld pending the run, and recorded as a prediction rather than a measurement. " + "Flagsmith stores floats and objects as strings because feature_state_value is " From 4323cb3771c2e63fb676e784abdf5920b1736afb Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Mon, 14 Sep 2026 07:35:42 +0200 Subject: [PATCH 06/11] test(flagsmith): declare the capabilities this provider gets wrong The base corrected the worked example that taught withhold-plus-deviate, and this adoption was doing exactly what the old example illustrated. KnownDeviation now states the rule plainly: withholding a capability in order to turn a failing scenario into a skip is the failure mode the field exists to prevent. @standard-reasons moves from withheld to declared. The previous pass withheld it precisely to turn eight failures into skips, which is the move the rule names. This provider does build a resolution and simply leaves the reason out, so running those scenarios establishes something real, and the failures belong in the results with the deviation attached. @numeric-coercion moves the same way, for the same reason: the provider has the accessors and gets the answer wrong rather than declining to answer. Adds an ungated deviation, with a null capability, for the two mandatory rows that fail because Flagsmith stores floats and objects as strings. That form exists for a gap against a scenario belonging to no tag. 65 scenarios: 35 pass, 13 fail, 17 skip -- from 31/5/29. Twelve more scenarios run; four of them pass and eight fail visibly, which is the point of the change. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 41 +++++++++++++------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index b137047c92..07fbbb9e3f 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -96,9 +96,12 @@ public FeatureProvider createUnavailableProvider() { * rather than an oversight: Java's integer accessor is a 32-bit {@code Integer}, so 2^53-1 * cannot be asked for at all. This is the same reason Java withholds it for flagd. * - *

{@code STANDARD_REASONS} is withheld, and it is a defect rather than a permitted absence - * -- see the deviation below. It is the capability that turned eight failures into skips here, - * which is the point of it being a claim a provider opts into rather than an exemption. + *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are both DECLARED and both fail. + * That is deliberate: KnownDeviation's rule is that withholding a capability in order to turn a + * failing scenario into a skip is the failure mode the field exists to prevent, and this + * adoption was doing exactly that. Running these scenarios establishes something real -- that + * the reason is null, and that a float cannot be read back as a float -- so the failures belong + * in the results with their deviations attached rather than hidden as skips. * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. @@ -110,7 +113,12 @@ public FeatureProvider createUnavailableProvider() { */ @Override public Set capabilities() { - return EnumSet.of(Capability.OBJECT, Capability.TARGETING, Capability.DISABLED_FLAGS); + return EnumSet.of( + Capability.OBJECT, + Capability.TARGETING, + Capability.DISABLED_FLAGS, + Capability.STANDARD_REASONS, + Capability.NUMERIC_COERCION); } @Override @@ -118,14 +126,23 @@ public List knownDeviations() { return Arrays.asList( KnownDeviation.untracked( Capability.STANDARD_REASONS, - "This provider never populates the resolution reason: every evaluation returns null, so " - + "not one of the reasons this capability claims is reported. The resolved values are " - + "correct throughout -- only the reason is missing. 2.2.5 makes the reason a SHOULD, " - + "so null is arguably permitted, which is exactly why this is a withheld claim rather " - + "than a failure. What makes it worth recording as a defect is that the Go Flagsmith " - + "provider reports STATIC, DISABLED and TARGETING_MATCH against the identical backend, " - + "so it is a gap rather than a considered choice. Withholding turns eight failures " - + "into skips carrying this reason."), + "This provider never populates the resolution reason: every evaluation returns null, so not " + + "one of the reasons this capability claims is reported. The resolved values are " + + "correct throughout -- only the reason is missing. The capability is declared and " + + "the scenarios fail rather than skip, because the provider does build a resolution " + + "and simply leaves the field out, so running them establishes something. The Go " + + "Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the " + + "identical backend, which is what makes this a gap rather than a considered choice."), + KnownDeviation.untracked( + null, + "float-flag requested as a String resolves to \"0.5\" rather than reporting TYPE_MISMATCH, in " + + "an untagged row of the wrong-type outline, and object-flag as a String resolves to " + + "the raw JSON text beside it. Flagsmith has no float type and no object type -- " + + "feature_state_value is natively boolean, integer or string -- so on this backend " + + "both really are strings and neither request is a type mismatch. Recorded because " + + "the scenarios are mandatory and fail, not because the provider is wrong: whether " + + "the type-mismatch matrix is satisfiable against a backend with a coarser type " + + "system is an open question for the suite. All four language adoptions fail these."), KnownDeviation.untracked( Capability.NUMERIC_COERCION, "Withheld pending the run, and recorded as a prediction rather than a measurement. " From ba51f0c14febd073b326290fdd0d55d8530d99e2 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Mon, 14 Sep 2026 12:18:40 +0200 Subject: [PATCH 07/11] docs(flagsmith): stop restating what the TCK and the testbed own Three things were written here that are documented elsewhere: the rule for when a deviation is the right shape (tck.KnownDeviation owns it), the control API's inability to hand connection parameters to a provider, and the backend's design and cross-language results (the testbed's README and FINDINGS own those). Each is now a sentence and a link. What stays is what only this adoption knows: which capabilities this provider has, which it gets wrong, and why. The deviation summaries stay verbose on purpose -- they travel into a conformance report read across languages, so they have to stand alone. No behaviour change; the suite reports the same numbers. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 29 ++++--------------- 1 file changed, 6 insertions(+), 23 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 07fbbb9e3f..b813972eb2 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -14,26 +14,13 @@ /** * The OpenFeature Provider Conformance Suite against the Flagsmith Java provider. * - *

An experiment rather than a finished adoption. The point is comparison: the Go adoption - * (go-sdk-contrib#959) reports 31 pass / 2 fail / 19 skip, and the Python provider -- which lives - * in the Flagsmith organisation rather than in a contrib repo -- reports 28 / 5 / 19 against the - * same container. Three providers written by different people against the same backend API is - * exactly the situation the cross-language suite exists for, and the Go and Python results already - * disagree. - * - *

The backend is the same container both of those use: - * aepfli/flagsmith-tck-testbed, the - * Flagsmith Edge Proxy with a launchpad implementing the control API. Nothing about it is - * language-specific -- it is a container with an HTTP control API, which is the point of the - * control API existing. + *

The backend, and how the four language adoptions compare against it, are documented once in + * aepfli/flagsmith-tck-testbed rather + * than restated in each adoption. */ public class FlagsmithProviderTckTest extends ContainerizedProviderTckTest { - /** - * Fixed by the testbed. The control API has no way to communicate connection parameters -- - * {@code POST /start} returns a bare 200 with no body -- so every adoption hardcodes these, - * exactly as a flagd adoption hardcodes a port. - */ + /** Fixed by the testbed, because the control API cannot hand connection parameters to a provider. */ private static final String SERVER_SIDE_KEY = "ser.provider-tck-server-key"; private static final int PROXY_PORT = 8000; @@ -96,12 +83,8 @@ public FeatureProvider createUnavailableProvider() { * rather than an oversight: Java's integer accessor is a 32-bit {@code Integer}, so 2^53-1 * cannot be asked for at all. This is the same reason Java withholds it for flagd. * - *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are both DECLARED and both fail. - * That is deliberate: KnownDeviation's rule is that withholding a capability in order to turn a - * failing scenario into a skip is the failure mode the field exists to prevent, and this - * adoption was doing exactly that. Running these scenarios establishes something real -- that - * the reason is null, and that a float cannot be read back as a float -- so the failures belong - * in the results with their deviations attached rather than hidden as skips. + *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are declared and both fail: this + * provider attempts each and gets it wrong, so the failures belong in the results. * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. From 94b32af49a192de3806d06b93fad634a74307130 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Mon, 14 Sep 2026 13:18:50 +0200 Subject: [PATCH 08/11] docs(flagsmith): say precisely which reason this provider does report The deviation said the provider never populates the reason. Reading the source rather than the run output: it returns DISABLED for a disabled flag and leaves the reason null on every successful resolution, so STATIC and TARGETING_MATCH are the ones never reported. DISABLED is one of the reasons the capability claims, and saying otherwise understated what the provider does. Signed-off-by: Simon Schrottner --- .../providers/flagsmith/FlagsmithProviderTckTest.java | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index b813972eb2..37aa582886 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -109,9 +109,10 @@ public List knownDeviations() { return Arrays.asList( KnownDeviation.untracked( Capability.STANDARD_REASONS, - "This provider never populates the resolution reason: every evaluation returns null, so not " - + "one of the reasons this capability claims is reported. The resolved values are " - + "correct throughout -- only the reason is missing. The capability is declared and " + "This provider reports exactly one reason. resolveFlagsmithEvaluation returns " + + "Reason.DISABLED for a disabled flag and leaves the reason null on every " + + "successful resolution, so STATIC and TARGETING_MATCH are never reported. The " + + "resolved values are correct throughout -- only the reason is missing. The capability is declared and " + "the scenarios fail rather than skip, because the provider does build a resolution " + "and simply leaves the field out, so running them establishes something. The Go " + "Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the " From 5ac2a779cc02b7f2846715ade060a9c48c714692 Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Tue, 15 Sep 2026 22:33:41 +0200 Subject: [PATCH 09/11] test(flagsmith): withhold @string-typing, and drop the deviation it replaces This is the backend the capability was created for, and the run says so rather than the prediction. With the tag declared: two of its four scenarios fail and two pass. float-flag through the String accessor resolves to "0.5" and object-flag to its raw JSON text, while boolean-flag and integer-flag report TYPE_MISMATCH correctly. The split is exactly Flagsmith's type system -- feature_state_value is natively boolean, integer or string, so a boolean really is a boolean and an integer really is an integer, while a float and a structure have no native type and are stored as strings. Asked for as strings, they are returned, and that is the resolved flag value. Withheld rather than declared-and-failing, which is the opposite call from @numeric-coercion beside it. Appendix F's scenario-level rule is subordinate to a prior question -- whether an answer is owed -- and here none is: TYPE_MISMATCH is obliged by no requirement, and the only normative statement about value type is Requirement 1.3.4, a SHOULD on the client. Where the specification permits declining, withholding is the honest report however askable the scenarios are. @numeric-coercion stays declared because flagd's ADR is a rule this suite binds providers to; nothing binds Flagsmith to report a type its backend lacks. The deviation that recorded these same two scenarios goes with it. It existed because they were mandatory and failing, and it called the question open; the pin answers it. Keeping it would assert a defect the specification says is not one, which is precisely what a knownDeviations entry must not do. Measured: 65 scenarios, 33 passing, 21 skipped, 11 failing -- against 35 passing, 17 skipped and 13 failing with the tag declared. The four scenarios now skip naming @string-typing, two of which this provider had been getting right; that cost is recorded in the javadoc rather than hidden, since the tag is the unit of declaration and this backend's typing is not uniform across the four flags. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 36 +++++++++++++------ 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 37aa582886..2fc87d7f81 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -86,6 +86,32 @@ public FeatureProvider createUnavailableProvider() { *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are declared and both fail: this * provider attempts each and gets it wrong, so the failures belong in the results. * + *

{@code STRING_TYPING} is withheld, and this is the backend the capability was + * created for. Measured rather than predicted: with the tag declared, two of its four + * scenarios fail and two pass. {@code float-flag} through the String accessor resolves to + * {@code "0.5"} and {@code object-flag} to its raw JSON text, while {@code boolean-flag} and + * {@code integer-flag} report {@code TYPE_MISMATCH} correctly. The split is exactly Flagsmith's + * type system: {@code feature_state_value} is natively boolean, integer or string, so a boolean + * flag really is a boolean and an integer really is an integer — but a float and a structure + * have no native type and are stored as strings. Asked for as strings, they are + * returned, and that is the resolved flag value. + * + *

Withheld rather than declared-and-failing, which is the opposite call from + * {@code NUMERIC_COERCION} above, and Appendix F's declaring rules are what separate them. The + * rule about scenarios being askable is subordinate to a prior question — whether the provider + * owes an answer at all — and for this tag it does not: {@code TYPE_MISMATCH} is obliged by no + * requirement, and the only normative statement about value type is Requirement 1.3.4, a + * {@code SHOULD} on the client. Where the specification permits declining, withholding is the + * honest report however askable the scenarios are, and a {@link KnownDeviation} would assert a + * defect that does not exist. {@code NUMERIC_COERCION} is declared because flagd's ADR is a rule + * this suite binds providers to; nothing binds Flagsmith to report a type its backend does not + * have. + * + *

The cost is stated rather than hidden: withholding skips the two scenarios this provider + * gets right along with the two it does not, because the tag is the unit of declaration and the + * backend's typing is not uniform across the four flags. Revisit if the capability is ever split + * by flag type. + * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. * @@ -117,16 +143,6 @@ public List knownDeviations() { + "and simply leaves the field out, so running them establishes something. The Go " + "Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the " + "identical backend, which is what makes this a gap rather than a considered choice."), - KnownDeviation.untracked( - null, - "float-flag requested as a String resolves to \"0.5\" rather than reporting TYPE_MISMATCH, in " - + "an untagged row of the wrong-type outline, and object-flag as a String resolves to " - + "the raw JSON text beside it. Flagsmith has no float type and no object type -- " - + "feature_state_value is natively boolean, integer or string -- so on this backend " - + "both really are strings and neither request is a type mismatch. Recorded because " - + "the scenarios are mandatory and fail, not because the provider is wrong: whether " - + "the type-mismatch matrix is satisfiable against a backend with a coarser type " - + "system is an open question for the suite. All four language adoptions fail these."), KnownDeviation.untracked( Capability.NUMERIC_COERCION, "Withheld pending the run, and recorded as a prediction rather than a measurement. " From 8b2e3b561989a95c210144fa0669771790fc380c Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Wed, 16 Sep 2026 09:35:35 +0200 Subject: [PATCH 10/11] test(flagsmith): declare @string-typing, withhold @fully-typed-values The previous pass withheld one tag over all four scenarios and wrote down what that cost: two of the four were skipped despite this provider getting them right, because the tag was the unit of declaration and Flagsmith's typing is not uniform across the four flags. That note ended "revisit if the capability is ever split by flag type". Specification revision bda599f1 split it, citing this measurement, so this is the revisit. feature_state_value is natively boolean, integer or string only. A boolean flag really is a boolean and an integer really is an integer, so @string-typing is declared and its two rows are held to. A float and a structure have no native type and are stored as strings, so asked for as strings they are returned -- there is no mismatch to report, and @fully-typed-values is withheld. Withheld rather than declared-and-failing for the reason the file already argued: TYPE_MISMATCH is obliged by no requirement, so no answer is owed and a deviation would assert a defect that does not exist. Measured, both sides of the re-pin: 65 scenarios either way, 33 passing / 11 failing / 21 skipped before, 35 / 11 / 19 after. The two scenarios that started running are the boolean and integer rows and both pass; the failure count does not move. The skips now attribute as VARIANTS 8, LIFECYCLE 6, FULLY_TYPED_VALUES 2, EVENTS 2 and LARGE_INTEGERS 1 -- where the withheld capability was answering for four scenarios it now answers for exactly the two whose question this backend cannot be asked. Also fixes a contradiction this file carried: the numeric-coercion deviation said the capability was "withheld pending the run" while capabilities() declared it, and its indentation had drifted out of line. The run has since happened, so it now records what was measured -- Java resembles Python rather than Go and does not parse Flagsmith's string-stored float, which fails both lossless coercion scenarios and the untagged float scenario with the caller's default. Declared and left failing, because the provider does attempt the coercion. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 71 ++++++++++++------- 1 file changed, 45 insertions(+), 26 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 2fc87d7f81..3619efb323 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -86,31 +86,40 @@ public FeatureProvider createUnavailableProvider() { *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are declared and both fail: this * provider attempts each and gets it wrong, so the failures belong in the results. * - *

{@code STRING_TYPING} is withheld, and this is the backend the capability was - * created for. Measured rather than predicted: with the tag declared, two of its four - * scenarios fail and two pass. {@code float-flag} through the String accessor resolves to - * {@code "0.5"} and {@code object-flag} to its raw JSON text, while {@code boolean-flag} and - * {@code integer-flag} report {@code TYPE_MISMATCH} correctly. The split is exactly Flagsmith's - * type system: {@code feature_state_value} is natively boolean, integer or string, so a boolean - * flag really is a boolean and an integer really is an integer — but a float and a structure - * have no native type and are stored as strings. Asked for as strings, they are - * returned, and that is the resolved flag value. + *

{@code STRING_TYPING} is declared and {@code FULLY_TYPED_VALUES} is withheld, and + * this is the backend that pair of capabilities was created for. The previous pass + * measured all four scenarios under one tag: {@code boolean-flag} and {@code integer-flag} + * reported {@code TYPE_MISMATCH} correctly, while {@code float-flag} through the String accessor + * resolved to {@code "0.5"} and {@code object-flag} to its raw JSON text. That split is exactly + * Flagsmith's type system: {@code feature_state_value} is natively boolean, integer or string, + * so a boolean flag really is a boolean and an integer really is an integer — but a float and a + * structure have no native type and are stored as strings. Asked for as strings, they + * are returned, and that is the resolved flag value. * - *

Withheld rather than declared-and-failing, which is the opposite call from + *

So the declaration now follows the backend rather than averaging over it. Specification + * revision {@code bda599f1} split the tag along that line — {@code @string-typing} keeps the + * boolean and integer rows, {@code @fully-typed-values} takes the float and the structure — and + * this provider declares the first and withholds the second. The reason for withholding is a + * fact about the store and not about the provider: Flagsmith records no native float or + * structure type, so there is no mismatch here to report. + * + *

Withholding rather than a {@link KnownDeviation}, which is the opposite call from * {@code NUMERIC_COERCION} above, and Appendix F's declaring rules are what separate them. The * rule about scenarios being askable is subordinate to a prior question — whether the provider - * owes an answer at all — and for this tag it does not: {@code TYPE_MISMATCH} is obliged by no + * owes an answer at all — and for these tags it does not: {@code TYPE_MISMATCH} is obliged by no * requirement, and the only normative statement about value type is Requirement 1.3.4, a * {@code SHOULD} on the client. Where the specification permits declining, withholding is the - * honest report however askable the scenarios are, and a {@link KnownDeviation} would assert a - * defect that does not exist. {@code NUMERIC_COERCION} is declared because flagd's ADR is a rule - * this suite binds providers to; nothing binds Flagsmith to report a type its backend does not - * have. + * honest report however askable the scenarios are, and a deviation would assert a defect that + * does not exist. {@code NUMERIC_COERCION} is declared because flagd's ADR is a rule this suite + * binds providers to; nothing binds Flagsmith to report a type its backend does not have. * - *

The cost is stated rather than hidden: withholding skips the two scenarios this provider - * gets right along with the two it does not, because the tag is the unit of declaration and the - * backend's typing is not uniform across the four flags. Revisit if the capability is ever split - * by flag type. + *

The cost the previous pass recorded is gone, and recording it is what removed it. Under one + * tag, withholding skipped the two scenarios this provider gets right along with the two it does + * not, and that note ended "revisit if the capability is ever split by flag type". It was — the + * measurement above is cited in Appendix F as the case that motivated the split, alongside a + * language that failed the boolean and integer rows and would have had that defect published as + * a permitted absence. Declaring the narrower tag now holds this provider to the two questions + * its backend can answer. * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. @@ -127,7 +136,11 @@ public Set capabilities() { Capability.TARGETING, Capability.DISABLED_FLAGS, Capability.STANDARD_REASONS, - Capability.NUMERIC_COERCION); + Capability.NUMERIC_COERCION, + // STRING_TYPING without FULLY_TYPED_VALUES: the two scenarios the second gates + // carry the first as well, so this runs the boolean and integer rows and leaves + // the float and structured ones skipped. See capabilities()' javadoc. + Capability.STRING_TYPING); } @Override @@ -144,11 +157,17 @@ public List knownDeviations() { + "Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the " + "identical backend, which is what makes this a gap rather than a considered choice."), KnownDeviation.untracked( - Capability.NUMERIC_COERCION, - "Withheld pending the run, and recorded as a prediction rather than a measurement. " - + "Flagsmith stores floats and objects as strings because feature_state_value is " - + "natively boolean, integer or string only. Go compensates by parsing the string in " - + "its Float accessor and Python does not, so Python cannot read a Flagsmith float at " - + "all. Which of the two Java resembles is what this run is for.")); + Capability.NUMERIC_COERCION, + "Java resembles Python rather than Go, which is what the run settled. Flagsmith " + + "stores floats as strings because feature_state_value is natively boolean, " + + "integer or string only; Go compensates by parsing that string in its Float " + + "accessor and Python does not. Neither does Java: both lossless coercion " + + "scenarios return the caller's default rather than the value -- " + + "integral-float-flag through the integer accessor gives 1 for 10, and " + + "integer-flag through the float accessor gives 0.1 for 10.0 -- and so does " + + "the untagged \"A float flag resolves as a float\", which gives 0.1 for 0.5. " + + "The capability is declared and left failing rather than withheld, because " + + "the provider does attempt the coercion; a Float accessor that parsed the " + + "stored string would fix all three at once.")); } } From 929583558c712e6044f4052c29996e53e75e0cfc Mon Sep 17 00:00:00 2001 From: Simon Schrottner Date: Mon, 21 Sep 2026 12:57:55 +0200 Subject: [PATCH 11/11] docs(flagsmith): cut the javadoc back to what the code does not say 173 lines to 132. The string-typing note argued the case for the capability split across five paragraphs; the split has landed, so one paragraph saying which half this backend answers is enough. The deviation summaries lose their narrative and keep their claim. No behaviour change. Signed-off-by: Simon Schrottner --- .../flagsmith/FlagsmithProviderTckTest.java | 73 ++++--------------- 1 file changed, 16 insertions(+), 57 deletions(-) diff --git a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java index 3619efb323..3e471ff507 100644 --- a/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java +++ b/providers/flagsmith/src/test/java/dev/openfeature/contrib/providers/flagsmith/FlagsmithProviderTckTest.java @@ -86,48 +86,18 @@ public FeatureProvider createUnavailableProvider() { *

{@code STANDARD_REASONS} and {@code NUMERIC_COERCION} are declared and both fail: this * provider attempts each and gets it wrong, so the failures belong in the results. * - *

{@code STRING_TYPING} is declared and {@code FULLY_TYPED_VALUES} is withheld, and - * this is the backend that pair of capabilities was created for. The previous pass - * measured all four scenarios under one tag: {@code boolean-flag} and {@code integer-flag} - * reported {@code TYPE_MISMATCH} correctly, while {@code float-flag} through the String accessor - * resolved to {@code "0.5"} and {@code object-flag} to its raw JSON text. That split is exactly - * Flagsmith's type system: {@code feature_state_value} is natively boolean, integer or string, - * so a boolean flag really is a boolean and an integer really is an integer — but a float and a - * structure have no native type and are stored as strings. Asked for as strings, they - * are returned, and that is the resolved flag value. - * - *

So the declaration now follows the backend rather than averaging over it. Specification - * revision {@code bda599f1} split the tag along that line — {@code @string-typing} keeps the - * boolean and integer rows, {@code @fully-typed-values} takes the float and the structure — and - * this provider declares the first and withholds the second. The reason for withholding is a - * fact about the store and not about the provider: Flagsmith records no native float or - * structure type, so there is no mismatch here to report. - * - *

Withholding rather than a {@link KnownDeviation}, which is the opposite call from - * {@code NUMERIC_COERCION} above, and Appendix F's declaring rules are what separate them. The - * rule about scenarios being askable is subordinate to a prior question — whether the provider - * owes an answer at all — and for these tags it does not: {@code TYPE_MISMATCH} is obliged by no - * requirement, and the only normative statement about value type is Requirement 1.3.4, a - * {@code SHOULD} on the client. Where the specification permits declining, withholding is the - * honest report however askable the scenarios are, and a deviation would assert a defect that - * does not exist. {@code NUMERIC_COERCION} is declared because flagd's ADR is a rule this suite - * binds providers to; nothing binds Flagsmith to report a type its backend does not have. - * - *

The cost the previous pass recorded is gone, and recording it is what removed it. Under one - * tag, withholding skipped the two scenarios this provider gets right along with the two it does - * not, and that note ended "revisit if the capability is ever split by flag type". It was — the - * measurement above is cited in Appendix F as the case that motivated the split, alongside a - * language that failed the boolean and integer rows and would have had that defect published as - * a permitted absence. Declaring the narrower tag now holds this provider to the two questions - * its backend can answer. + *

{@code STRING_TYPING} is declared and {@code FULLY_TYPED_VALUES} withheld, which is the + * split this backend motivated. {@code feature_state_value} is natively boolean, integer or + * string: a boolean really is a boolean and an integer really is an integer, so both report + * {@code TYPE_MISMATCH} through the String accessor, while a float and a structure have no + * native type, are stored as text, and are correctly returned. Withheld rather than deviated + * because nothing obliges a provider to report a type its store does not have. * *

{@code DISABLED_FLAGS} is declared. Flagsmith's native model is {@code enabled} plus a * value, so the canonical set's four disabled-* flags map straight onto it. * - *

The lifecycle and event capabilities are withheld pending the run. Go's provider - * implements no {@code StateHandler} whatsoever; whether Java's does is the first thing this - * run answers, and declaring them afterwards is the correct follow-up. Withholding a capability - * a provider genuinely has is the expensive mistake, because it makes the suite blind to it. + *

The lifecycle and event capabilities are withheld: this provider has no observable + * initialisation for the suite to assert against. */ @Override public Set capabilities() { @@ -137,9 +107,6 @@ public Set capabilities() { Capability.DISABLED_FLAGS, Capability.STANDARD_REASONS, Capability.NUMERIC_COERCION, - // STRING_TYPING without FULLY_TYPED_VALUES: the two scenarios the second gates - // carry the first as well, so this runs the boolean and integer rows and leaves - // the float and structured ones skipped. See capabilities()' javadoc. Capability.STRING_TYPING); } @@ -151,23 +118,15 @@ public List knownDeviations() { "This provider reports exactly one reason. resolveFlagsmithEvaluation returns " + "Reason.DISABLED for a disabled flag and leaves the reason null on every " + "successful resolution, so STATIC and TARGETING_MATCH are never reported. The " - + "resolved values are correct throughout -- only the reason is missing. The capability is declared and " - + "the scenarios fail rather than skip, because the provider does build a resolution " - + "and simply leaves the field out, so running them establishes something. The Go " - + "Flagsmith provider reports STATIC, DISABLED and TARGETING_MATCH against the " - + "identical backend, which is what makes this a gap rather than a considered choice."), + + "resolved values are correct throughout -- only the reason is missing. The Go " + + "Flagsmith provider reports all three against the identical backend, which makes " + + "this a gap rather than a considered choice."), KnownDeviation.untracked( Capability.NUMERIC_COERCION, - "Java resembles Python rather than Go, which is what the run settled. Flagsmith " - + "stores floats as strings because feature_state_value is natively boolean, " - + "integer or string only; Go compensates by parsing that string in its Float " - + "accessor and Python does not. Neither does Java: both lossless coercion " - + "scenarios return the caller's default rather than the value -- " - + "integral-float-flag through the integer accessor gives 1 for 10, and " - + "integer-flag through the float accessor gives 0.1 for 10.0 -- and so does " - + "the untagged \"A float flag resolves as a float\", which gives 0.1 for 0.5. " - + "The capability is declared and left failing rather than withheld, because " - + "the provider does attempt the coercion; a Float accessor that parsed the " - + "stored string would fix all three at once.")); + "Flagsmith stores floats as strings, and this provider does not parse them back. " + + "Both lossless coercion scenarios return the caller's default instead of the " + + "value, as does the untagged \"A float flag resolves as a float\". A Float " + + "accessor that parsed the stored string would fix all three; Go has one, " + + "Python and Java do not.")); } }