docs: specify experiment holdouts in local flag evaluation - #76
Conversation
Missing holdout supportFollow-up SDK implementation work from the source audit. All 17 requested repositories were checked against freshly fetched default-branch snapshots. Links below pin those audited commits, rather than moving These evaluators proceed to release conditions and regular variant assignment without checking
8 repositories, 9 SDK implementations need follow-up fixes. Go has hash tests containing Already fixedPython PR #978 is merged. The audited default branch includes holdout handling, and its changelog lists the fix under 7.59.1. Not affected by this local-evaluation bug
Follow-up scopeImplement the contract in each affected evaluator and preserve holdout metadata through typed definitions and existing caches. Cover holdout precedence, exact hash parity, fractional/inclusive boundaries and clamping, group/device identity, and bulk/dependency evaluation. The This is a source audit, not a report of passing or failing runtime reproductions across these SDKs. No SDK implementation changes are included in this PR. |
Problem
SDKs evaluating flag definitions locally can ignore
filters.holdoutand assign excluded experiment participants to regular variants. Python fixed this in PostHog/posthog-python#978, but the shared SDK specification did not cover holdouts.Changes
holdout-<id>result.SDK implementation fixes are follow-up work. This PR does not change SDK code.
Validation
openspec validate define-local-eval-holdouts --strictpassed before archiving.openspec validate --specs --strict: 63 passed.git diff --checkpassed.The Gherkin files document acceptance contracts. SDK conformance tests were not run. Autoreview was skipped because this is a documentation-only change.