Repository navigation
Require Experimental detectors to have an ExperimentalConfiguration - #1896
Draft
Ryan Brandenburg (ryanbrandenburg) wants to merge 1 commit into
Draft
Ryan Brandenburg (ryanbrandenburg) wants to merge 1 commit into
Ryan Brandenburg (ryanbrandenburg) wants to merge 1 commit into
Conversation
Ryan Brandenburg (ryanbrandenburg)
requested a review
from a team
as a code owner
October 14, 2026 22:53
Ryan Brandenburg (ryanbrandenburg)
requested a review
from Sebastian Gomez (sebasgomez238)
October 14, 2026 22:53
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
Static review found no blocking issues, and added tests cover the main discovery and validation paths; tests were not run.
0 open findings
What changed in this PR
Makes experiment configuration mandatory for experimental detectors, preventing discoveries from being silently excluded from experiment reporting.
Changes:
- Requires detector-owned configurations and automatically discovers and validates them.
- Adds empty-baseline configurations and removes the inactive SimplePip experiment.
- Updates tests and documentation for the configuration lifecycle.
| File | Description |
|---|---|
| test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/SwiftDetectorExperimentTests.cs | Updates configuration imports. |
| test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/LinuxApplicationLayerExperimentTests.cs | Updates configuration imports. |
| test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/ExperimentServiceTests.cs | Tests discovery, validation, sharing, and filtering. |
| test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/ExperimentConfigTests.cs | Discovers configurations through detectors. |
| test/Microsoft.ComponentDetection.Orchestrator.Tests/Experiments/DefaultExperimentProcessorTests.cs | Uses the relocated contract. |
| test/Microsoft.ComponentDetection.Detectors.Tests/EmptyBaselineExperimentTests.cs | Tests name validation and detector matching. |
| test/Microsoft.ComponentDetection.Detectors.Tests/ComponentDetectorTests.cs | Verifies configuration invariants and SimplePip status. |
| src/Microsoft.ComponentDetection.Orchestrator/Extensions/ServiceCollectionExtensions.cs | Removes separate configuration registrations. |
| src/Microsoft.ComponentDetection.Orchestrator/Experiments/IExperimentProcessor.cs | Uses the relocated contract. |
| src/Microsoft.ComponentDetection.Orchestrator/Experiments/ExperimentService.cs | Discovers, validates, and deduplicates configurations. |
| src/Microsoft.ComponentDetection.Orchestrator/Experiments/DefaultExperimentProcessor.cs | Uses the relocated contract. |
| src/Microsoft.ComponentDetection.Orchestrator/Experiments/Configs/SimplePipExperiment.cs | Deletes the inactive experiment. |
| src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftResolvedComponentDetector.cs | Supplies its experiment configuration. |
| src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftDetectorExperiment.cs | Relocates configuration beside Swift detection. |
| src/Microsoft.ComponentDetection.Detectors/poetry/PoetryComponentDetector.cs | Supplies an empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/linux/LinuxApplicationLayerExperiment.cs | Relocates configuration beside Linux detection. |
| src/Microsoft.ComponentDetection.Detectors/linux/LinuxApplicationLayerDetector.cs | Supplies its experiment configuration. |
| src/Microsoft.ComponentDetection.Detectors/ivy/IvyDetector.cs | Supplies an empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/helm/HelmComponentDetector.cs | Supplies an empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/Experiments/EmptyBaselineExperiment.cs | Adds reusable empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/dockerfile/DockerfileComponentDetector.cs | Supplies an empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/dockercompose/DockerComposeComponentDetector.cs | Supplies an empty-baseline configuration. |
| src/Microsoft.ComponentDetection.Detectors/conda/CondaLockDetectorExperiment.cs | Relocates configuration beside Conda detection. |
| src/Microsoft.ComponentDetection.Detectors/conda/CondaLockComponentDetector.cs | Supplies its experiment configuration. |
| src/Microsoft.ComponentDetection.Contracts/IExperimentConfiguration.cs | Moves the contract and requires explicit initialization. |
| src/Microsoft.ComponentDetection.Contracts/IComponentDetector.cs | Requires experimental detectors to expose configuration. |
| docs/detectors/swift.md | Documents automatic configuration discovery. |
| docs/detectors/pip.md | Clarifies legacy detectors’ default-off status. |
| docs/creating-a-new-detector.md | Documents configuration requirements and examples. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Ryan Brandenburg (ryanbrandenburg)
marked this pull request as draft
October 16, 2026 22:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Right now it is possible (and even common) for a Detector to be Experimental without having an associated Config. This will cause the experiment to run, but prevent the data it produces from being used and is probably not really an intended state. This bit us in #1866 when I promoted the Swift Detector but did not realize that I needed to create an associated config (in part because I referenced another Detector which did not have one).
This PR attempts to make it the default position to create an ExperimentalConfiguration when promoting your Detector to experimental by forcing you to make an active decision not to if that's your goal (though again, I'm not clear what the scenario for that would be).
Existing Experimental detectors without a config have been given a configuration which does not have a control group (as was done in VCPKG and now Swift) and in the case of SimplePip a seemingly orphaned Configuration was deleted. Alternatively we might want to demote some of these experimental Detectors if we don't want to produce data about them.