Skip to content

fix: update harness for latest dataplane - #4

Open
lucarlig wants to merge 1 commit into
mainfrom
user/luca/import-mcp-manual-test-tools
Open

fix: update harness for latest dataplane#4
lucarlig wants to merge 1 commit into
mainfrom
user/luca/import-mcp-manual-test-tools

Conversation

@lucarlig

Copy link
Copy Markdown
Contributor

Summary

  • Update the published dataplane default to ghcr.io/contextforge-org/contextforge-data-plane:latest and align the harness with the current control-plane publisher, token, catalog, and dataplane startup contracts.
  • Enforce the public routing split: /servers/{virtual_host_id}/mcp is dataplane-only, while raw /mcp and UI/API traffic remain on the control plane.
  • Add lifecycle-aware MCP support for the modern stateless dataplane path (server/discover, protocol 2026-07-28, per-request metadata) while retaining the legacy control-plane flow.
  • Make probe, Goose, Locust, token management, debug/Inspector, live-test grouping, and CLI validation consistent across the supported execution lanes.
  • Refresh the official conformance runner to alpha.11, add the backend-only Host proxy, fix initial Compose overlay ordering, and record the three-lane readiness results.
  • Document the intended future split between the management plane, legacy control-plane slow path, and Rust dataplane fast path.

Why

The integration harness still assumed the former dataplane image/repository, bootstrap and token behavior, lifecycle shape, and container routing behavior. Those assumptions caused real CLI commands to fail or exercise the wrong backend. Conformance also needed a stable backend-facing Host header and the dataplane overlay on the first gateway startup.

Impact

The normal path continues to use published images; local dataplane builds remain an explicit override. Catalog tokens are issued and revoked through the control-plane APIs, routed responses are checked for dataplane identity, and containerized load/conformance traffic uses valid host allowlisting.

The tested latest image resolved to sha256:f227d4ea4922cda2c6ee3c529f6a224b54ca906379ac62b9700b0b1e1ef90740.

The committed conformance report records current upstream readiness rather than masking failures: fixture direct 31/40, control plane 0/40, and dataplane 7/40, with no missing or ambiguous checks.

Validation

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test --workspace --locked — 330 passed across 31 suites
  • Exercised all 16 public CLI help paths
  • Exercised stack up/status/logs/config/down, both probe modes, admin and scoped token issuance/revocation, debug Inspector, and all live-test groups
  • Dataplane load tests: Goose 820 requests / 0 failures; Locust 73 requests / 0 failures
  • Live suites: MCP 19 passed / 3 skipped; RBAC 40 passed; fixture-direct protocol 33 passed
  • Generated the official three-lane conformance comparison report

Signed-off-by: lucarlig <luca.carlig@ibm.com>
@lucarlig

lucarlig commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

CLI results

Tests

CLI command Result Passed Failed Skipped / notes
cargo test --workspace --locked ✅ PASS 330 tests across 31 suites 0 Final run after rebase
cf-integration probe --lane controlplane ✅ PASS Legacy MCP lifecycle completed 0 Raw /mcp route
cf-integration probe --lane dataplane ✅ PASS Stateless MCP lifecycle completed 0 Routed /servers/{id}/mcp path
cf-integration live --lane dataplane --group mcp ✅ PASS 19 0 3 skipped
cf-integration live --lane dataplane --group rbac ✅ PASS 40 0 0
cf-integration live --lane fixture-direct --group protocol ✅ PASS 33 0 0
cf-integration live --lane dataplane --group protocol ⚠️ ENVIRONMENT-LIMITED 0 0 66 skipped because the reference server was isolated in nested Docker

Performance

CLI command Engine Requests Failures Result
cf-integration load --lane dataplane --engine goose --smoke Goose 820 0 ✅ PASS
cf-integration load --lane dataplane --engine locust --smoke Locust 73 0 ✅ PASS

Conformance

CLI command Result Detail
cf-integration conformance run ✅ COMPLETED Ran all 40 official scenarios against fixture-direct, control-plane, and dataplane lanes
cf-integration conformance report ✅ PASS Report generated with 0 ambiguous and 0 missing results
Target Compliant Failed scenarios Failed checks Ambiguous Missing
Fixture direct 31 / 40 9 17 0 0
Control plane 0 / 40 40 110 0 0
Dataplane 7 / 40 33 50 0 0
Comparison classification Scenarios
Control-plane-only failure 7
Both gateways failed 24
Shared fixture and gateway failure 9
All compliant 0

Full scenario results: reports/mcp-conformance-comparison.md.

@lucarlig
lucarlig marked this pull request as ready for review August 19, 2026 07:40
@lucarlig
lucarlig requested a review from Lang-Akshay August 19, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant