test: docker-compose + containerless harness for the master HTTP surface - #171
Merged
Conversation
Bring up the full control-plane + data-plane stack (MinIO, etcd, master, two workers) so the real HTTP surface can be exercised end to end. This closes the coverage gap for the record-list source selector (?source=fragments|wal|all, PR #170), which is only reachable through live etcd + object store and is otherwise #[ignore]d in CI. - test/Dockerfile: single multi-stage build (ui -> builder -> master/worker) - test/docker-compose.yml: MinIO-backed s3://lance-context DATA_DIR shared by master + workers; WAL self-merge disabled so source split is observable - test/harness/{up,down,smoke}.sh: bring up/tear down + assert source semantics over HTTP (fragments=0, wal=3, all=3 before merge; default= fragments; unknown source -> 400) - test/harness/README.md: usage + manual curl recipes for agents - .dockerignore: keep build context small Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Some environments have a Docker daemon but a kernel that forbids unshare/netlink, so no container can start (docker run fails with "failed to register layer: unshare: operation not permitted"). Add a native-process variant of the same stack so the source-selector HTTP surface can still be validated end to end: - native-up.sh: downloads a static etcd, builds the two binaries, and runs etcd + lance-context-server + lance-context-master as host processes over a local-filesystem DATA_DIR (no MinIO). --smoke chains smoke.sh. - native-down.sh: stop processes (+ --purge to wipe state). - README: document containerless mode. Verified locally: native-up.sh --smoke passes all assertions (fragments=0, wal=3, all=3 before merge; default=fragments; unknown source -> 400). Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a runnable test environment + harness so the master's rollout-record HTTP
surface — in particular the
?source=fragments|wal|allselector shipped in #170 —can be exercised end to end. That path is only reachable with a live etcd + object
store, so its integration test is
#[ignore]d and never runs in CI; this closesthat gap.
Two interchangeable ways to bring the same stack up (etcd + master + worker(s),
shared
DATA_DIR):test/docker-compose.yml+ single multi-stagetest/Dockerfile):MinIO-backed
s3://lance-contextshared by master + two workers. WAL self-mergedisabled so the source split is observable.
test/harness/native-*.sh): same stack as plain host processes(static etcd binary, local-filesystem
DATA_DIR,cargo-built binaries) forsandboxes where the kernel forbids
unshare/netlink and no container can start.test/harness/smoke.shmakes real HTTP calls and asserts:fragments=0,wal=3,all=3 before any merge; response echoes the resolved source; default (no param) =fragments; unknown source → HTTP 400.Files
test/Dockerfile— multi-stage (ui → builder → master/worker)test/docker-compose.yml— MinIO + etcd + master + 2 workerstest/harness/{up,down,smoke}.sh— Docker path + shared smoke assertionstest/harness/native-{up,down}.sh— containerless pathtest/harness/README.md,.dockerignoreTest plan / what was actually verified
etcd + master + worker over HTTP):
fragments=0 / wal=3 / all=3,default→fragments, unknown source→400.
npm run build); compose YAML parses; all shell scriptsbash -nclean.because this sandbox's kernel blocks container startup
(
unshare: operation not permitted). Needs a runner with a workingcontainer runtime to exercise
up.sh && smoke.sh.🤖 Generated with Claude Code