You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test: run a conformance smoke test on tmachine through a test runner #4934
I'm moving OpenShell's end-to-end suites onto testkit (#3954). I need to run a testkit test against a real OpenShell install in a tmachine VM, locally and on CI, without writing an Ansible playbook per suite, so suites can move off the old tmachine testsuites one at a time.
Description
Add the smallest path that runs a testkit suite inside a tmachine environment:
a tmachine Session API that boots an environment with an installer, runs commands with timeouts, copies files in and out, collects diagnostics and always tears the VM down;
openshell-test-runner (tests/runner/), which builds a test family's packages for the guest, boots one named tmachine environment, waits for the gateway, runs the tests with nextest and checks the JUnit results;
a one-test sandbox smoke suite in tests/suites/conformance/sandbox;
an advisory CI lane (e2e-suites.yml) with an explicit matrix, starting with Ubuntu + rootful Docker + the Debian package.
nix run .#openshell-test-runner -- \
--environment ubuntu-docker-rootful --installer deb \
--family conformance --package sandbox
Out of scope, deferred until the features suites move: test plans, an environment registry, a source test catalog and generated CI matrices. Also out of scope: per-test requirements and waivers; a broken test is #[ignore = "<issue>"].
Definition of Done
The runner passes the smoke suite in ubuntu-docker-rootful with the deb installer, locally (host architecture) and on CI (amd64).
A run fails if nextest fails or no test ran.
The CI lane uploads junit.xml, nextest.log and, after a failure, guest diagnostics, and is advisory.
Activity