ci.md

title	CI
description	How to run the test matrix locally, without Docker, and in air-gapped environments, and how the hosted pipeline is pinned.

CI

make ci is the single source of truth. Any hosted pipeline is a thin wrapper around the same targets, so a green local run means a green pipeline. If the two ever disagree, the Makefile is right and the workflow is wrong.

make lint      # phpcs
make compat    # PHPCompatibilityWP, testVersion 7.4-
make analyse   # phpstan, level 10
make test      # phpunit, single site
make test-ms   # phpunit, multisite
make test-stand-down  # on a WordPress that ships the API: the plugin stands down
make reference-check  # docs/reference/ matches the source docblocks
make smoke     # WP-CLI end to end against a throwaway install in .smoke/
make ci        # all of the above, except test-stand-down

make test-stand-down needs a WordPress that ships the Secrets API, such as trunk, so it is not part of make ci. It runs the stand-down group with SECRETS_API_TESTS_EXPECT_CORE=1, which makes the stand-down tests fail instead of skip on a WordPress without the API. The group also holds the Site Health parity tests: plugin/class-secrets-api-site-health.php copies methods from core's WP_Site_Health and WP_Debug_Data for WordPress before 7.2, and on trunk each copy is compared with core's. When one fails, copy the method again from core.

The always-works path

composer install
bin/ci-local.sh

bin/ci-local.sh brings up wp-env (Docker), installs the WordPress core PHPUnit suite inside the tests container, runs everything, and tears down. --keep leaves the environment running between iterations.

Nothing in this path depends on a hosted runner, a Marketplace Action, or network access beyond the initial composer install and the first Docker image pull.

Without Docker

If you already have a WordPress test suite and a MySQL database:

make install   # composer install + bin/install-wp-tests.sh
make ci

bin/install-wp-tests.sh takes <db-name> <db-user> <db-pass> [db-host] [wp-version] and honours WP_TESTS_DIR and WP_CORE_DIR. A version with a trailing .x, such as 7.1.x, means the newest release on that branch. bin/resolve-wp-version.sh looks it up in the version-check API's offers, and bin/smoke-install.sh uses it too.

The WP-CLI smoke test

make smoke (bin/smoke-install.sh then tests/smoke/smoke.sh) is a bash harness that drives a real, pinned wp-cli.phar against a real WordPress install it provisions itself under .smoke/ — never wp-env's own wp-content, never the PHPUnit suite's database. It exists because every other test in this repository instantiates WP_CLI_Secret_Command directly and calls its methods, which covers nothing about how WP-CLI actually dispatches to them; see docs/journal/2026-09-24-testing-the-cli-for-real.md and tests/smoke/SPEC.md.

Variables it reads: SMOKE_DB_NAME (defaults to wordpress_smoke, and refuses to run if set to wordpress_test, which belongs to the PHPUnit suite), DB_USER, DB_PASS, DB_HOST, WP_VERSION, and SMOKE_URL. It needs egress to GitHub releases (the wp-cli.phar download) and wordpress.org (wp core download). Through bin/ci-local.sh it runs inside wp-env's cli container, against wp-env's own development MySQL rather than a separate service. The install under .smoke/ is disposable: bin/smoke-install.sh drops and recreates its database and overwrites wp-config.php on every run.

Air-gapped and mirrored environments

bin/install-wp-tests.sh reads two environment variables so it never has to reach wordpress.org:

Variable	Effect
WP_MIRROR_BASE	Substituted for https://wordpress.org when downloading WordPress itself
WP_TESTS_ZIP_URL	Full URL to a WordPress archive, overriding version lookup entirely
WP_API_BASE	Substituted for https://api.wordpress.org, used only to resolve "latest" and versions such as 7.1.x
WP_DEVELOP_BASE	Substituted for https://github.com/WordPress/wordpress-develop, where the test suite comes from

Three separate hosts, deliberately: a mirror of wordpress.org downloads is not automatically a mirror of the version-check API or of the wordpress-develop repository, so collapsing them into one variable would quietly send two of the three somewhere that cannot serve them.

The test suite is fetched as a tarball, not an svn checkout. svn is not installed on GitHub's ubuntu-24.04 runners and has not shipped with macOS since Xcode's command line tools dropped it, so requiring it meant make install — the documented no-Docker path — could not run in CI or on a stock Mac. curl-or-wget plus tar is a dependency both already have.

Composer's cache (~/.composer/cache) should be restored between runs on any runner without reliable egress to Packagist.

Where this runs

make ci is the source of truth and depends on none of the below. What follows is only about the hosted pipeline.

github.com, hosted runners. Static analysis gates a PHP 7.4 / 8.0 / 8.3 / 8.5 × WordPress 7.1.x matrix with a WordPress 6.8 floor leg, a multisite job, and a stand-down job on trunk. shivammathur/setup-php provides the interpreter and asks for the sodium extension by name. The whole API is built on libsodium, so relying on whatever the runner image happens to ship wasn't good enough. The examples job is the only one with service containers beyond the database: a pinned Moto instance, which the AWS Secrets Manager conformance run and the AWS KMS keyring tests run against, and a pinned Vault dev server for the Vault example. It runs make test-examples single site and multisite.

The workflow declares permissions: contents: read. Nothing in it writes to the repository, publishes anything, or needs a token beyond reading the code under test.

WP_MIRROR_BASE and the air-gapped path above still work and are still supported by bin/install-wp-tests.sh. CI as configured just doesn't depend on them.

Pinning

Every action is pinned by full commit SHA rather than by tag. Tags move; SHAs don't. Given this repository implements a credential store, an action that could change underneath a moved tag isn't a risk worth taking.

Each SHA in the workflow came from the GitHub API at the version noted beside it, rather than copied out of documentation or a README. An unverified pin is really a pin to whatever the last person pasted.

wp-cli.phar is pinned the same way, by version and SHA-256, in bin/smoke-install.sh.

Matrix

Job	PHP	WordPress	Notes
static	8.3	—	lint + compat + analyse. Gates everything else.
test	7.4, 8.0, 8.3, 8.5; 7.4	7.1.x; 6.8	Single site
stand-down	7.4, 8.5	trunk	make test-stand-down: core serves the API, the plugin loads none of its own layer, and the plugin's copies of core's Site Health methods match core's. Core is wordpress-develop trunk's src/, from the same tarball as the test suite, not the nightly zip, which can lag a day behind it.
test-multisite	8.3	7.1.x	Multisite suite
smoke	7.4, 8.3, 8.5; 8.3	7.1.x; nightly	WP-CLI end to end, single site then multisite. The nightly leg runs the commands on core's implementation.
examples	8.3	7.1.x	make test-examples against Moto (an AWS emulator) and a Vault dev server, both pinned by digest, single site then multisite. Not part of make ci.
reference-docs	8.3	—	bin/gen-reference.php --check: the committed docs/reference/ matches the source. No Composer install.

The full suite runs only on WordPress below 7.2, the versions the plugin serves. On 7.2 and trunk the plugin stands down and core's implementation serves every call, so the full suite there would only test core with core's tests, and it fails whenever a port gets ahead of the nightly build. 7.1.x rather than latest keeps the matrix on 7.1 when 7.2 ships.

The 7.4 leg is not optional. Core's floor is 7.4 and src/ must run there; PHPCompatibilityWP catches syntax statically, but only a running 7.4 catches runtime behaviour differences.

Local wp-env is pinned to PHP 7.4 for the same reason — the floor is where bugs hide, so it is the default you develop against rather than something CI discovers later. Change phpVersion in .wp-env.json to reproduce a failure on a newer PHP.

The top of the matrix follows core's. Until 0.2.1 it stopped at 8.3, and core's suite found what that missed. PHP 8.5 deprecates ReflectionMethod::setAccessible() and ReflectionProperty::setAccessible(), which have done nothing since 8.1, and the test suite treats deprecations as errors. Tests that reach a private member through reflection call setAccessible( true ) only when PHP_VERSION_ID < 80100, as core's own tests do.

Static analysis

make analyse runs PHPStan at level 10, its strictest, over src/, plugin/, cli/, and secrets-api.php. src/ is analyzed a second time inside wordpress-develop, under core's own configuration, and it has to be clean in both places.

Two things in phpstan.neon.dist exist because of that:

• tests/phpstan/GlobalDocBlockVisitor.php is core's file, copied unchanged. Core documents a global with @global wpdb $wpdb on the function's docblock, and core's PHPStan setup reads that tag through this visitor. Without it, $wpdb is untyped here and typed there.
• One ignored message, for interpolating a $wpdb table name into the query given to prepare(). The WordPress stubs ask for a literal string, which core's own signature does not.

The tests are not analyzed here. They need the WordPress test classes, which the static job does not install. The files under tests/phpunit/ that also ship in the core patch are checked at level 10 in wordpress-develop and copied back.

A test that passes the wrong type on purpose carries the comment core's tests use for that:

$result = wp_set_secret( 'myplugin/key', array() ); // @phpstan-ignore argument.type (Intentionally passing an invalid value.)

Publishing the docs site

.github/workflows/docs-publish.yml is a deployment workflow, not part of make ci. On every v*.*.* tag, and on demand, it runs npm run docs:build and publishes site/dist to a fixed Spacefast Space with the Spacefast CLI, using the SPACEFAST_TOKEN repository secret. It publishes once, checks the receipt, and fails without retrying. site/README.md has the details.