Repository navigation
feat: add an orchestration layer for testing packages - #10727
Conversation
0258881 to
bb1c0cd
Compare
|
Have you looked at the forum? This is a very useful feature; I haven’t added it to the framework yet, as such innovations aren’t always added to production. |
neznaika0
left a comment
There was a problem hiding this comment.
I don’t fully understand the entire process, but I grasped the main idea.
All my tests passed on Arch.
- The reason for using two vendor/autoload.php files is unclear.
- The documentation should describe the limitations — the current version specified in the project is being tested. That to test another version, you need a copy of the framework in some folder.
- I tried to launch spark directly after creation, but nothing happens. Is this correct?
orchestrator/$ ORCHESTRATOR_FLOOD=1 php spark orchestrator:pro
be
CodeIgniter v4.8.0-dev Command Line Tool - Server Time: 2026-11-05 19:26:58 UTC+00:00That’s all for now; we need to try it with a real package.
bb1c0cd to
741b010
Compare
It's a shim to the package/library's own vendor/autoload.php. Instead of creating the skeleton's own autoload.php, it instead reuses the library's own autoload.php, which then boots the orchestrator.
Added a note admonition in the docs.
If you have not called |
|
Shield can be a first-party consumer to this feature. I tried this on my clone of 1. The orchestrator // tests/_support/ShieldOrchestrator.php
namespace Tests\Support;
use CodeIgniter\Test\Orchestrator\AbstractOrchestrator;
final class ShieldOrchestrator extends AbstractOrchestrator
{
protected function getPackageRoot(): string
{
return dirname(__DIR__, 2);
}
protected function getEnv(): array
{
return [
'app.baseURL' => 'https://example.com/',
'auth.hashCost' => '4',
];
}
}2. The bootstrap // tests/bootstrap.php
require_once __DIR__ . '/../vendor/autoload.php';
require_once __DIR__ . '/_support/ShieldOrchestrator.php';
(new Tests\Support\ShieldOrchestrator())->boot();3. - bootstrap="vendor/codeigniter4/framework/system/Test/bootstrap.php"
+ bootstrap="tests/bootstrap.php"
...
- <const name="HOMEPATH" value="./"/>
- <const name="CONFIGPATH" value="vendor/codeigniter4/framework/app/Config/"/>
- <const name="PUBLICPATH" value="./public/"/>The What I ran A probe test, extending Shield's existing public function testShieldConfigIsDiscoveredInProcess(): void
{
$this->assertInstanceOf(Auth::class, config('Auth'));
$this->assertStringContainsString('/build/orchestrator/app/', APPPATH);
$this->assertSame('https://example.com/', config('App')->baseURL);
}
public function testShieldCommandsAreDiscoveredBySpark(): void
{
$result = (new ShieldOrchestrator())->spark(['list']);
$this->assertSame(0, $result->exitCode);
$this->assertStringContainsString('shield:setup', $result->stdout);
}Both pass. The package namespace is found with no fixture or I also ran these existing Shield tests under the new bootstrap: What this does not cover
What it adds is |
neznaika0
left a comment
There was a problem hiding this comment.
The choice of var_export() for strings is a bit unclear.
Overall, I like it.
There was a problem hiding this comment.
🟡 Changes recommended
A symlinked ancestor can redirect clean() and build() into recursively deleting an external directory, leaving a critical filesystem-safety flaw.
3 open issues
What changed in this PR
Adds a package-testing orchestration layer that builds disposable CodeIgniter applications, allowing libraries to test against real framework configuration and commands.
Changes:
- Introduces skeleton creation, PHPUnit bootstrapping, fixture overlays, and guarded file operations.
- Adds subprocess
spark()execution with input, environment overrides, timeout handling, and structured results. - Adds tests, fixtures, and user-guide documentation.
Validation was limited to source inspection; automated tests and full validation remain for CI.
| File | Description |
|---|---|
tests/system/Test/Orchestrator/AbstractOrchestratorTest.php |
Tests skeleton operations, path safety, environment handling, and subprocess execution. |
tests/_support/Orchestrator/OrchestratorStub.php |
Provides a configurable orchestrator test double. |
tests/_support/Orchestrator/Fixtures/override.txt |
Supplies the initial fixture-overlay value. |
tests/_support/Orchestrator/Fixtures/app/Commands/OrchestratorProbe.php |
Exercises subprocess input, output, failures, and timeouts. |
tests/_support/Orchestrator/ExtraFixtures/override.txt |
Verifies later fixtures overwrite earlier files. |
tests/_support/Orchestrator/ExtraFixtures/extra.txt |
Verifies additional fixture files are copied. |
system/Test/Orchestrator/SparkResult.php |
Holds process output, exit code, and timeout status. |
system/Test/Orchestrator/AbstractOrchestrator.php |
Implements application orchestration and subprocess execution. |
docs/testing/packages.md |
Documents setup, customization, testing, and Worker Mode limitations. |
docs/testing/index.md |
Links the package-testing guide. |
docs/index.md |
Adds the guide to the main documentation index. |
docs/changelogs/v4.8.0.md |
Announces the new testing API. |
docs/.nav.yml |
Adds the guide to documentation navigation. |
Review effort: Balanced
741b010 to
02031c4
Compare
When you |
michalsn
left a comment
There was a problem hiding this comment.
Looks good on paper. I guess we will have to test it out with one of our packages to catch any issues.


Description
Libraries that supplement CodeIgniter have no standard way to replicate an appstarter environment in their tests. Each package copies the same
<const>block intophpunit.xml.dist, pointsCONFIGPATHatvendor/codeigniter4/framework/app/Config, and fakes everything else intests/_support. A customisedapp/, a.envfile, or a realsparkrun cannot be tested.This adds
CodeIgniter\Test\Orchestrator\AbstractOrchestrator. A package extends it, implementsgetPackageRoot(), and gets a throwaway skeleton built from the installed framework:build()copiesapp,public,writableandsparkfrom the framework, pointsPaths.phpat itssystemdirectory, writes a.env, shares the package's autoloader through avendor/autoload.phpshim, mirrors the fixture directories, then callsafterBuild().boot()builds the skeleton, defines the path constants against it, and loadssystem/Test/bootstrap.php, soCIUnitTestCasetests run unchanged against it.spark()runs the skeleton'ssparkin a separate process and returns aSparkResultwith the exit code,stdout,stderrandtimedOut. It acceptsstdin, environment overrides and a timeout.write(),delete()andclean()change or remove the skeleton. They refuse paths that leave it, including through..and symlinks.getFixtures(),getEnv(),getPath(),getFrameworkPath()andafterBuild().Parallel runs get separate skeletons from
ORCHESTRATOR_TEST_TOKEN, or fromTEST_TOKENas set by ParaTest.Not covered: starting FrankenPHP for Worker Mode. The user guide shows how to publish the worker files with
worker:installfromafterBuild().Checklist: