diff --git a/docs/testing/packages.md b/docs/testing/packages.md index f48241df1672..7d03db1ad980 100644 --- a/docs/testing/packages.md +++ b/docs/testing/packages.md @@ -59,12 +59,12 @@ The following methods can be overridden: - `getFixtures()` returns a list of directories that are mirrored onto the root of the skeleton, in order, after the framework files. A later directory overwrites an earlier one. Use them to add controllers, routes, configuration or a `Registrar`, for example **Fixtures/app/Config/Routes.php**. A fixture **app/Config/Paths.php** must keep the default `$systemDirectory`, which `build()` then points at the framework. - `getEnv()` returns settings for the skeleton's **.env** file. Values are quoted, so they may contain spaces, `#` and quotes, but a value with a line break is refused. Names may only contain letters, digits, `.`, `_`, `-` and `\`, which covers forms such as `app.baseURL` and `Config\App.baseURL`. `CI_ENVIRONMENT` defaults to `development`, because `spark` does not run in the `testing` environment. -- `getFrameworkPath()` returns the framework that the skeleton is copied from, which is the one that provides the orchestrator. Expose it from your subclass if your tests need the framework's location. +- `getFrameworkPath()` returns the framework that the skeleton is copied from, which is the one that provides the orchestrator. - `afterBuild()` runs when the skeleton is complete, for example to create a database. -- `getPath()` returns the skeleton's location, **build/orchestrator** in the package root by default. A token is appended to it so that parallel processes do not share a skeleton. The token is read from `ORCHESTRATOR_TEST_TOKEN`, or else from `TEST_TOKEN`, which parallel runners such as ParaTest set for each process. It may only contain letters, digits, hyphens and underscores: an invalid `ORCHESTRATOR_TEST_TOKEN` throws an exception, and an invalid `TEST_TOKEN` is ignored. Set `ORCHESTRATOR_TEST_TOKEN` to an empty string to opt out of `TEST_TOKEN`. +- `getSkeletonPath()` returns the skeleton's location, **build/orchestrator** in the package root by default. `getPath()` appends `-console` to it after `bootConsole()`, and then a token so that parallel processes do not share a skeleton. The token is read from `ORCHESTRATOR_TEST_TOKEN`, or else from `TEST_TOKEN`, which parallel runners such as ParaTest set for each process. It may only contain letters, digits, hyphens and underscores: an invalid `ORCHESTRATOR_TEST_TOKEN` throws an exception, and an invalid `TEST_TOKEN` is ignored. Set `ORCHESTRATOR_TEST_TOKEN` to an empty string to opt out of `TEST_TOKEN`. !!! note "Note" - The skeleton is generated, so add its location to your **.gitignore**, which is **build/** unless `getPath()` is overridden. + The skeleton is generated, so add its location to your **.gitignore**, which is **build/** unless `getSkeletonPath()` is overridden. !!! note "Note" The skeleton is copied from the framework installed with your package, so only that version is tested. To build it from another version, keep a copy of that framework in a separate directory and return its path from `getFrameworkPath()`. @@ -90,6 +90,47 @@ Point `bootstrap` at this file in your **phpunit.dist.xml**. Since it replaces C !!! important "Important" The path constants and the framework's static state last for the whole process, so one process hosts one skeleton. +`getPathConstants()` returns the constants that `boot()` defines, keyed by name: `HOMEPATH`, `CONFIGPATH` and `PUBLICPATH` for the skeleton, and `TESTPATH` and, when **tests/_support** exists, `SUPPORTPATH` for the package. Call it after `build()`. Use it with `getFrameworkPath()` to load another bootstrap file against the skeleton: + +```php +$orchestrator->build(); + +foreach ($orchestrator->getPathConstants() as $name => $value) { + defined($name) || define($name, $value); +} + +require $orchestrator->getFrameworkPath() . '/system/Test/bootstrap.php'; +``` + +## Booting Outside PHPUnit + +Code that runs outside PHPUnit, such as a package script or a static analysis bootstrap file, can call `bootConsole()` to use the framework's functions and services, like `env()`, `helper()` and `service()`. It loads the framework's **system/util_bootstrap.php** against a skeleton of its own, in **build/orchestrator-console** by default, so that it does not share a skeleton with a running test suite: + +```php +bootConsole(); +``` + +Unlike `boot()`, it builds the skeleton only when it is missing or a previous build did not finish. A reused skeleton skips `getEnv()` and `afterBuild()`, so work in those hooks must tolerate being skipped. Pass `rebuild: true` to rebuild it after changing your fixtures or updating the framework, since `build()` called before `bootConsole()` rebuilds the PHPUnit skeleton instead, and called after it, changes files that are already loaded: + +```php +(new BlogOrchestrator())->bootConsole(rebuild: true); +``` + +The environment is read from `CI_ENVIRONMENT`, `development` by default, a `CLIRequest` is created, and the routes are loaded, as with **spark**. Routes are loaded last, so an error in a route file is thrown from `bootConsole()`. The test helpers of **system/Test/bootstrap.php**, like the mocked `is_cli()`, are not loaded. + +!!! note "Note" + Processes that boot at the same time share **build/orchestrator-console**, so a build in one removes the skeleton in use by the other. Give each tool its own `ORCHESTRATOR_TEST_TOKEN`, as described under `getSkeletonPath()`. + +After `bootConsole()`, the instance keeps addressing the console skeleton, and `boot()` on it throws an exception. + ## Testing Through spark `spark()` runs the skeleton's **spark** in a separate process and returns a `SparkResult` with the `exitCode`, `stdout`, `stderr` and `timedOut` properties: diff --git a/system/Test/Orchestrator/AbstractOrchestrator.php b/system/Test/Orchestrator/AbstractOrchestrator.php index 4f0d1d118442..0bea90809de6 100644 --- a/system/Test/Orchestrator/AbstractOrchestrator.php +++ b/system/Test/Orchestrator/AbstractOrchestrator.php @@ -28,10 +28,17 @@ abstract class AbstractOrchestrator */ private const KILL_GRACE_PERIOD = 2.0; + private const BUILT_MARKER = '.orchestrator-built'; + + private bool $console = false; + /** - * Absolute path of the skeleton, suffixed with `ORCHESTRATOR_TEST_TOKEN`, or else a valid `TEST_TOKEN`, so parallel runs do not collide. + * Absolute path of the skeleton, which is `getSkeletonPath()` followed by: + * + * - `-console` after `bootConsole()` + * - `-` and `ORCHESTRATOR_TEST_TOKEN`, or else a valid `TEST_TOKEN`, so parallel runs do not collide */ - public function getPath(): string + final public function getPath(): string { $token = getenv('ORCHESTRATOR_TEST_TOKEN'); @@ -47,8 +54,9 @@ public function getPath(): string } return sprintf( - '%s/build/orchestrator%s', - rtrim($this->getPackageRoot(), '\\/'), + '%s%s%s', + rtrim($this->getSkeletonPath(), '\\/'), + $this->console ? '-console' : '', is_string($token) && $token !== '' ? '-' . $token : '', ); } @@ -66,10 +74,11 @@ final public function build(): void $this->ensureUnlinkedAncestors(); $this->removePath($path); - foreach (['app', 'public', 'writable'] as $directory) { + foreach (['app', 'public'] as $directory) { directory_mirror($framework . '/' . $directory, $path . '/' . $directory); } + $this->mirrorWritable($framework . '/writable', $path . '/writable'); $this->failUnless(copy($framework . '/spark', $path . '/spark'), 'copy spark'); $this->writeAutoloader(); $this->writeEnv(); @@ -80,6 +89,8 @@ final public function build(): void $this->pointPathsAtTheFramework(); $this->afterBuild(); + + $this->failUnless(touch($path . '/' . self::BUILT_MARKER), 'mark the build complete'); } /** @@ -89,9 +100,48 @@ final public function build(): void */ final public function boot(): void { + if ($this->console) { + throw new RuntimeException('Cannot call boot() after bootConsole().'); + } + $this->build(); + $this->defineConstants(); + + require $this->getFrameworkPath() . '/system/Test/bootstrap.php'; + } + + /** + * Boots the framework in this process through **system/util_bootstrap.php** against its own skeleton. + * + * `getEnv()` and `afterBuild()` run only when the skeleton is incomplete or `$rebuild` is true. + */ + final public function bootConsole(bool $rebuild = false): void + { + $this->console = true; + + if ($rebuild || ! is_file($this->getPath() . '/' . self::BUILT_MARKER)) { + $this->build(); + } + + $this->defineConstants(); - $path = realpath($this->getPath()) . DIRECTORY_SEPARATOR; + require $this->getFrameworkPath() . '/system/util_bootstrap.php'; + } + + /** + * Path constants of the built skeleton and the package, keyed by name. + * + * @return array + */ + public function getPathConstants(): array + { + $path = $this->getRealRoot(); + + if ($path === '') { + throw new RuntimeException('Cannot read the path constants before the orchestrated application is built.'); + } + + $path .= DIRECTORY_SEPARATOR; $testPath = rtrim($this->getPackageRoot(), '\\/') . '/tests' . DIRECTORY_SEPARATOR; $constants = [ 'HOMEPATH' => $path, @@ -104,13 +154,7 @@ final public function boot(): void $constants['SUPPORTPATH'] = $testPath . '_support' . DIRECTORY_SEPARATOR; } - foreach ($constants as $name => $value) { - if (! defined($name)) { - define($name, $value); - } - } - - require $this->getFrameworkPath() . '/system/Test/bootstrap.php'; + return $constants; } /** @@ -206,11 +250,27 @@ final public function clean(): void $this->removePath($this->getPath()); } + /** + * Absolute path of the framework that the skeleton is copied from. + */ + public function getFrameworkPath(): string + { + return dirname(__DIR__, 3); + } + /** * Absolute path of the package under test, the directory holding its `vendor/autoload.php`. */ abstract protected function getPackageRoot(): string; + /** + * Absolute path of the skeleton before the suffixes added by `getPath()`. + */ + protected function getSkeletonPath(): string + { + return rtrim($this->getPackageRoot(), '\\/') . '/build/orchestrator'; + } + /** * Absolute paths of directories mirrored onto the skeleton, in order, after the framework files. * @@ -238,12 +298,13 @@ protected function afterBuild(): void { } - /** - * Absolute path of the framework that the skeleton is copied from. - */ - protected function getFrameworkPath(): string + private function defineConstants(): void { - return dirname(__DIR__, 3); + foreach ($this->getPathConstants() as $name => $value) { + if (! defined($name)) { + define($name, $value); + } + } } private function isValidToken(string $token): bool @@ -395,6 +456,33 @@ private function wait(mixed $process, ?float $timeout): array return [$exitCode, $timedOut]; } + private function mirrorWritable(string $origin, string $target): void + { + $this->failUnless(is_dir($target) || mkdir($target, 0755, true), 'create writable'); + + foreach (['index.html', '.htaccess'] as $file) { + if (is_file($origin . '/' . $file)) { + $this->failUnless(copy($origin . '/' . $file, $target . '/' . $file), sprintf('copy writable/%s', $file)); + } + } + + $entries = scandir($origin); + + $this->failUnless($entries !== false, 'read writable'); + + foreach (array_diff($entries, ['.', '..']) as $directory) { + if (! is_file($origin . '/' . $directory . '/index.html')) { + continue; + } + + $this->failUnless(mkdir($target . '/' . $directory, 0755), sprintf('create writable/%s', $directory)); + $this->failUnless( + copy($origin . '/' . $directory . '/index.html', $target . '/' . $directory . '/index.html'), + sprintf('copy writable/%s/index.html', $directory), + ); + } + } + private function pointPathsAtTheFramework(): void { $paths = $this->getPath() . '/app/Config/Paths.php'; @@ -405,7 +493,7 @@ private function pointPathsAtTheFramework(): void $rewritten = str_replace( "__DIR__ . '/../../system'", var_export($this->getFrameworkPath() . '/system', true), - (string) $contents, + $contents, $count, ); @@ -446,6 +534,9 @@ private function writeEnv(): void $this->failUnless(write_file($this->getPath() . '/.env', $contents), 'write .env'); } + /** + * @phpstan-assert true $succeeded + */ private function failUnless(bool $succeeded, string $step): void { if (! $succeeded) { diff --git a/system/util_bootstrap.php b/system/util_bootstrap.php index 135208a78ea9..eabb0dac0096 100644 --- a/system/util_bootstrap.php +++ b/system/util_bootstrap.php @@ -65,8 +65,8 @@ } if (is_dir(HOMEPATH . 'vendor/')) { - define('VENDORPATH', realpath(HOMEPATH . 'vendor/') . DIRECTORY_SEPARATOR); - define('COMPOSER_PATH', (string) realpath(HOMEPATH . 'vendor/autoload.php')); + defined('VENDORPATH') || define('VENDORPATH', realpath(HOMEPATH . 'vendor/') . DIRECTORY_SEPARATOR); + defined('COMPOSER_PATH') || define('COMPOSER_PATH', (string) realpath(HOMEPATH . 'vendor/autoload.php')); } /* diff --git a/tests/_support/Orchestrator/OrchestratorStub.php b/tests/_support/Orchestrator/OrchestratorStub.php index d15acc284aed..e2cb4b5e791d 100644 --- a/tests/_support/Orchestrator/OrchestratorStub.php +++ b/tests/_support/Orchestrator/OrchestratorStub.php @@ -31,9 +31,9 @@ public function __construct( ) { } - public function getPath(): string + protected function getSkeletonPath(): string { - return $this->skeleton !== '' ? $this->skeleton : parent::getPath(); + return $this->skeleton !== '' ? $this->skeleton : parent::getSkeletonPath(); } protected function getPackageRoot(): string diff --git a/tests/system/Test/Orchestrator/AbstractOrchestratorTest.php b/tests/system/Test/Orchestrator/AbstractOrchestratorTest.php index 06188f4195f0..74b98d1b7a29 100644 --- a/tests/system/Test/Orchestrator/AbstractOrchestratorTest.php +++ b/tests/system/Test/Orchestrator/AbstractOrchestratorTest.php @@ -20,6 +20,7 @@ use PHPUnit\Framework\Attributes\CoversClass; use PHPUnit\Framework\Attributes\DataProvider; use PHPUnit\Framework\Attributes\Group; +use PHPUnit\Framework\MockObject\MockObject; use Tests\Support\Orchestrator\OrchestratorStub; /** @@ -73,6 +74,32 @@ public function testBuildCreatesTheSkeleton(): void } } + public function testBuildCopiesOnlyTheShippedWritableSkeleton(): void + { + $stray = WRITEPATH . 'orchestrator-stray-' . bin2hex(random_bytes(4)); + + mkdir($stray); + file_put_contents($stray . '/runtime.log', 'runtime state'); + file_put_contents($stray . '.txt', 'runtime state'); + + try { + self::$orchestrator->build(); + } finally { + unlink($stray . '/runtime.log'); + rmdir($stray); + unlink($stray . '.txt'); + } + + $writable = self::$orchestrator->getPath() . '/writable'; + + foreach (['.htaccess', 'index.html', 'cache/index.html', 'debugbar/index.html', 'logs/index.html', 'session/index.html', 'uploads/index.html'] as $entry) { + $this->assertFileExists($writable . '/' . $entry); + } + + $this->assertFileDoesNotExist($writable . '/' . basename($stray)); + $this->assertFileDoesNotExist($writable . '/' . basename($stray) . '.txt'); + } + public function testBuildPointsPathsAtTheFramework(): void { $contents = (string) file_get_contents(self::$orchestrator->getPath() . '/app/Config/Paths.php'); @@ -132,9 +159,11 @@ public function testBootDefinesThePathsAndLoadsTheSkeletonInAFreshProcess(): voi var_export($root, true), )); - exec(sprintf('%s %s 2>&1', escapeshellarg(PHP_BINARY), escapeshellarg($root . '/boot.php')), $output, $exitCode); + [$exitCode, $output] = $this->runInFreshProcess($root . '/boot.php'); + $this->assertSame(0, $exitCode, $output); - $this->assertSame(0, $exitCode, implode("\n", $output)); + $result = json_decode($output, true); + $this->assertIsArray($result, $output); $skeleton = realpath((new OrchestratorStub($root))->getPath()) . DIRECTORY_SEPARATOR; @@ -148,7 +177,165 @@ public function testBootDefinesThePathsAndLoadsTheSkeletonInAFreshProcess(): voi 'SUPPORTPATH' => $root . '/tests/_support' . DIRECTORY_SEPARATOR, 'env' => 'a b #c "d" \\e', 'fixture' => true, - ], json_decode(implode("\n", $output), true)); + ], $result); + } + + public function testBootConsoleBootsItsOwnSkeletonThroughUtilBootstrap(): void + { + $root = self::$buildPath . '/console-root'; + + $this->createPackageRoot($root); + mkdir($root . '/tests/_support', 0755, true); + file_put_contents($root . '/console.php', sprintf( + <<<'PHP' + bootConsole(rebuild: ($argv[1] ?? '') === 'rebuild'); + + echo json_encode([ + 'path' => $orchestrator->getPath(), + 'HOMEPATH' => HOMEPATH, + 'APPPATH' => APPPATH, + 'SYSTEMPATH' => SYSTEMPATH, + 'TESTPATH' => TESTPATH, + 'SUPPORTPATH' => SUPPORTPATH, + 'VENDORPATH' => VENDORPATH, + 'env' => env('orchestrator.quoted'), + 'request' => service('request') instanceof CodeIgniter\HTTP\CLIRequest, + 'builds' => $orchestrator->afterBuildCalls, + ]); + + PHP, + var_export(ROOTPATH . 'vendor/autoload.php', true), + var_export(SUPPORTPATH . 'Orchestrator/OrchestratorStub.php', true), + var_export($root, true), + )); + + [$exitCode, $output] = $this->runInFreshProcess($root . '/console.php'); + $this->assertSame(0, $exitCode, $output); + + $result = json_decode($output, true); + $this->assertIsArray($result, $output); + $this->assertArrayHasKey('path', $result); + $this->assertIsString($result['path']); + + $skeleton = realpath($result['path']) . DIRECTORY_SEPARATOR; + $this->assertStringContainsString('/build/orchestrator-console', $result['path']); + + $this->assertNotSame((new OrchestratorStub($root))->getPath(), $result['path']); + $this->assertSame([ + 'path' => $result['path'], + 'HOMEPATH' => $skeleton, + 'APPPATH' => $skeleton . 'app' . DIRECTORY_SEPARATOR, + 'SYSTEMPATH' => SYSTEMPATH, + 'TESTPATH' => $root . '/tests' . DIRECTORY_SEPARATOR, + 'SUPPORTPATH' => $root . '/tests/_support' . DIRECTORY_SEPARATOR, + 'VENDORPATH' => '/predefined/vendor/', + 'env' => 'a b #c "d" \\e', + 'request' => true, + 'builds' => 1, + ], $result); + + file_put_contents($result['path'] . '/stale', ''); + + $this->assertSame([0, 0], $this->countBuilds($this->runInFreshProcess($root . '/console.php'))); + $this->assertFileExists($result['path'] . '/stale'); + + unlink($result['path'] . '/.orchestrator-built'); + + $this->assertSame([0, 1], $this->countBuilds($this->runInFreshProcess($root . '/console.php'))); + $this->assertFileDoesNotExist($result['path'] . '/stale'); + + file_put_contents($result['path'] . '/stale', ''); + + $this->assertSame([0, 1], $this->countBuilds($this->runInFreshProcess($root . '/console.php', 'rebuild'))); + $this->assertFileDoesNotExist($result['path'] . '/stale'); + $this->assertDirectoryDoesNotExist((new OrchestratorStub($root))->getPath()); + } + + public function testBootConsoleLeavesAFailedBuildUnmarked(): void + { + $orchestrator = $this->createOrchestratorWithAFailingBuild(self::$buildPath . '/console-failure-root'); + + try { + $orchestrator->bootConsole(); + $this->fail('bootConsole() should have rethrown the build failure.'); + } catch (RuntimeException $e) { + $this->assertSame('The hook failed.', $e->getMessage()); + } + + $this->assertStringContainsString('/build/orchestrator-console', $orchestrator->getPath()); + $this->assertDirectoryExists($orchestrator->getPath()); + $this->assertFileDoesNotExist($orchestrator->getPath() . '/.orchestrator-built'); + } + + public function testBootRefusesAnOrchestratorInConsoleMode(): void + { + $orchestrator = $this->createOrchestratorWithAFailingBuild(self::$buildPath . '/console-boot-root'); + + try { + $orchestrator->bootConsole(); + } catch (RuntimeException $e) { + $this->assertSame('The hook failed.', $e->getMessage()); + } + + $this->expectException(RuntimeException::class); + $this->expectExceptionMessage('Cannot call boot() after bootConsole().'); + + $orchestrator->boot(); + } + + public function testPathSuffixesAnOverriddenSkeletonPath(): void + { + $root = self::$buildPath . '/console-override-root'; + + $this->createPackageRoot($root); + + $orchestrator = $this->getMockBuilder(AbstractOrchestrator::class) + ->onlyMethods(['getSkeletonPath', 'getPackageRoot', 'afterBuild']) + ->getMock(); + $orchestrator->method('getSkeletonPath')->willReturn($root . '/skeleton/'); + $orchestrator->method('getPackageRoot')->willReturn($root); + $orchestrator->method('afterBuild')->willThrowException(new RuntimeException('The hook failed.')); + + $this->withTokens(['ORCHESTRATOR_TEST_TOKEN' => '7'], function () use ($orchestrator, $root): void { + $this->assertSame($root . '/skeleton-7', $orchestrator->getPath()); + + try { + $orchestrator->bootConsole(); + $this->fail('bootConsole() should have rethrown the build failure.'); + } catch (RuntimeException $e) { + $this->assertSame('The hook failed.', $e->getMessage()); + } + + $this->assertSame($root . '/skeleton-console-7', $orchestrator->getPath()); + }); + } + + public function testGetPathConstantsDescribeTheBuiltSkeleton(): void + { + $skeleton = realpath(self::$orchestrator->getPath()) . DIRECTORY_SEPARATOR; + + $this->assertSame([ + 'HOMEPATH' => $skeleton, + 'CONFIGPATH' => $skeleton . 'app/Config' . DIRECTORY_SEPARATOR, + 'PUBLICPATH' => $skeleton . 'public' . DIRECTORY_SEPARATOR, + 'TESTPATH' => self::$buildPath . '/tests' . DIRECTORY_SEPARATOR, + ], self::$orchestrator->getPathConstants()); + } + + public function testGetPathConstantsRefuseAMissingSkeleton(): void + { + $this->expectException(RuntimeException::class); + $this->expectExceptionMessage('Cannot read the path constants before the orchestrated application is built.'); + + (new OrchestratorStub(self::$buildPath . '/unbuilt'))->getPathConstants(); } public function testBuildWritesTheEnvFile(): void @@ -654,6 +841,46 @@ public function testSparkDrainsLargeOutputOfBothStreams(): void $this->assertStringEndsWith(str_repeat('e', 300_000), $result->stderr); } + private function createOrchestratorWithAFailingBuild(string $root): AbstractOrchestrator&MockObject + { + $this->createPackageRoot($root); + + $orchestrator = $this->getMockBuilder(AbstractOrchestrator::class) + ->onlyMethods(['getPackageRoot', 'afterBuild']) + ->getMock(); + $orchestrator->method('getPackageRoot')->willReturn($root); + $orchestrator->method('afterBuild')->willThrowException(new RuntimeException('The hook failed.')); + + return $orchestrator; + } + + /** + * @return array{int, string} + */ + private function runInFreshProcess(string $script, string ...$arguments): array + { + exec(sprintf( + '%s 2>&1', + implode(' ', array_map(escapeshellarg(...), [PHP_BINARY, $script, ...$arguments])), + ), $output, $exitCode); + + return [$exitCode, implode("\n", $output)]; + } + + /** + * @param array{int, string} $run + * + * @return array{int, mixed} + */ + private function countBuilds(array $run): array + { + [$exitCode, $output] = $run; + + $result = json_decode($output, true); + + return [$exitCode, is_array($result) ? $result['builds'] : $output]; + } + private function createPackageRoot(string $root): void { mkdir($root . '/vendor', 0755, true);