Skip to content

Tools: Add PHPStan static analysis at rule level 5 - #66693

Merged
westonruter merged 111 commits into
WordPress:trunkfrom
justlevine:tools/phpstan
Oct 31, 2026
Merged

westonruter merged 111 commits into
WordPress:trunkfrom
justlevine:tools/phpstan

Conversation

@justlevine

@justlevine justlevine commented Nov 2, 2024 •

Copy link
Copy Markdown
Contributor

What?

Closes #66598

This adds PHPStan static analysis to Gutenberg, at rule level 5, with a baseline for the existing errors, and runs it in CI. It covers the plugin's own PHP and all the PHP that WordPress core ships from this repository, including the wp-build PHP templates.

Why?

WordPress core has run PHPStan since Core-61175, and is now at rule level 5. Much of core's PHP comes from Gutenberg, though, and core cannot analyze the most important part of it: the block PHP, core-fields, core-actions, and the PHP generated from the wp-build templates are copied into core verbatim and excluded from core's analysis, since they are generated there. Gutenberg is the only place this code can be analyzed, because it is the only place it can be fixed.

For the PHP in lib/ that is backported to core, analyzing it here catches problems before they reach core, and keeps the two codebases in sync as fixes are made on either side. The PHPStan fixes that preceded this PR (#68386, #68389, #68392, #68396, #68397, #68398, #68399, #68401, #68404, #68405, #68407) found, among others, a bug in core's shadow block support and a fatal error in WP_Theme_JSON::get_feature_declarations_for_node(), both since fixed in core in r64727.

How?

This was originally opened by @justlevine with per-level baselines and custom handling for the generated functions. It has been reworked to follow core's setup more closely:

  • Dependencies: PHPStan 2 with szepeviktor/phpstan-wordpress 2 and the WordPress stubs (see below). As in core, these are pinned to exact versions: there is no Composer lock file, so a new release of PHPStan or of the stubs could otherwise change the results and fail CI on every pull request at once, whereas pinned, an upgrade is a deliberate change that regenerates the baseline. The Composer scripts are composer phpstan and composer phpstan:baselines, and the npm scripts typecheck:php and typecheck:php:baselines run them in wp-env, as lint:php does for PHPCS; all four are named as in core. The bare typecheck script remains the TypeScript check, since it runs in CI and in editor workflows without wp-env, the way lint does not include lint:php.
  • Rule level 5 with a single baseline: phpstan.neon.dist sets level 5, as core does, with the 226 existing errors in one standard tools/phpstan/baseline.neon. reportUnmatchedIgnoredErrors is on, so a fixed error has to be removed from the baseline too. composer phpstan:baselines always uses phpstan.neon.dist, so a local phpstan.neon, such as one with a higher level, can never end up in the baseline.
  • Tests (opt-in): the tests in phpunit/ are not analyzed by default, since they do not ship in core, but phpstan/phpstan-phpunit, which core also uses, and php-stubs/wordpress-tests-stubs, for the WordPress test library that wp-env provides at runtime, are installed. A local phpstan.neon that adds phpunit to the paths can opt in to them, as the coding guidelines show.
  • Settings matching core: phpVersion covers PHP 7.4 through 8.5, so the results do not depend on the PHP version PHPStan runs on, and treatPhpDocTypesAsCertain is false, since PHPDoc types are not enforced at runtime.
  • What is analyzed: gutenberg.php and lib/, plus the PHP in block-library, the widget blocks, core-fields, core-actions, style-engine, block-serialization-default-parser, and icons.
  • Generated functions and classes: lib/ calls many gutenberg_* functions and *_Gutenberg classes that only exist once the build has renamed them into build/. That directory is scanned, so they are known with their real signatures, and the original ignore rule for unknown gutenberg_* functions is no longer needed. This means npm run build has to run before composer phpstan.
  • WordPress stubs generated from core trunk, on demand: Gutenberg tracks core trunk, so the stubs released for a WordPress version lack what trunk has added since, and would have the analysis report code that uses it. Until php-stubs/wordpress-stubs publishes stubs built from trunk, tools/phpstan/generate-wordpress-stubs.php generates them with its generator, currently from core trunk at r65047. The generator is pinned to the head of Preserve existing phpstan-return tags when applying function map php-stubs/wordpress-stubs#498, without which it adds a second @phpstan-return to the docblocks that now have their own in core, and the packages that determine its output, such as nikic/php-parser and phpdocumentor/reflection-docblock, are pinned to exact versions, so that a new release of one cannot change the stubs without a commit here. The stubs live in tools/phpstan/wordpress-stubs/ as a Composer path package named php-stubs/wordpress-stubs, which replaces the released package, since szepeviktor/phpstan-wordpress loads the stubs from that package's path. The path repository symlinks it, since the stubs are generated after Composer installs the package. The stubs themselves are not committed. At 6.3 MB, every update would carry them as a diff. Instead, the package's composer.json records the source they are generated from, and composer phpstan generates them before analyzing, returning at once when they are up to date. The stub header records a key for the source, the generator, its pinned packages, and the script itself, so a pull that changes any of them generates the stubs again. Generating needs Git, and network access unless the commit is in the wordpress-develop checkout that Gutenberg is in. The script is written in PHP rather than Bash, since every contributor who runs PHPStan now runs it, including on Windows. The generator is set up once in .cache/wordpress-stubs-generator/, in a directory named for its commit and pinned packages, and setups that have not been used for a week are removed. While generating, the script reports the source, the generator, and the result in a few rows, along with the generator's warnings and any declarations it found more than once, of which the stubs keep the first. When the stubs are missing, such as when vendor/bin/phpstan is run directly, as an IDE does, wordpress-stubs.php throws an exception explaining how to generate them. The source is changed with an option of composer phpstan:generate-wordpress-stubs: --commit=<ref>, such as trunk; --pr=<number>, for the head of a wordpress-develop pull request; or --checkout, for the working tree of the wordpress-develop checkout, including uncommitted changes. Stubs generated from uncommitted changes are only used locally, since CI cannot reproduce them, so they have to be generated with --pr or --commit before pushing. This supports companion pull requests: a Gutenberg change that depends on a wordpress-develop pull request records it as the source, so that the analysis knows what it adds, and switches to --commit=trunk once the core change is committed. After changing the source, run composer update and composer phpstan:baselines, and commit the package's composer.json. Scanning the checkout's src/ directly was also tried, since PHPStan prefers scanned declarations to the stubs, but the stubs are not a plain copy of core: the generator adds the types from the function map of wordpress-stubs, so against raw src/, 32 baseline entries no longer matched and 2 new errors were reported. Core's global type aliases, which the stubs could not carry, were removed in r64756 for this, and the invalid PHPDoc types in core's AVIF info class, which the generator had to correct, were fixed in r65047. Generating the stubs here is meant to be temporary; see the roadmap below.
  • Code for older WordPress versions: with stubs from trunk, the code that only WordPress 7.0 or 7.1 needs can look wrong. The two cases are ignored inline with the reason, rather than baselined, so that they are not taken for errors to fix: the block that wp_get_elements_class_name() requires in 7.0 but no longer takes, and the method_exists() guard for a WP_REST_Comments_Controller method added in 7.2. The flip side is that calling something added since 7.0 without a guard is not reported.
  • Polyfills: tools/phpstan/polyfills.stub declares the polyfills that WordPress trunk provides for newer PHP functions, such as str_contains(), array_any(), and clamp(). The stubs leave out compat.php, and could not include it, since they are executed as a bootstrap file, where a polyfill would redeclare a function the running PHP has.
  • The wp-build templates: these contain {{PLACEHOLDER}} tokens, so they are not valid PHP until the build replaces them. tools/phpstan/src/WpBuildTemplateParser.php wraps the parsers PHPStan uses for analyzed files and for discovering the functions a file declares, and substitutes a representative value for each token first. No substitution spans more than one line, so errors, including syntax errors, are reported on the template lines themselves rather than on each file generated from them. The templates are rendered with a prefix that no built file uses, so that a call in a template resolves to the template's own function rather than to the one built from it in build/, which may be out of date.
  • CI: a PHP static analysis job in the unit test workflow reuses the build-assets artifact that the PHP unit test jobs already download, so the build does not run twice. Errors are reported as annotations on the pull request, and the job is part of the status check. Since an annotation only appears in the diff when its line was changed, and GitHub shows at most ten per step, the job summary also lists every error with a link to its line, up to 500, with the errors that belong to no file first, and explains an out-of-date baseline, the usual error with no changed line to attach to: when a baselined error occurs fewer times, it is expected after fixing it, and composer phpstan:baselines resolves it, unless other errors are reported too, which are to be fixed first; when it occurs more times, the pull request likely added an occurrence, which should be fixed rather than baselined. The generated stubs are cached by their recorded source and the script, restored in every run, and saved only on pushes to trunk; on a miss, the job generates them, which takes about 25 seconds. The PHPStan result cache is likewise restored in every run, but only saved on pushes to trunk, since a cache saved for a pull request could only be restored by that pull request. It is keyed on the configuration, dependencies, and generated stubs, but not the baseline, which does not change the analysis. Unlike in core, errors stay errors rather than warnings, since Gutenberg merges on GitHub and relies on its checks to keep new errors out. Zizmor reports every cache in this workflow as open to cache poisoning, including those the PHP unit test and PHPCS jobs already restore, because it runs on pushes to the release/* branches and so is taken to be a release workflow. Nothing built in it is published, so that finding is ignored, with the reasoning in a comment on the push trigger. The version comments on the ramsey/composer-install pins are also corrected to 4.0.0, the tag the pinned commit has.
  • Docs: the coding guidelines explain that the stubs are generated rather than committed, and describe how to run the analysis, regenerate the baseline, update the stubs, and generate them from a core pull request.

Follow-ups

This PR analyzes the PHP that ships, in the plugin and in core. The rest of the PHP in the repository could be analyzed in follow-up PRs:

  • Tests: the PHPUnit tests in phpunit/, which can already be opted in to locally. At level 5, they add only a few errors to the baseline.
  • End-to-end test plugins: the 171 files in packages/e2e-tests/plugins and packages/e2e-tests/mu-plugins, which wp-env loads for the end-to-end tests.
  • Coding standards: the custom PHPCS sniffs and their tests in test/php/gutenberg-coding-standards, which would need the PHPCS classes in vendor to be scanned.
  • wp-env runtime: packages/env/lib/runtime/php/install.php and router.php, which are new in trunk.
  • Test themes and fixtures: the themes in test/emptytheme and test/gutenberg-test-themes, and tools/validation/fixtures/theme-compatibility/theme-compatibility.php.
  • Parser test runners: test/test-parser.php in block-serialization-default-parser, which is excluded on purpose for now, and in block-serialization-spec-parser.
  • The PHPStan tooling itself: the files in tools/phpstan, including the template parser.

The analysis of code for older WordPress versions could also be improved:

  • Reporting what the minimum WordPress version lacks: with stubs from trunk, calling a function added after 7.0 without a guard is not reported, and declaring a polyfill conditionally would not change that, since PHPStan discovers a declaration without evaluating its condition. A custom rule could report it instead: collect each call to a function whose @since is newer than the plugin's Requires at least, outside a function_exists() guard, along with the functions that Gutenberg itself declares, such as those in lib/compat, and report the calls to functions that Gutenberg does not declare. The polyfills in tools/phpstan/polyfills.stub would get @since tags for it, and methods and classes could be checked the same way, with their method_exists() and class_exists() guards.

Roadmap for the WordPress stubs

Generating the stubs in Gutenberg is a stopgap. The goal is for php-stubs/wordpress-stubs to publish the stubs Gutenberg needs, so that Gutenberg can require them like any other dependency:

  1. Fix the generator for trunk: Preserve existing phpstan-return tags when applying function map php-stubs/wordpress-stubs#498 stops the function map from adding a second @phpstan-return to the docblocks that now have their own in core. Until it is merged, the generator is pinned to the head of that PR, and then to the release that includes it.
  2. Publish builds from trunk: propose that wordpress-stubs publish stubs built from core trunk, such as nightly. Each build needs to be pinnable, ideally by a version of its own derived from the SVN revision, like the 7.2.0-beta3.65047 used here, or at least by commit. A moving dev- branch alone would not do: Gutenberg has no Composer lock file, so a floating build would let any change in core trunk fail the analysis on every pull request at once.
  3. Require a pinned build: Gutenberg then requires an exact build and removes tools/phpstan/generate-wordpress-stubs.php, the tools/phpstan/wordpress-stubs/ package and its path repository in composer.json, the phpstan:generate-wordpress-stubs script and its call in composer phpstan, and the steps that cache and generate the stubs in CI. tools/phpstan/polyfills.stub stays, since the upstream stubs also leave out compat.php, as they are executed as a bootstrap file, and so do the inline ignores for WordPress 7.0, since the stubs are still newer than the minimum supported version. Generating stubs from a core pull request, for companion pull requests, would still need the generator, unless the upstream builds cover pull requests too, so a slimmed-down script may be kept for that alone.
  4. Keep the stubs current: a scheduled workflow bumps the pinned build, regenerates the baseline, and opens a pull request. Until step 3, the same workflow can run composer phpstan:generate-wordpress-stubs -- --commit=trunk instead.

Testing Instructions

  1. Run composer install and npm run build. Have Git available, and network access unless Gutenberg is in a wordpress-develop checkout that has r65047.
  2. Run composer phpstan, and confirm that it first generates the WordPress stubs in tools/phpstan/wordpress-stubs/, then reports no errors. Run it again, and confirm that it skips generating them.
  3. Introduce an error in one of the analyzed files, such as passing an integer to str_contains() in a file in lib/, and confirm that it is reported. Do the same in a template in packages/wp-build/templates, and confirm that the error is reported on the template line.
  4. Remove an entry from tools/phpstan/baseline.neon, and confirm that the error it covered is reported again. Then run composer phpstan:baselines, and confirm that the entry is restored.
  5. Confirm that the PHP static analysis job passes in CI.

Testing Instructions for Keyboard

Not applicable, since there are no UI changes.

Screenshots or screencast

Initial PHPStan run

This shows the worst case, where wordpress-stubs needs to be installed and the stubs need to be generated:

run-with-stub-install-and-stub-generation.mov

PHPStan run with stale wordpress-stubs

This shows where wordpress-stubs is installed, but the stubs need to be re-generated:

run-with-stub-generation.mov

PHPStan run with with stubs generated

This is the normal case where the stubs are already generated:

run-with-stubs-cached.mov

Use of AI Tools

The rework was done with Claude Code (Claude Opus 5.5): upgrading the configuration, writing the template parser and the polyfill stubs, generating the WordPress stubs from core trunk, and on demand with a PHP generator, adding the CI job, regenerating the baseline, and drafting the commit messages and this description, under the direction and review of @westonruter.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 2, 2024

Copy link
Copy Markdown

👋 Thanks for your first Pull Request and for helping build the future of Gutenberg and WordPress, @justlevine! In case you missed it, we'd love to have you join us in our Slack community.

If you want to learn more about WordPress development in general, check out the Core Handbook full of helpful information.

@github-actions github-actions Bot added the First-time Contributor Pull request opened by a first-time contributor to Gutenberg repository label Nov 2, 2024
Comment thread phpstan.neon.dist Resolved
@westonruter

Copy link
Copy Markdown
Member

I've added demo videos to the description showing the different developer experiences when running PHPStan as it relates to the stubs.

@westonruter
westonruter merged commit 877af6a into WordPress:trunk Oct 31, 2026
95 of 96 checks passed
@github-actions github-actions Bot added this to the Gutenberg 24.4 milestone Oct 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please move this file to some workspace in tools/ to ensure it’s the part of linting and type checking tools? Also, that helps with the dependencies if this script ever needs an external one.

CC: @aduth @Mamaduka @ciampo

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manzoorwanijk I've opened #86222 to implement this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

First-time Contributor Pull request opened by a first-time contributor to Gutenberg repository No Core Sync Required Indicates that any changes do not need to be synced to WordPress Core [Package] wp-build /packages/wp-build [Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PHPStan analysis

6 participants