Skip to content

Improve Data hot-path performance with automatic execution recipes - #563

Merged
binaryfire merged 6 commits into
0.4from
feature/data-lean-tier
Sep 4, 2026
Merged

Improve Data hot-path performance with automatic execution recipes#563
binaryfire merged 6 commits into
0.4from
feature/data-lean-tier

Conversation

@binaryfire

@binaryfire binaryfire commented Sep 4, 2026

Copy link
Copy Markdown
Member

Summary

This change makes common hypervel/data construction and transformation paths substantially faster without adding a second data-object API or changing the public contract.

Data classes now receive immutable execution recipes when their declarations can be handled safely by fixed operations. The runtime selects those recipes automatically. Classes using custom casts, normalizers, lifecycle hooks, validation, lazy values, partials, or unsupported union shapes continue through the existing general engine.

Motivation

The Data package supports validation, property mapping, nested objects and collections, lazy values, partial transformations, HTTP resources, request casting, and Eloquent persistence. Its general engine preserves all of those behaviors, but simple objects were paying much of that orchestration cost even when their declarations needed only fixed scalar, enum, date, or nested Data conversion.

The goal here is to make work proportional to the features a declaration uses. Applications should not need to choose a fast mode or a separate lightweight object type.

Design

Class metadata now records one of three transformation states: bulk copy, a fixed property recipe, or the general property loop. Construction similarly records a fixed recipe only for declarations whose behavior is known at metadata-build time.

The implementation:

  • preflights the complete input node before performing conversions, so a later fallback cannot run a nested factory, hook, or constructor twice;
  • shares built-in, enum, and date conversion through one ValueCaster, keeping lean and general behavior identical;
  • lets nested values choose the lean or general path independently while sharing the root operation memo;
  • caches one immutable default creation context per used Data class while still returning a fresh public factory;
  • preserves named factories, late-static factory() overrides, validation decisions, partials, transformation overrides, resource finalization, and persistence behavior;
  • keeps recipe metadata bounded to used classes and declared properties, with no payloads, runtime objects, closures, discovery, generated files, or eviction policy;
  • builds the PHPDoc parser lazily and skips annotation work when native types prove iterable metadata cannot apply;
  • resolves the fixed creator and transformer graph after application providers boot and before production workers fork, while skipping that warm-up during unit tests.

No public API changes are introduced. from(), factory(), collect(), transform(), all(), and toArray() retain their signatures and extension boundaries.

Performance

Measurements used PHP 8.4.23 on Linux x86-64 with CLI OPcache and JIT disabled. Results are the median across three alternating fresh-process runs against the current 0.4 branch. Each standard run used 2,000 operations, seven measured samples, and 100 warm-up operations.

Scenario 0.4 p50 This PR p50 Change 0.4 p95 This PR p95 Change
Flat construction 5.957 us 3.867 us -35.1% 7.306 us 4.050 us -44.6%
Nested construction 32.183 us 6.267 us -80.5% 32.985 us 6.484 us -80.3%
Deep/wide construction 82.337 us 11.302 us -86.3% 87.461 us 11.645 us -86.7%
Eager 1,000-item collection 4.074 ms 2.980 ms -26.9% 5.145 ms 3.171 ms -38.4%
Lazy 1,000-item traversal 3.934 ms 2.387 ms -39.3% 4.026 ms 2.469 ms -38.7%
Simple transformation 3.014 us 2.001 us -33.6% 3.225 us 2.075 us -35.7%
Nested transformation 5.831 us 4.660 us -20.1% 6.622 us 4.840 us -26.9%
Plain five-property transformation 1.829 us 1.809 us -1.0% 2.021 us 1.893 us -6.4%
Validate 5,000 nested items 406.324 ms 406.698 ms +0.1% 424.479 ms 426.319 ms +0.4%

AutoLazy, customized collection factories, lazy partials, Eloquent relation loading, and other fallback-heavy scenarios remained within run-to-run noise.

The tradeoffs are deliberately bounded:

  • metadata analysis adds about 16 us once per used class;
  • across 500 representative five-property classes, retained recipe metadata adds about 235 KB and cached default contexts add about 212 KB;
  • production application startup performs about 15 ms of fixed service initialization before worker fork, moving that cost out of the first request;
  • alternating Testbench runs showed no unit-test startup regression.

The committed benchmark harness reports p50, p95, throughput, query counts, memory, environment details, and checksums. A separate historical comparison fixture remains isolated under tests/Benchmarks/Data and is loaded only by its developer benchmark command.

Verification

  • Full composer fix, including formatting, static analysis, parallel tests, Testbench, and dogfood checks.
  • Focused construction, transformation, metadata, annotation, enum, FormRequest, resource, collection, lazy, and persistence coverage.
  • Lean/general equivalence tests for supported conversions, mappings, defaults, constructors, exceptions, named factories, hooks, and mixed nested graphs.
  • Fallback tests for customized and ambiguous declarations.
  • Repeated before/after performance, startup, and retained-memory measurements.

Summary by CodeRabbit

  • New Features

    • Added faster automatic construction and transformation for eligible data objects.
    • Improved handling of nested data, mapped fields, enums, dates, defaults, nulls, and optional values.
    • Added bulk-copy transformation for compatible data.
    • Added lazy metadata parsing and improved first-use performance.
  • Bug Fixes

    • Improved consistency between optimized and general construction and transformation paths.
    • Preserved fallback behavior for unsupported or complex data shapes.
  • Documentation

    • Added implementation details and expanded benchmark guidance.

Extract the built-in, backed-enum, and date conversion rules into one internal ValueCaster used by both ordinary casts and lean construction recipes.

This keeps coercion behavior, date formats and timezones, concrete date targets, and existing exception contracts in one authoritative implementation without retaining cast instances in worker metadata.
Compile immutable per-class construction and transformation recipes from existing Data metadata, then select them automatically for supported runtime shapes while preserving the general engine as the single fallback.

Reuse immutable default creation contexts, preflight complete nodes before conversion, share nested operation state, retain named-factory and hook boundaries, and keep bulk-copy transformation for plain objects. Unsupported declarations and ambiguous conversion families continue through the established path before construction.

Defer PHPDoc parser setup when native types prove iterable metadata cannot apply. Add focused equivalence, fallback, metadata, lazy, partial, mapping, constructor, and transformation coverage so lean execution stays behaviorally identical to the general path.
Resolve DataCreator and DataTransformer from an application booted callback after all providers have configured their dependencies. Production workers inherit the initialized immutable service graph instead of making the first request pay that fixed setup cost.

Keep unit-test applications on demand so repeated Testbench boots remain fast, and cover both testing and non-testing application lifecycles with stable instance assertions.
Expand the developer benchmark matrix across construction, validation, transformation, collections, resources, persistence, metadata, and first-use boundaries.

Add a dedicated historical DataObject comparison harness under the test namespace so supported shapes can be measured against the removed mapper without restoring it as framework API. Document both commands and keep raw reports opt-in and outside the repository.
Record the measured baseline, immutable recipe design, one-engine fallback rules, lifecycle constraints, integration boundaries, test matrix, performance acceptance criteria, and rejected alternatives.

The plan also preserves the required post-checkpoint merge and benchmark work so the current 0.4 enum and morph behavior is reconciled before final integration.
Bring the latest 0.4 framework changes into the automatic Data execution branch, including the first-party FormRequest casting redesign, shared enum coercion helpers, and the queue, database, validation, and lifecycle fixes already accepted on 0.4.

Reconcile the Data fast path with the shared enum_from semantics so lean and general construction accept integer-backed numeric strings consistently. Preserve enum_try_from morph selection and add a focused regression that proves the compiled operation, target enum, and concrete result.

Finish the execution terminology sweep, retain the measured performance and memory conclusions in the active plan, and record the completed verification state. The merged tree passes composer fix and focused Data, enum, FormRequest, and capability coverage; repeated benchmarks retain the material construction and transformation gains without a systematic fallback-path regression.
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 114127dd-cee7-4cdb-b01c-37302b343bed

📥 Commits

Reviewing files that changed from the base of the PR and between 70b69d2 and c156c8c.

📒 Files selected for processing (27)
  • docs/plans/2026-09-04-0853-data-automatic-lean-execution.md
  • src/data/src/Casts/BuiltinTypeCast.php
  • src/data/src/Casts/DateTimeInterfaceCast.php
  • src/data/src/Casts/EnumCast.php
  • src/data/src/DataServiceProvider.php
  • src/data/src/Enums/DataPropertyOperation.php
  • src/data/src/Support/Annotations/DataIterableAnnotationReader.php
  • src/data/src/Support/Creation/CreationContextFactory.php
  • src/data/src/Support/Creation/DataCreationRecipe.php
  • src/data/src/Support/Creation/DataCreator.php
  • src/data/src/Support/Creation/DataInstantiator.php
  • src/data/src/Support/Creation/ValueCaster.php
  • src/data/src/Support/DataClass.php
  • src/data/src/Support/DataProperty.php
  • src/data/src/Support/Factories/DataClassFactory.php
  • src/data/src/Support/Factories/DataPropertyFactory.php
  • src/data/src/Support/Transformation/DataTransformationRecipe.php
  • src/data/src/Support/Transformation/DataTransformer.php
  • tests/Benchmarks/Data/Fixtures/DataObject.php
  • tests/Benchmarks/Data/README.md
  • tests/Benchmarks/Data/benchmark.php
  • tests/Benchmarks/Data/compare-data-object.php
  • tests/Data/DataServiceProviderTest.php
  • tests/Data/Support/Creation/DataCreatorTest.php
  • tests/Data/Support/DataClassTest.php
  • tests/Data/Support/DataIterableAnnotationReaderTest.php
  • tests/Data/Support/Transformation/DataTransformerTest.php

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

Automatic lean execution is implemented for Hypervel Data. Immutable metadata recipes classify construction and transformation paths. Creation contexts and parser dependencies are cached selectively. Casting, transformation, provider warming, tests, and benchmark harnesses are updated.

Changes

Automatic lean execution

Layer / File(s) Summary
Recipe metadata and operation classification
src/data/src/Enums/*, src/data/src/Support/DataClass.php, src/data/src/Support/DataProperty.php, src/data/src/Support/Factories/*, src/data/src/Support/*Recipe.php, tests/Data/Support/DataClassTest.php
Data metadata now stores property operations and immutable creation and transformation recipes. Factories resolve eligibility and bulk-copy support.
Lean creation and shared casting
src/data/src/Support/Creation/*, src/data/src/Casts/*, tests/Data/Support/Creation/DataCreatorTest.php
DataCreator uses cached contexts, normalized factory payloads, recipe-based construction, fallback paths, and shared built-in, enum, and date casting.
Transformation, parser, and provider wiring
src/data/src/Support/Transformation/*, src/data/src/Support/Annotations/*, src/data/src/DataServiceProvider.php, tests/Data/Support/*, tests/Data/DataServiceProviderTest.php
DataTransformer adds bulk-copy and recipe paths. Iterable parsing becomes lazy. Production boot warms DataCreator and DataTransformer.
Benchmark fixtures and comparison harness
tests/Benchmarks/Data/*
Benchmarks compare construction, transformation, memory, metadata, cold startup, service resolution, resource responses, and Eloquent casts. The README documents the new scenarios.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: ⚪ Minimal · up to c156c

The optimized creation and transformation paths retain documented fallbacks and equivalence coverage, with no actionable merge-blocking issue identified.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant DataCreator
  participant CreationContextFactory
  participant ValueCaster
  participant DataInstantiator
  Caller->>DataCreator: create payload
  DataCreator->>CreationContextFactory: get cached create context
  DataCreator->>ValueCaster: convert typed property
  DataCreator->>DataInstantiator: instantiate lean node
  DataInstantiator-->>DataCreator: data instance
  DataCreator-->>Caller: return created data
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 228 functions across 25 files. (2 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: improving Data hot-path performance through automatic execution recipes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 228 functions across 25 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/data-lean-tier

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@binaryfire

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@binaryfire
binaryfire merged commit 44a4c4b into 0.4 Sep 4, 2026
38 checks passed
@binaryfire
binaryfire deleted the feature/data-lean-tier branch September 5, 2026 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant