Filter and sort computed fields by a DQL expression

CLAUDE.md

@@ -93,7 +93,7 @@ Attributes are in `src/Attribute/`:
 - `#[Entity]` - Marks an entity for GraphQL exposure
 - `#[Field]` - Exposes a field
 - `#[Association]` - Exposes an association (relationship)
-- `#[ComputedField]` - Exposes derived values from entity methods (placed on public methods); its `type` may be an entity class exposed in the group, and `list: true` makes it a list. An entity type's field is a thunk, as an association's is, so entities may refer to each other. Each method parameter is an argument (int/float/string/bool inferred, others typed by the attribute's `args`); values of fields with arguments are cached per set of arguments in `FieldResolver`, and `extract()` leaves those fields out
+- `#[ComputedField]` - Exposes derived values from entity methods (placed on public methods); its `type` may be an entity class exposed in the group, and `list: true` makes it a list. An entity type's field is a thunk, as an association's is, so entities may refer to each other. Each method parameter is an argument (int/float/string/bool inferred, others typed by the attribute's `args`); values of fields with arguments are cached per set of arguments in `FieldResolver`, and `extract()` leaves those fields out. An `expression` (DQL of the value) makes it filterable and sortable: every alias in it is a `{placeholder}` (`{entity}` is the root, others are renamed per use by `Filter\ComputedFieldExpression`), and `{:name}` is an argument, taken from the filter's `args`. `MetadataFactory` parses each expression with `Query::getAST()` (no database) after the `metadata.build` event, and adds to its `excludeFilters` the filters DQL rejects for it (a subquery takes no IN, IS NULL or LIKE); `between` is applied as two comparisons. A sort selects the expression `AS HIDDEN` and orders by it; the batch id-pair query re-adds hidden selects after replacing the select
 - `#[ExcludeFilters]` - Excludes specific filters
 
 ### Event System (PSR-14)

docs/attributes.rst

@@ -276,6 +276,12 @@ is built.
 * ``name`` - An override for the field name in GraphQL. If not provided,
   the name is derived from the method name (``getFullName`` becomes ``fullName``).
 * ``group`` - You can have multiple GraphQL configurations organized by ``group``.
+* ``expression`` - The DQL expression of the field's value, by which it is
+  filtered and sorted.  Every alias in it is a placeholder, such as
+  ``{entity}``; see `computed fields <computed-fields.html#filters-and-sorting>`_.
+* ``excludeFilters`` - Filters to exclude for a field with an ``expression``.
+* ``includeFilters`` - The only filters to allow for a field with an
+  ``expression``.  Mutually exclusive with ``excludeFilters``.
 
 Example:
 
@@ -300,9 +306,8 @@ Example:
       }
   }
 
-**Important**: Computed fields cannot be filtered at the database level
-and will not appear in filter InputObjects.  They are calculated in PHP
-after data is retrieved from the database.
+**Important**: A computed field is calculated in PHP after the rows are
+fetched, so it is filtered and sorted only when it has an ``expression``.
 
 For more information, see `computed-fields <computed-fields.html>`_.
 

docs/computed-fields.rst

@@ -72,6 +72,11 @@ The ``#[ComputedField]`` attribute accepts these parameters:
 * ``name`` - Optional. Override the field name in the GraphQL schema.  If not provided,
   the name is derived from the method name.
 * ``group`` - Optional. The attribute group (default: ``'default'``).
+* ``expression`` - Optional. The DQL expression of the field's value, by which it is
+  filtered and sorted; see `Filters and Sorting`_.
+* ``excludeFilters`` - Optional. Filters to exclude for a field with an ``expression``.
+* ``includeFilters`` - Optional. The only filters to allow for a field with an
+  ``expression``.  It is mutually exclusive with ``excludeFilters``.
 
 Custom Field Names
 ------------------
@@ -146,8 +151,9 @@ Computed fields are:
   with the regular fields, except those of fields with arguments, which have no one value
 * **Cached** - If you enable ``useHydratorCache``, a computed value is cached with the
   entity's other values for as long as the entity exists
-* **Not filterable** - Computed fields cannot be used in database filters since they're
-  calculated in PHP, not at the database level
+* **Filtered by an expression** - A computed field's value is calculated in PHP, so it is
+  filtered and sorted only when it has an ``expression``, its value in DQL; see
+  `Filters and Sorting`_
 
 Arguments
 ---------
@@ -259,12 +265,142 @@ pagination, filters or ``totalCount``, and it holds what the method returns.
 For a large collection, expose the association instead, whose rows the
 database filters and pages.
 
-Filtering Limitations
----------------------
+Filters and Sorting
+-------------------
+
+A computed field's value is calculated in PHP, after the rows are fetched, but
+a filter must be applied by the database: the rows of a page, and its
+``totalCount``, are counted and limited there.  So a computed field is filtered
+and sorted only when it has an ``expression``: the DQL expression of its value.
+The field's filters, ``sort`` and ``sortPriority`` among them, then apply to the
+expression as a field's apply to its column.
+
+.. code-block:: php
+
+  use ApiSkeletons\Doctrine\ORM\GraphQL\Attribute as GraphQL;
+
+  #[GraphQL\Entity]
+  class Artist
+  {
+      #[GraphQL\ComputedField(
+          type: 'int',
+          expression: '(SELECT COUNT({p}.id) FROM App\ORM\Entity\Performance {p} WHERE {p}.artist = {entity})',
+      )]
+      public function getPerformanceCount(): int
+      {
+          return $this->performances->count();
+      }
+  }
+
+.. code-block:: graphql
+
+  {
+    artists (filter: { performanceCount: { gt: 0, sort: DESC } }) {
+      edges { node { name performanceCount } }
+    }
+  }
+
+The method still computes the value a query returns; the expression is used
+only to filter and sort.  They must compute the same value, or a filter will
+disagree with the value shown, and a sorted list will appear out of order.
+
+Placeholders
+~~~~~~~~~~~~
+
+.. important::
+
+  **Every alias in an expression must be a placeholder**, written in braces:
+  ``{entity}`` for the field's own entity, and any other name, such as
+  ``{p}``, for an alias of the expression's own.
+
+DQL aliases must be unique across a query, subqueries included, and an
+expression may be used more than once in one query: by two filters, such as
+``{ gt: 0, lt: 5 }``, or by a filter and a sort.  Each placeholder is given a
+name unique in the query each time the expression is used, so its aliases
+never collide with each other's, or with those of a QueryBuilder listener.  An
+alias written plainly, such as ``p``, works until the expression is used twice;
+the check described below finds it.
+
+A placeholder is any ``{name}`` in the expression, including one within a
+string literal.
+
+Arguments
+~~~~~~~~~
+
+``{:name}`` in an expression is the field's `argument <#arguments>`_
+``name``, bound as a parameter.  A filter is applied before any field is
+resolved, so its arguments cannot be those the field is given where it is
+selected.  The field's filters take them instead, in ``args``, of the
+arguments the expression uses:
+
+.. code-block:: php
+
+  #[GraphQL\ComputedField(
+      type: 'int',
+      expression: '(SELECT COUNT({r}.id) FROM App\ORM\Entity\Recording {r} '
+          . 'WHERE {r}.artist = {entity} AND {r}.year >= {:year})',
+  )]
+  public function getRecordingCountSince(int $year): int
+
+.. code-block:: graphql
+
+  {
+    artists (filter: { recordingCountSince: { args: { year: 2003 }, gt: 0 } }) {
+      edges { node { name recordingCountSince(year: 2003) } }
+    }
+  }
 
-Computed fields do not appear in filter InputObjects because they cannot be filtered
-at the database level.  If you need to filter on computed values, consider storing
-them in the database or using the `QueryBuilder Event <events.html>`_ to add custom filters.
+An argument not given in ``args`` is its default value, or null when it has
+none.  ``args`` is required when an argument the expression uses is.  The
+filter's ``args`` and the field's own arguments are independent: a filter may
+use 2003 while the field shows 2004.
+
+Which Filters Apply
+~~~~~~~~~~~~~~~~~~~
+
+The filters of a computed field are those of its ``type``, less those
+excluded by the configuration, by its entity, by the association whose
+filters they are, and by its own ``excludeFilters`` or ``includeFilters``.
+
+DQL compares any expression, but takes a subquery, or an arithmetic
+expression, for none of ``IN``, ``IS NULL`` and ``LIKE``.  So the ``in``,
+``notin``, ``isnull``, ``contains``, ``startswith`` and ``endswith`` filters
+are each excluded for an expression DQL does not take them for, and added to
+the field's ``excludeFilters`` in the metadata.  ``between`` is applied as two
+comparisons, so it applies to every expression.  A function, such as
+``UPPER({entity}.name)`` or ``SIZE({entity}.performances)``, takes every
+filter.
+
+A sort orders by the expression selected as a hidden result variable, as DQL
+does not order by a subquery.  A `QueryBuilder listener <events.html>`_ must
+not replace the query's select, as ``->select()`` does, which would remove it.
+
+Checks
+~~~~~~
+
+Each expression is checked when the metadata is built, after the
+``metadata.build`` event, and throws
+``ApiSkeletons\Doctrine\ORM\GraphQL\Exception\Metadata`` when:
+
+* it is not valid DQL, such as for a syntax error, an unknown field or class,
+  or an alias written without a placeholder.  Doctrine's ``QueryException`` is
+  the previous exception.  The expression is parsed, not run, so no database
+  is needed.
+* it uses ``{:name}`` for a name which is not a parameter of the method.
+* its field is a ``list``, or of an entity type, neither of which has a single
+  value to filter or sort by.
+* a field without an ``expression`` has ``excludeFilters`` or
+  ``includeFilters``.
+
+Metadata read from a cache is not checked again.
+
+Costs
+~~~~~
+
+* A subquery is evaluated for each row the filters consider, and a sort by one
+  orders every row before the page is taken, without an index.  For a large
+  table, store the value in a column instead.
+* Databases order nulls differently, as they do for a field.
 
 Hydrator Strategies
 -------------------

docs/events.rst

@@ -105,6 +105,11 @@ all resolve parameters:
     adds is fetched with Doctrine's ``Paginator``, which limits the entities
     rather than the rows.
 
+    A listener may add to the select, as ``->addSelect('p')`` does, but must
+    not replace it, as ``->select()`` does.  A sort by a `computed field
+    <computed-fields.html#filters-and-sorting>`_ orders by a hidden result
+    variable in the select, which replacing it removes.
+
 Association QueryBuilder Event
 ==============================
 

docs/versions.rst

@@ -59,6 +59,8 @@ ORM versions do not support ``doctrine/persistence`` 4.
   hydrator strategy
 - A computed field may be of an entity type, or a list of any type, and has an argument for
   each parameter of its method
+- 14.1: a computed field with an ``expression``, its value in DQL, is filtered and sorted
+  by it; see `computed fields <computed-fields.html#filters-and-sorting>`_
 - Metadata is read as typed value objects, and cached metadata carries its format version
   and the config it was built with
 - Configuration mistakes, such as a misplaced attribute, an invalid name or a field

src/Attribute/ComputedField.php

@@ -4,6 +4,7 @@
 
 namespace ApiSkeletons\Doctrine\ORM\GraphQL\Attribute;
 
+use ApiSkeletons\Doctrine\ORM\GraphQL\Filter\Filters;
 use Attribute;
 
 /**
@@ -12,8 +13,11 @@
  * Computed fields are generated from entity methods rather than database columns.
  * They integrate with the hydrator pipeline and are cached along with regular fields.
  *
- * Important: Computed fields cannot be filtered at the database level and will not
- * appear in filter InputObjects.
+ * A computed field is filtered and sorted only when it has an expression: the
+ * DQL expression of its value, which the filters compare and sort by.  Every
+ * alias in it is a {placeholder}: {entity} is the entity, and any other is
+ * given a unique name each time the expression is used.  {:name} is the
+ * method parameter $name, whose value is given by the filter's args.
  *
  * @example
  * ```php
@@ -30,15 +34,23 @@
 #[Attribute(Attribute::TARGET_METHOD | Attribute::IS_REPEATABLE)]
 final class ComputedField
 {
+    use ExcludeFilters;
+
     /**
-     * @param string                $type        A type registered in the TypeContainer, or the class of an entity
-     *                                           exposed in the group, whose type the field is
-     * @param string                $group       GraphQL schema group (default: 'default')
-     * @param string|null           $name        Field name in GraphQL schema. If null, derived from method name.
-     * @param string|null           $description Field description for GraphQL schema
-     * @param bool                  $list        Whether the method returns a list of the type
-     * @param array<string, string> $args        The registered type of each method parameter which is
-     *                                           not an int, float, string or bool, by its name
+     * @param string                $type           A type registered in the TypeContainer, or the class of an entity
+     *                                              exposed in the group, whose type the field is
+     * @param string                $group          GraphQL schema group (default: 'default')
+     * @param string|null           $name           Field name in GraphQL schema. If null, derived from method name.
+     * @param string|null           $description    Field description for GraphQL schema
+     * @param bool                  $list           Whether the method returns a list of the type
+     * @param array<string, string> $args           The registered type of each method parameter which is
+     *                                              not an int, float, string or bool, by its name
+     * @param string|null           $expression     The DQL expression of the field's value, by which it
+     *                                              is filtered and sorted
+     * @param array<Filters|string> $excludeFilters Filters cases or their values, for a field with an
+     *                                              expression
+     * @param array<Filters|string> $includeFilters Filters cases or their values, for a field with an
+     *                                              expression
      */
     public function __construct(
         private readonly string $type,
@@ -47,6 +59,9 @@ public function __construct(
         private readonly string|null $description = null,
         private readonly bool $list = false,
         private readonly array $args = [],
+        private readonly string|null $expression = null,
+        private readonly array $excludeFilters = [],
+        private readonly array $includeFilters = [],
     ) {
     }
 
@@ -80,4 +95,9 @@ public function getArgs(): array
     {
         return $this->args;
     }
+
+    public function getExpression(): string|null
+    {
+        return $this->expression;
+    }
 }

src/Filter/ComputedFieldExpression.php

@@ -0,0 +1,76 @@
+<?php
+
+declare(strict_types=1);
+
+namespace ApiSkeletons\Doctrine\ORM\GraphQL\Filter;
+
+use Closure;
+
+use function array_values;
+use function preg_match_all;
+use function preg_replace_callback;
+
+use const PREG_SET_ORDER;
+
+/**
+ * The placeholders of a computed field's expression
+ *
+ * DQL aliases must be unique across a query, subqueries included, and an
+ * expression may be used more than once in a query: by two filters, or by a
+ * filter and a sort.  So every alias in an expression is a {placeholder},
+ * given a unique name each time the expression is used.  {entity} is the root
+ * entity of the query.  {:name} is the computed field's argument $name, bound
+ * as a parameter.
+ */
+final class ComputedFieldExpression
+{
+    private const string PLACEHOLDER = '/\{(:?)([A-Za-z_][A-Za-z0-9_]*)\}/';
+
+    /** The placeholder of the root entity */
+    public const string ENTITY = 'entity';
+
+    /**
+     * The names of the arguments the expression uses, each once
+     *
+     * @return list<string>
+     */
+    public static function argumentNames(string $expression): array
+    {
+        preg_match_all(self::PLACEHOLDER, $expression, $matches, PREG_SET_ORDER);
+
+        $names = [];
+        foreach ($matches as $match) {
+            if ($match[1] !== ':') {
+                continue;
+            }
+
+            $names[$match[2]] = $match[2];
+        }
+
+        return array_values($names);
+    }
+
+    /**
+     * The expression for one use: {entity} is the root alias, each other
+     * alias is prefixed, and each argument is the parameter the function
+     * returns for its name
+     *
+     * @param Closure(string): string $parameter The parameter, with its colon, of an argument
+     */
+    public static function substitute(
+        string $expression,
+        string $rootAlias,
+        string $aliasPrefix,
+        Closure $parameter,
+    ): string {
+        return (string) preg_replace_callback(
+            self::PLACEHOLDER,
+            static fn (array $match): string => match (true) {
+                $match[1] === ':' => $parameter($match[2]),
+                $match[2] === self::ENTITY => $rootAlias,
+                default => $aliasPrefix . $match[2],
+            },
+            $expression,
+        );
+    }
+}