Skip to content

feat(aggregate): type, name and decode rows as a read returns them - #1024

Merged
abnegate merged 1 commit into
mainfrom
fix-aggregate-rows
Oct 15, 2026
Merged

abnegate merged 1 commit into
mainfrom
fix-aggregate-rows

Conversation

@abnegate

Copy link
Copy Markdown
Member

Problem

aggregate() returned each row exactly as the driver fetched it:

  • Numbers came back as text on MariaDB, MySQL and PostgreSQL. A sum of integers on MariaDB was a DECIMAL string such as "7.0000".
  • Booleans came back as 0/1, datetimes in the engine's format, and arrays and objects as JSON.
  • Attributes with filters came back still encoded.
  • Internal groups came back under their storage column (_uid, _createdAt, _tenant), and joined groups under their bare name.

Every consumer had to rebuild the row against the schema. appwrite/appwrite#14537 does that in Appwrite\Utopia\Database\Aggregation (416 lines). This moves that behaviour into the library so the class can be deleted.

Change

A new Utopia\Database\Aggregation shapes each row from the queries and the collections read.

Aggregates

Aggregate Type
count, countDistinct, bitAnd, bitOr, bitXor int
avg, stddev*, variance, varPop, varSamp float
sum int over an integer attribute, float otherwise (a whole DECIMAL like "7.0000" becomes 7)
min, max the attribute's own type (int, float, bool, formatted datetime, decoded JSON)

An aggregate over no rows stays null.

Groups

  • Each group is named as a read names its attribute: $id, $createdAt, $permissions, and so on.
  • A joined group is qualified by its join alias (o.label, o.$id), including when the queries name it bare and only the join declares it.
  • Each group is cast as find() casts it and passed through its attribute's filters (Database::decode), from the joined collection's attribute for a joined group.
  • The tenant group is returned as the adapter reads it, as find() returns it.

Other

  • The default alias (<method>_<attribute>) is built without a regex.
  • fetch() hands back the joined collections it already resolved, so nothing is resolved twice. JoinResolutionTest holds that.

MongoDB, Memory and Redis still refuse aggregates, as before.

Release

This changes what aggregate() returns: value types, and group keys for internal and joined attributes. aggregate() is new in 8.x, and Appwrite applies exactly this shape itself today. The commit is a feat and the change is additive for the API surface, so 8.3.0 (minor) rather than a patch. Anyone reading _uid or a bare joined key from aggregate() rows should switch to $id / alias.key.

Tests

  • tests/unit/AggregationTest.php ports Appwrite's AggregationTest and runs every case on SQLite, with stringified fetches (as MariaDB, MySQL and PostgreSQL return them) and native fetches. It covers aggregate typing, statistics as floats, a whole DECIMAL sum as an int, null over no rows, group typing (bool, array, datetime, int, float), internal groups under public names, joined groups qualified (alone, beside a main group of the same name, internal, named bare), joined aggregates typed by the joined attribute, unaliased default names, filter decoding for main and joined groups, and rows carrying no ungrouped internal attribute. 18 of the 22 cases fail against main's aggregate().
  • AggregationTests::testAggregateRowsAreTypedNamedAndDecodedAsAReadReturnsThem checks typing and naming on every adapter. It fails on main on MySQL.
  • Existing unit and E2E tests that asserted _uid / _tenant keys or bare joined group names now assert the read names ($id, note.$id, it.name, purchase.status, ...).

Verified locally:

  • Full E2E matrix in Docker passes on MySQL, MariaDB, Postgres, SharedTables/{MySQL, MariaDB, Postgres, MongoDB, Redis}, Mirror, Pool, Memory, Redis, MongoDB and Schemaless/MongoDB. The one MongoDB failure, testListCollectionsReturnsOnlyReadableDefinitions, came from running three adapters in parallel against the same Mongo instance and passes when run on its own.
  • SQLite E2E runs natively. The only failures are cache tests that need Redis, and they fail the same way on main there.
  • The unit suite passes in Docker. Pint and both PHPStan configurations are clean.

🤖 Generated with Claude Code

aggregate() handed back each row as the driver fetched it: numbers as
text under MariaDB, MySQL and PostgreSQL, a sum of integers as a DECIMAL
string such as "7.0000", booleans as 0/1, datetimes in the engine's
format, arrays as JSON, filtered attributes still encoded, internal groups
under their storage column (`_uid`) and joined groups bare, so a consumer
had to rebuild each row against the schema. Appwrite does that in
Appwrite\Utopia\Database\Aggregation; this moves it into the library.

Database\Aggregation shapes each row from the queries and the read
collections: counts and bitwise aggregates are integers, averages,
deviations and variances floats, a sum an integer over an integer
attribute and a float otherwise, and min()/max() take their attribute's
type. Each group is named as a read names its attribute (`$id`, and a
joined one qualified by its join alias, also when the queries name it
bare), cast as a read casts it and passed through its attribute's
filters. The tenant group is returned as read, as find() returns it.
fetch() hands back the joined collections it resolved, so no join is
resolved twice, and the default alias is built without a regex.

Mongo, Memory and Redis still refuse aggregates. Tests that asserted the
old storage-column and bare joined names now assert the read names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 50 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: utopia-php/database/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 388a0681-b204-49c2-a314-e0a317c8faf4
📥 Commits

Reviewing files that changed from the base of the PR and between 0555ec9 and d1b9f2f.

📒 Files selected for processing: 14
  • src/Database/Aggregation.php
  • src/Database/Trait/Documents.php
  • tests/e2e/Adapter/Scopes/AggregationTests.php
  • tests/e2e/Adapter/Scopes/DocumentTests.php
  • tests/e2e/Adapter/Scopes/JoinTests.php
  • tests/unit/AggregationTest.php
  • tests/unit/Documents/AggregateOrderTest.php
  • tests/unit/Documents/AggregateSelectTest.php
  • tests/unit/Joins/AggregateMinorsTest.php
  • tests/unit/Joins/AggregateResultNamesTest.php
  • tests/unit/Joins/JoinInternalColumnsTest.php
  • tests/unit/Joins/JoinedAttributeResolutionTest.php
  • tests/unit/Joins/JoinedShapesTest.php
  • tests/unit/PermissionScope/FullOuterJoinAggregateVisibilityTest.php
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@abnegate
abnegate merged commit 91d5e8b into main Oct 15, 2026
24 checks passed
@abnegate
abnegate deleted the fix-aggregate-rows branch October 15, 2026 11:28
abnegate added a commit to appwrite/appwrite that referenced this pull request Oct 15, 2026
…rns them

utopia-php/database#1024 types counts and bitwise aggregates as int,
statistics as float, an integer sum as int and min/max as the attribute,
and names a joined group by its alias. The assertions that cast numeric
strings or fell back to the bare joined column now require the typed value
and the qualified name, so they fail on a library that returns the raw row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abnegate added a commit to appwrite/appwrite that referenced this pull request Oct 15, 2026
utopia-php/database#1024 (8.3.0) types, names and decodes aggregate rows
the way find() returns documents, and owns the tests for it, so
Aggregation and its unit tests go. The list wraps each row in an Aggregate,
which the Document and Row response models use to leave out the stored
attributes an aggregate row does not group. The AggregateShape e2e tests
now cover the library's shaping end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant