feat: add Query.Builder, aggregates, joins, and nested ON

[abnegate]

Why

Appwrite query-lib added aggregates, grouping, joins, extra spatial operators, and a POST /query body so large queries arrays are not capped by URL length. SDKs that already expose Query helpers need those methods, a fluent collector, and a way to nest JOIN ON conditions without leaking SQL.

What changed

• New Query helpers on every SDK that already had Query: aggregates (count, sum, avg, …), groupBy/having/distinct, joins (join/leftJoin/rightJoin/fullOuterJoin/crossJoin), extra spatial (covers, …).
• Fluent collector nested as Query.Builder where the language allows it (C#/Swift/Kotlin/Ruby/Python/TS namespace). PHP is Appwrite\Query\Builder imported as QueryBuilder. Go/Rust use package Builder with NewBuilder() / query::Builder. Dart cannot nest classes and must not export a top-level Builder (Flutter widget clash), so the type is QueryBuilder in lib/query/builder.dart.
• Nested JOIN ON: Query.on(...) plus join overloads that take an alias, an ON query, and extra filters. The wire shape is [alias?, Query.on, filters…].
• Client flatten is internal. There is no public Query.normalize. Flattening is structural (builder / nested query-string lists) and does not special-case the queries parameter name.
• Closed live-spec string enums encoded as type: string plus oneOf of single-value schemas are treated as wire enums. Untitled oneOf members take their name from the parent property.
• PHP from() and generated mocks wrap a scalar example for an array-of-enum field into a one-element list (OAuth prompt in the live spec is a string example on an array field).
• E2E QUERY_HELPER_RESPONSES covers each new helper in every SDK language test.

Raw/union/json/naturalJoin stay unexposed — those are not on the Documents validator allow-list.

Vector helpers were already on main; duplicate copies introduced during the rebase were removed so each language has a single definition.

Why this approach

A client-side JSON collector (not SQL) matches the existing Query string format {method, attribute?, values?}. Nesting the builder under Query keeps the public surface one type. Flattening inside the Client, without a queries key check, means any param that already carries query strings (join ON lists, having, or/and) is handled the same way.

POST /query itself lives in Appwrite (appwrite/appwrite#13300, stacked on #11649). SDK methods will be generated as createQuery once that spec is published.

Verified

• composer lint (phpcs PSR-12)
• composer lint-twig (djLint, 0 errors)
• composer refactor:check (Rector dry-run)
• Twig line length ≤ 1200
• composer test tests/e2e/PHP83Test.php — OK, 1511 assertions (includes array-enum scalar hydration)
• Generated Dart lib/ + tests/tests.dart analyze clean for the Builder/part-of/MockKind failures
• Generated Go query.NewBuilder().Limit(1).Build() compiles and prints the expected limit query

Not verified

• Full sdk-generator language e2e matrix on this HEAD. CI is the authority for that (previous HEAD was red on Dart/Flutter/Go Tests and Dart/Flutter/PHP/CLI Validation).
• Live Appwrite POST /query round-trip from a generated SDK (depends on appwrite#13300 + spec publish).
• CLI compiles in this repo against ../go via a replace in go.mod.twig (generated from the same spec as the CLI). Compiling the published CLI package against the last published Go module is not verified here.

[greptile-apps]

Confidence Score: 5/5

The latest revision addresses the Unity generation blocker, with no new blocking findings.

Summary

Adds fluent query builders, aggregate and grouping helpers, joins with nested ON conditions, and additional spatial operators across SDKs.

• Adds request-side flattening and cross-language query transport coverage.
• Updates enum naming and array-enum model handling.
• The latest revision registers the shared C# Builder template in Unity’s independent output plan, addressing the previous missing-declaration finding.
• No new actionable findings were identified in this revision. The full language E2E matrix was not run during this review.

_{Reviews (41) · Last reviewed commit: "fix(unity): emit Query/Builder.cs in the..."}

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[Copilot]

Pull request overview

Adds cross-language query aggregation, grouping, joins, spatial operators, fluent builders, request flattening, and enum-handling improvements to generated SDKs.

Changes:

• Expands Query APIs and adds fluent builders across supported SDK languages.
• Normalizes nested query inputs in clients and adds broad E2E/unit coverage.
• Improves closed oneOf enum detection and scalar array-enum hydration.

Reviewed changes

Copilot reviewed 110 out of 110 changed files in this pull request and generated 29 comments.

Show a summary per file

File(s)	Description
tests/resources/spec-openapi3.json	Adds closed oneOf enum fixtures.
tests/e2e/Base.php	Adds helper-output and enum hydration assertions.
tests/e2e/languages/{web,node,react-native,deno}/*	Exercises TypeScript query helpers and builders.
tests/e2e/languages/{swift,apple}/Tests.swift	Exercises Swift query helpers.
tests/e2e/languages/{kotlin,android}/Tests.kt	Exercises Kotlin query helpers.
tests/e2e/languages/{go,go-v2}/tests.go	Exercises Go query helpers.
tests/e2e/languages/{dart,flutter}/tests.dart	Exercises Dart query helpers.
tests/e2e/languages/{dotnet,unity}/Tests.cs	Exercises C# query helpers.
tests/e2e/languages/{ruby,python,php}/*	Exercises dynamic-language query helpers.
templates/web/src/{query,client,index}.ts.twig	Adds Query builder/input normalization and exports.
templates/web/src/services/realtime.ts.twig	Normalizes realtime query inputs.
templates/node/src/{client,index}.ts.twig	Adds Node request flattening and exports.
templates/node/test/query.test.js.twig	Tests Node query behavior.
templates/react-native/src/{query,client,index}.ts.twig	Adds React Native query building and flattening.
templates/deno/{mod.ts,src/query.ts,src/client.ts}.twig	Adds Deno query building and flattening.
templates/deno/test/query.test.ts.twig	Tests Deno query behavior.
templates/swift/Sources/Query*.twig	Adds Swift helpers and nested builder.
templates/swift/Sources/Client.swift.twig	Flattens Swift request parameters.
templates/swift/{Tests/Tests.swift,Package.swift}.twig	Adds tests and conditional package targets.
templates/apple/{Sources/Client.swift,Package.swift}.twig	Applies inherited Swift client/package updates.
templates/kotlin/src/main/kotlin/io/appwrite/{Query,Client}.kt.twig	Adds Kotlin query APIs and flattening.
templates/kotlin/src/test/kotlin/io/appwrite/QueryTest.kt.twig	Adds Kotlin Query tests.
templates/android/library/src/main/java/io/package/{Query,Client}.kt.twig	Adds Android query APIs and flattening.
templates/go/query*.twig	Adds Go helpers, builder, and tests.
templates/go/client.go.twig	Flattens Go request values.
templates/rust/src/query/{mod,builder}.rs.twig	Adds Rust Query APIs and builder.
templates/rust/src/client.rs.twig	Normalizes Rust request parameters.
templates/rust/tests/tests.rs	Exercises Rust query helpers.
templates/dotnet/Package/Query*.twig	Adds .NET helpers and nested builder.
templates/dotnet/Package/Client.cs.twig	Flattens .NET request parameters.
templates/php/src/{Query,Client}.php.twig	Adds PHP helpers, builder integration, and flattening.
templates/php/src/Query/Builder.php.twig	Implements the PHP fluent builder.
templates/php/src/Models/{Model,RequestModel}.php.twig	Wraps scalar array-enum values during hydration.
templates/php/tests/QueryTest.php.twig	Tests PHP Query behavior.
templates/python/package/{query,client}.py.twig	Adds Python builder APIs and flattening.
templates/python/test/test_query.py.twig	Tests Python Query behavior.
templates/ruby/lib/container/{query,client}.rb.twig	Adds Ruby helpers and request flattening.
templates/ruby/lib/container/query/builder.rb.twig	Implements the Ruby fluent builder.
templates/dart/lib/{query,package}.dart.twig	Adds Dart Query APIs and builder registration.
templates/dart/lib/query/builder.dart.twig	Implements QueryBuilder.
templates/dart/lib/src/client_mixin.dart.twig	Flattens Dart request parameters.
templates/dart/lib/src/models/{model,request_model}.dart.twig	Handles scalar array-enums and enum serialization.
templates/dart/test/{query_test,src/models/model_test}.dart.twig	Extends Dart Query/model tests.
templates/flutter/lib/{package,src/client_mixin}.dart.twig	Registers builders and flattens Flutter parameters.
templates/skills/*.md.twig	Documents query aggregation, joins, and builders.
templates/cli/internal/typegen/templates/{types,databases}.ts.hbs	Extends typed CLI Query helpers.
templates/cli/internal/generator/typescript.go	Reuses the generated QueryBuilder type.
templates/cli/internal/generator/testdata/*	Updates expected TypeScript output.
templates/cli/go.mod.twig	Redirects CLI builds to a local generated Go SDK.
src/SDK/SDK.php	Improves enum-name fallback resolution.
src/SDK/Language.php	Resolves untitled enum names from parent properties.
src/SDK/Language/{Swift,Apple}.php	Registers Swift builder output.
src/SDK/Language/{Kotlin,Android}.php	Registers Query tests.
src/SDK/Language/{Dart,Flutter}.php	Registers Dart builder output.
src/SDK/Language/{Go,Rust,PHP,Ruby,DotNet}.php	Registers new builder/module templates.
example.php	Generates Go alongside CLI and updates the Go SDK version.

Suppressed comments (2)

templates/cli/internal/typegen/templates/types.ts.hbs:76

• These fixed scalar signatures cannot express the new nested JOIN ON form, and the facade exposes no on helper. Add on plus overloads/unions for (table, on) and (table, alias, on) so the typed callback can construct the nested wire shape supported by Query.

  join: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  leftJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  rightJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  fullOuterJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;

templates/cli/internal/typegen/templates/databases.ts.hbs:44

• The facade delegates only the scalar join form and does not expose Query.on, so typed CLI callbacks cannot use the nested JOIN ON feature added by this PR. Mirror the nested overloads and on helper exposed by Query.

  join: (table, left, right, operator = '=', alias = '') => Query.join(table, left, right, operator, alias),
  leftJoin: (table, left, right, operator = '=', alias = '') => Query.leftJoin(table, left, right, operator, alias),
  rightJoin: (table, left, right, operator = '=', alias = '') => Query.rightJoin(table, left, right, operator, alias),
  fullOuterJoin: (table, left, right, operator = '=', alias = '') => Query.fullOuterJoin(table, left, right, operator, alias),

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[abnegate]

Went through Copilot's 29 threads.

Fixed

• Rust flatten no longer treats numeric nested arrays as query input
• Go flatten uses reflect.DeepEqual (no panic on [][]float64)
• Dart/Flutter flatten spreads [Query.builder()...] into a flat string[]
• Regenerating Rust deletes leftover src/query.rs
• Typed CLI QueryBuilder includes stddev/variance/bitwise aggregates
• Go skill examples match GroupBy([]interface{}) and 5-arg Join
• Skills drop createQuery until the spec has it, and only show build() into the generated string[] APIs

Left as-is

• CLI replace => ../go is for this generator (examples/cli + examples/go). Test-mode CLI omits it. The published CLI repo is a different tree.
• Generated service methods keep wire type string[]. Flattening is internal. Pass builder.build(). QueryInput stays on realtime, where a union already existed. Widening every query param is a separate API change and would still have to avoid a queries-key special case.

[abnegate]

Pushed two commits on top of the flatten-bugfix head:

• Language e2e tests now assert flatten: wrapped builders expand, nested numeric arrays stay nested, already-flat query lists stay lists. That is the coverage Copilot's flatten bugs needed.
• CLI generator testdata includes stddev/variance/bitwise so TestGenerateMatchesBaseline matches the typed QueryBuilder templates.

CLIGo126 / Validation cli (console) were red on the previous head because those baselines still expected groupBy at the old offset.

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

Addressed the Tests Mirror Query Internals finding (tests/e2e/Base.php:209-240).

What the assertions pinned before

• 32 lines of the form console.log(Query.sum('price', 'total')), each compared against a hard-coded JSON string. Nothing left the process, so the client, its parameter encoding and the wire were all outside the assertion.
• Three flatten-*:ok labels the harness computed for itself, matched by name in QUERY_FLATTEN_RESPONSES.
• Query.page(2, 10)[0] / [1] and Query.builder().limit(1).build()[0], read by index.
• The .NET and Unity harnesses required Query.Flatten to hand back the same collection instance (object.ReferenceEquals).

What they assert now

Every language harness builds those helpers into the queries parameter of a real general.listRows call and prints the list the mock server echoes back. Base::QUERY_TRANSPORT_RESPONSE marks that line, and Base::assertQueriesReachedTheServer decodes the echo and compares it element by element against QUERY_TRANSPORT_QUERIES. What is pinned is the request the server received, decoded, so key order and whitespace are free to change. Query.page and Query.builder feed the request list instead of being indexed, and the identity check is gone.

Flatten keeps its coverage, on the path a caller can reach

QUERY_FLATTEN_RESPONSES and the labels are gone. Every request now carries its queries array through the client's flatten before transport in all 20 SDKs, so a flatten that mangles a query list turns the suite red everywhere instead of being self-reported. The languages whose service signature accepts a builder (JS/TS, PHP, Python, Ruby) pass an unbuilt Query.builder().limit(1) and let the server prove it was spread. The generated SDKs' own unit tests (node, deno, php, python, dart, go, swift, rust) still assert the helper directly.

Seen red, then green

vendor/bin/phpunit tests/e2e/Node20Test.php:

• Renaming the method Query.sum emits in templates/web/src/query.ts.twig:
  Query 6 did not reach the server as the helper described it, -'method' => 'sum' / +'method' => 'summ'.
• Collapsing the array index for queries in Client.flatten (templates/node/src/client.ts.twig), a pure transport regression the previous in-process shape could not have seen: The SDK did not print the query list the mock server echoes back, it printed: AppwriteException: Invalid queries param ....
• Reordering the keys Query.toString emits, a representation change the server does not care about: still OK (1 test, 193 assertions).
• Both breaks reverted: OK (1 test, 193 assertions).

One real defect fell out of this

templates/deno/src/query.ts.twig called an undefined JSONbig in nestedJoin, so Query.leftJoin(table, alias, [on...]) threw ReferenceError: JSONbig is not defined at runtime. The previous shape could not see it because it never made a request, and the Deno suites are absent from the CI matrix. Fixed here; adding those suites to the matrix is filed separately.

Run locally: Node18, Node20, Node26, WebNode, WebChromium, Deno1303, PHP85, Python313, Ruby27, Ruby31, Go113, Go118, DotNet80, DotNet90, DartStable, FlutterStable, KotlinJava17, Swift61, Rust183, plus --testsuite Generation, vendor/bin/phpcs and vendor/bin/rector process --dry-run.

[abnegate]

@greptileai review

Addressed the Tests Mirror Serialization Internals finding.

The three generated-SDK test files that compared a helper against a literal JSON string now decode the query and assert its fields, so key order, whitespace and number formatting are free to change while method, attribute and values stay pinned:

• templates/node/test/query.test.js.twig — expectQuery(query) parses before asserting; the parameterised fixtures hold real arrays instead of JSON text.
• templates/deno/test/query.test.ts.twig — same, via a parsed() helper.
• templates/python/test/test_query.py.twig — same, via parsed(); expectations are dicts.

The finding also named PHP, Go, Dart, Kotlin, Swift and Rust. Those six already decode the query and assert its fields (json_decode in QueryTest.php.twig, json.Unmarshal in query_test.go.twig, jsonDecode in query_test.dart.twig, JsonParser in QueryTest.kt.twig, JSONSerialization in Tests.swift.twig, to_value() in query/mod.rs.twig), so there was nothing to change there. The Operator tests still compare strings, but this PR does not touch them.

Converting Deno surfaced five expectations that had drifted from the SDK: isNull and isNotNull carried an empty values array, select put its attributes under attribute, and both cursor helpers put their id there too. The Deno unit tests run in no job, so nothing had said so. Corrected to the shape the SDK emits, which is the shape Base::QUERY_HELPER_RESPONSES already pins.

Verified

• node: npx jest test/query.test.js → Tests: 86 passed, 86 total; npx prettier --check test/query.test.js → All matched files use Prettier code style!
• deno: deno test --allow-net --allow-read test/query.test.ts → ok | 1 passed (112 steps) | 0 failed
• python: python -m unittest test.test_query → Ran 54 tests ... OK; python -m black --check test/test_query.py → 1 file would be left unchanged
• Seen red: renaming the method Query.sum emits gives - "method": "sum" / + "method": "summ" on the node sum test, and green once reverted.
• Re-ran Node20, Deno1303, Python313 E2E (193 / 172 / 175 assertions), --testsuite Generation (85 tests, 1130 assertions), vendor/bin/phpcs, vendor/bin/rector process --dry-run, djlint templates/ --lint (0 errors) and the twig line-length script.

[greptile-apps]

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

•  Request-only Swift models lose target src/SDK/SDK.php:971 ▶

  For an operation whose body references an unmarked component model and whose response has no model, this classification now moves the component from definitions to requestModels. Swift and Apple still emit its source into Sources/<Title>Models, but both package templates declare the Models target and dependency only when definitions is nonempty. A request-only SDK therefore fails to build because its services import a Models module that was never declared.

  Update both package templates to include the Models target whenever either model collection is nonempty.

[hansi-codes]

🟢 Tier S · Ready to merge

The incremental Go compatibility change is valid, and no actionable defect was found in the reviewed changes.

This pull request adds language-native Query helpers and fluent builders across the supported SDKs, including aggregate, grouping, spatial, join, and nested JOIN ON APIs. It updates client-side parameter flattening and serialization to preserve ordinary nested payloads while expanding Query values, and extends enum handling for live-spec shapes. Generated SDK tests, CLI type-generation fixtures, and language usage guidance are expanded to exercise the new surface.

Latest changes: The newest commit switches the Go client test body reader to ioutil.ReadAll so the generated test remains compatible with the Go 1.13 test target.

Verdict	New comments	Fixed	Still open
✅ Approved	0	0	0

📂 Walkthrough · 8

File	Change
example.php; src/SDK/SDK.php; src/SDK/Language.php; src/SDK/Language/*.php	Wire in the expanded Query support and recognize closed single-value oneOf string enums, including scalar examples for array enum fields.
templates/*/query*; templates/*/Query*; templates/*/query/**	Add each language's Query helpers and fluent builder, including aggregate, grouping, spatial, join, and nested ON forms.
templates/*/client*; templates/*/Client*; templates/*/src/client*	Flatten Query and builder values within client request serialization while preserving unrelated nested payload values.
templates/* model templates and related model tests	Adjust model hydration and generated mocks for array-of-enum values represented by scalar examples.
templates/cli/internal/typegen/**; templates/cli/internal/generator/testdata/**; templates/cli/go.mod.twig	Update CLI type generation fixtures and callback declarations for query joins and nested ON conditions, and configure the local Go module replacement.
templates/skills/*.md.twig	Document the new Query helper and builder APIs in the language-specific guidance.
tests/e2e/**; tests/generation/GenerationTest.php; tests/resources/spec-openapi3.json	Extend SDK E2E and generation coverage for the new Query behavior and the enum-shape cases.
templates/go/client_test.go.twig	Use ioutil.ReadAll for reading the test HTTP request body, retaining Go 1.13 compatibility.

---

_{Reviewed the commits since be1e57e · Details · Comment @hansi-codes review to re-run, or mention @hansi-codes with a question.}

[hansi-codes]

🟡 Tier B · 2 blocking findings to address. Summary

The findings that requested changes are resolved. See the summary comment for what is still open.

[abnegate]

Merged main in (733 commits). Conflict resolutions:

• templates/cli/go.mod.twig: keeps this PR's sdkImportPath parameter with main's v7.10.0.
• templates/dart/test/src/models/model_test.dart.twig: keeps both this PR's object → {} branch and main's number_example macro.
• templates/deno/test/query.test.ts.twig: keeps this PR's parsed(...) comparisons.
• templates/web/src/index.ts.twig: keeps main's push and analytics exports and adds QueryInput.
• Go e2e tests: keep main's testGenericUpload and this PR's testQueries(client).

Follow-up fixes:

• 8930180, CLI excludes. configureSDK in example.php deep-merged only exclude.services, so the CLI's exclude.methods override (added on main) replaced the whole exclude block. That brought back documentsDB and vectorsDB for the CLI, and the CLI then failed to build against the locally pinned Go SDK. Every exclude kind is now merged.
• f2714c1 → be1e57e, review findings. These address all four Hansi findings; details are on each thread.
• Android16Java17. This job runs the unit tests against an R8-shrunk library, a pass added on main in b7d2060. The PR's nested join … on and having parsed nested queries back into Query through Gson reflection, but R8 renames those fields, so the nested queries lost their method. They now parse into maps, the way main already handles or/and/elemMatch. I reproduced testR8UnitTest locally: it failed before the fix and all 34 tests pass after.

Behaviour change to review: a list is no longer spliced unless an item is a builder, so [Query.page(2, 10)] must be spread (...Query.page(2, 10)). That matches how every e2e script and the typed signatures already use it.

@greptileai review

[hansi-codes]

🟢 Tier S · Looks good to merge. Summary