diff --git a/CHANGELOG.md b/CHANGELOG.md index a26b62ae..c1c7f433 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ section is the GitHub release's notes. ### Added - Generators: a plain value object that declares `IAdditionOperators`, `ISubtractionOperators`, `IMultiplyOperators`, `IDivisionOperators`, `IModulusOperators`, `IUnaryNegationOperators` or `IUnaryPlusOperators` gets their operators, `Money + Money` or `Money * 3m`, each the underlying type's own, overflow and `checked` context included; nothing without the interface, nothing on a validated value object, and an operator you wrote is kept ([Add, subtract and scale](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/docs/value-objects.md#add-subtract-and-scale)). +- EF Core: those operators run in a query as the underlying type's arithmetic on SQLite, PostgreSQL and SQL Server, `o.Price * 2m > limit` and `o.Tax / o.Price` included ([Queries](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries)). ### Changed diff --git a/design/adr/0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md b/design/adr/0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md new file mode 100644 index 00000000..9f5d1ff3 --- /dev/null +++ b/design/adr/0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md @@ -0,0 +1,59 @@ +# 0124. An operator in a query is the wrapped type's arithmetic + +Date: 2026-10-30 +Status: accepted + +Context: [#186](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/issues/186), under the 2.4.0 epic +[#177](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/issues/177); [EF Core](../ef-core.md); refines +[decision 122](0122-arithmetic-operators-are-opt-in-and-a-hand-written-one-is-a-seam.md), whose EF Core bullet it +replaces, on [decision 123](0123-a-hand-written-operator-is-refused-in-a-query.md)'s mark. + +EF translates an operator node as the database's arithmetic over its operands, and never calls the operator's +method. Between two value objects into one, that is what a generated operator does where TypeKit's converter +stores the column: `o.Price + o.Tax` is `ef_add("p"."Price", "p"."Tax")` on SQLite. Where the application's own +converter stores it, as `"M:" + value`, it added the texts: `Price + Tax > 4` became +`"p"."Price" + "p"."Tax" > 'M:4'` and kept no row where memory keeps two, and `-Price < -4` kept all three. With +another operand or result EF applied the value object's converter to the other side: `o.Price * 2m > limit` threw +"Invalid cast from 'System.Decimal' to 'ProbeMoney'", `o.Tax / o.Price` as a `decimal` "No coercion operator is +defined", and unary `+` was not translated (measured with EF Core 10.0.12 on SQLite). + +**Every generated operator, marked `[TranslatedAsWrappedArithmetic]`, is rewritten to the wrapped type's +arithmetic over what the value objects wrap, before EF translates the query.** + +- **The rewrite.** The interceptor and the filter convention of decision 123 rewrite a marked operator to the + wrapped type's operator over `ValueObjectConverter<,>.ProviderValue` of each value-object operand, the other + operand as it is. `ProviderValue` translates only where TypeKit's converter stores the column + ([decision 56](0056-value-translated-only-through-typekit-converter.md)), so an operator over a column the + application converts itself is EF's "could not be translated"; as a query's last `Select`, EF computes it in + memory. A result of the value object's type is `ValueObjectConverter<,>.Materialize` of the wrapped type's, which + the method translator re-types as the value object, with its mapping for the same store type, so it compares with + and is read as one. Both methods are reached through `GeneratedValueObjectAttribute.Accept`, never by name, and + are public, so a compiled model writes a rewritten filter out. Unary `+` of a number is its operand. +- **A `Nullable` operand is checked, as the lifted operator is.** The rewrite of an operator with an optional + operand is null unless every optional operand is there, and an operator over such a rewrite takes its check over, + so `o.Optional * 2m + o.Tax` checks `Optional` once: `CASE WHEN "p"."Optional" IS NOT NULL THEN + ef_add(ef_multiply("p"."Optional", '2.0'), "p"."Tax") END` on SQLite. Unwrapped by conversions alone, which EF drops + from the SQL, the null reached the arithmetic where EF computes the last `Select` in memory: over a column the + application's own converter stores, `Select(o => o.Amount * 2m)` threw "Nullable object must have a value" on a + null row (an outside review of #190). +- **Re-typed, not cast.** The database's arithmetic, and `.Value` of it, is re-typed as a column is: cast to the + column's store type, SQL Server's default `decimal(18,2)`, `(o.Price / 32m).Value` over 5 read 0.16 where memory + reads 0.15625, and a filter on it lost the row (measured on Azure SQL Edge). SQLite's decimal arithmetic is + functions, `ef_add`, `ef_multiply`, `ef_divide`, `ef_mod` and `ef_negate`, re-typed by those names. So is what + holds such a value without computing one, branch by branch: a `CASE` (`?:`, and the null check above), a + `COALESCE` (`??`), `MAX` and `MIN`, and a subquery of one value. Cast whole, `(o.Optional / 32m ?? o.Price / + 32m).Value` read 0.09 where memory reads 0.09375, a filter on it lost its row, and arithmetic over a `?:` and a + navigation's largest quotient rounded the same (an outside review of #190, measured on Azure SQL Edge). A branch of + any other shape, such as a parameter, is converted alone, and any other function's result is converted. Each + branch's own converter is checked, since EF gives a `CASE` the first branch's mapping. +- **A wrapped type without an expression of its own.** `byte`, `short`, `sbyte`, `ushort` and `char` have no + arithmetic in an expression tree, where C# computes it as an `int`'s. Their operators are refused naming the value + object, between two value objects too, which no release translated (#184 is 2.4's). + +`ArithmeticQueryTests` holds it as a differential oracle: 45 queries (filters, projections, orderings and an +aggregate; a column, a nullable column, a JSON property, a navigation, a captured factor, a decimal column and an +optional one as the other operand; `checked`, unary `+` and `-`; past the column's scale, through `?:`, `??`, +`ValueOrNull()` and a navigation's `Max`) against the same queries over the objects +in memory, on SQLite, PostgreSQL 17 and SQL Server. `ArithmeticQueryFilterTests` holds the filters, named and not, +and `ForeignConvertedArithmeticTests` a converter of the application's own, optional operands computed in memory +included. diff --git a/design/adr/README.md b/design/adr/README.md index 84d1d716..e9f53595 100644 --- a/design/adr/README.md +++ b/design/adr/README.md @@ -128,5 +128,6 @@ per area ([design](../plan.md)) link to the records they rest on. | [0119](0119-a-gap-found-before-the-tag-ships-in-that-release.md) | A form an Error misses, found before its version is tagged, is reported in that version when it loses what the rule protects in every build, though code built on the previous minor can contain it: 2.3.0's CMTK0003 on a `Task.WhenAny` of results (#159), `Wait()` on a branch (#160), an array in `Task.WaitAll`/`WaitAny` (#161), a dropped `Task.WhenEach` (#162), and a `WhenAny` of held tasks silent; the rule stops at the tag, after which decision 34 holds; refines 34 and 118 | 2026-10-29 | accepted | | [0120](0120-a-net11-0-target-and-documented-floor-raises-join-a-minor.md) | A `net11.0` target joins a 2.x minor when a feature needs it, and a dependency floor may rise in a minor, on every target or on `net11.0` alone, if it is documented (the Semantic Versioning FAQ calls a dependency update with an unchanged public API compatible; a project that references the dependency lower gets NU1605, measured on SDK 11 RC1, so the changelog names the floor, who it reaches and both ways out, and the smoke test shows it); a union new on `net11.0` is an addition and may join a 2.x minor (a `LangVersion` 14 consumer who matches on it gets CS9327, and on Metalama 2026.1 CS8652, measured); Metalama is one version for every target and TypeKit follows the maintained Metalama year in a minor: 2.3 and 2.4 stay on 2026.1 and 2.5.0 moves both targets to 2027.0 once stable (a direct 2026.1 reference gets NU1605 and stays on 2.4.x; a woven `net11.0` project goes from C# 14 to 15; conditions: a license-free CI and clean-container build, the ceiling and the 2.4.x line documented); the SDK 11 spike comes first (#170); 3.0.0 stays the `net10.0` drop and a change of an existing contract, timed to .NET 12 LTS, and .NET 11 is no reason for a major; refines decision 44 | 2026-10-29 | accepted | | [0121](0121-a-generated-member-is-refused-wherever-the-value-object-reaches-it.md) | CMTK1011, and CMTK1008 for comparison, refuse a generated member written by hand wherever a value object or closed set reaches it: on it, in a base record, public or explicit, or as an interface's explicit implementation, in any assembly, found by the interface mapping since Metalama lists no private member of another assembly's type; a base record's `Create` beside the value object's own (#195) and a seam a base record declares (#194) too; `IConvertible`, `IValueObject<,>`, `IValueObjectMaterializer<,>`, `IPlainValueObject<,>` and a set's `IParsable` declared through an interface or by a base record are CMTK1011 where the aspect crashed (#192); a closed set's base class counts as a base record does, its `sealed` `ToString()` the seam and one not sealed refused (#205); an interface's `==` and .NET's own `INumberBase` bridges are not refused; an exception to 34 taken in the patch 2.3.1 (#191, #192, #194, #195, #197, #205) | 2026-10-30 | accepted | -| [0122](0122-arithmetic-operators-are-opt-in-and-a-hand-written-one-is-a-seam.md) | A plain value object gets the arithmetic operators (`+`, `-`, `*`, `/`, `%`, unary `-` and `+`, and their `checked` forms) of the operator interfaces it declares, where the wrapped type implements the same interface with the value object replaced by itself; each calls the wrapped type's through `GeneratedArithmetic`, a constrained generic call with its context stated; never by default and never on a validated value object, which keeps CS0535; a hand-written operator for the same operands, or an explicit implementation of the interface, its own or a base record's, is a seam, kept with its `checked` form, so 2.3 code builds unchanged; refines decision 28 | 2026-10-30 | accepted; refined by [0123](0123-a-hand-written-operator-is-refused-in-a-query.md) | +| [0122](0122-arithmetic-operators-are-opt-in-and-a-hand-written-one-is-a-seam.md) | A plain value object gets the arithmetic operators (`+`, `-`, `*`, `/`, `%`, unary `-` and `+`, and their `checked` forms) of the operator interfaces it declares, where the wrapped type implements the same interface with the value object replaced by itself; each calls the wrapped type's through `GeneratedArithmetic`, a constrained generic call with its context stated; never by default and never on a validated value object, which keeps CS0535; a hand-written operator for the same operands, or an explicit implementation of the interface, its own or a base record's, is a seam, kept with its `checked` form, so 2.3 code builds unchanged; refines decision 28 | 2026-10-30 | accepted; refined by [0123](0123-a-hand-written-operator-is-refused-in-a-query.md) and [0124](0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md) | | [0123](0123-a-hand-written-operator-is-refused-in-a-query.md) | An arithmetic, bitwise or shift operator a value object declares by hand is refused in an EF Core query, naming the value object and `.Value`, by an interceptor and, for query filters, when the model is built; the generated ones carry `[TranslatedAsWrappedArithmetic]`, which is what tells them apart; a conversion operator is left to EF; refines decision 122 | 2026-10-30 | accepted | +| [0124](0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md) | In an EF Core query, every generated arithmetic operator is rewritten to the wrapped type's operator over `ProviderValue`, which translates only through TypeKit's converter, its value-object result `Materialize`d and re-typed, never cast; `byte`, `short` and `char` operators are refused; refines decision 122, on decision 123's mark | 2026-10-30 | accepted | diff --git a/design/ef-core.md b/design/ef-core.md index 7d45a38e..0814f21c 100644 --- a/design/ef-core.md +++ b/design/ef-core.md @@ -59,12 +59,21 @@ model is built (and so at `dbcontext optimize` and `migrations add`). A comparison in a filter that reads neither the entity nor a query is left alone: EF evaluates it in memory, by `CompareTo` (measured: `Floor > From(2)` on the context kept the rows memory keeps). - - **A hand-written operator is refused in a query (decided 2026-10-30, [decision 123](adr/0123-a-hand-written-operator-is-refused-in-a-query.md)).** + - **An operator is the wrapped type's arithmetic, or refused (decided 2026-10-30, [decision 123](adr/0123-a-hand-written-operator-is-refused-in-a-query.md) and [decision 124](adr/0124-an-operator-in-a-query-is-the-wrapped-type-s-arithmetic.md)).** EF translates an operator node as the database's arithmetic over its operands and never calls the - method, so a hand-written `+` that rounds ran as the plain sum of the columns. The arithmetic aspect - marks every operator it generates `[TranslatedAsWrappedArithmetic]`; an interceptor, and a finalizing - convention for query filters, refuses any other arithmetic, bitwise or shift operator a value object - declares, `&&` and `||` over its `&` and `|` included, naming the value object (#187). A conversion operator is left to EF. + method. For a generated operator between two value objects that is right where TypeKit's converter + stores the column, and added the texts where the application's own stores `"M:" + value`; with another + operand or result it applied the value object's converter to the other side and threw (#186). An + interceptor, and a finalizing convention for query filters, rewrites every generated operator marked + `[TranslatedAsWrappedArithmetic]` to the wrapped type's operator over `ProviderValue` of each + value-object operand, its result wrapped by `ValueObjectConverter<,>.Materialize`, which the method + translator re-types as the value object with its mapping for the same store type; unary `+` of a number + is the operand. Any other arithmetic, bitwise or shift operator a value object declares, `&&` and `||` over its + `&` and `|` included, is hand-written, and refused naming the value object (#187), in a filter when the model is built. The database's + arithmetic, and `.Value` of it, is re-typed, not cast, through a `?:`, a `??`, `MAX`, `MIN` and a subquery of + one value too: a cast to the column's store type, SQL Server's default `decimal(18,2)`, read + `(o.Price / 32m).Value` over 5 as 0.16 where memory reads 0.15625. An optional operand is checked, so EF's + computing the last `Select` in memory reads null where the operator does. - **A null passes no comparison in a query (decided 2026-10-15, [decision 98](adr/0098-a-null-passes-no-comparison-in-a-query.md)).** A record class's generated operators sort null first, so `null < x` is true in memory, and SQL's `"N" < @x` leaves the null row out, as for the underlying type's nullable column. Not refused: a diff --git a/docs/value-objects.md b/docs/value-objects.md index f6e3e2b4..d53b6498 100644 --- a/docs/value-objects.md +++ b/docs/value-objects.md @@ -232,11 +232,10 @@ object or another type, as long as the underlying type has that operator itself. base record or an interface of yours, and TypeKit keeps it, with its checked form. It generates only the operators you haven't written. - **The checked context.** `checked(a + b)` uses the checked operator, as it would for an `int`. A value object over an `int` throws `OverflowException` there, and wraps around outside it. -- **In an EF Core query.** An operator between two value objects runs in the database, as it would for - the underlying type: `o.Price + o.Tax > limit` keeps the rows memory keeps. One with another operand or - result, such as `o.Price * 2m`, throws there for now. Write `o.Price.Value * 2m` instead. An operator - you wrote yourself can't run there, so a query that uses it throws, naming the value object - ([EF Core, Queries](../src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries)). +- **In an EF Core query.** Each operator runs in the database as the underlying type's arithmetic: + `o.Price + o.Tax > limit`, `o.Price * 2m > limit` and `o.Tax / o.Price < 0.3m` keep the rows memory + keeps. An operator you wrote yourself can't run there, so a query that uses it throws, naming the value + object ([EF Core, Queries](../src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries)). ## Enums, dates and other types diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ConverterMethods.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ConverterMethods.cs new file mode 100644 index 00000000..5cd1bb1e --- /dev/null +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ConverterMethods.cs @@ -0,0 +1,41 @@ +using System.Collections.Concurrent; +using System.Linq.Expressions; +using System.Reflection; +using CodoMetis.TypeKit.CompilerServices; +using CodoMetis.TypeKit.ValueObjects; + +namespace CodoMetis.TypeKit.EntityFrameworkCore; + +/// +/// and +/// of one value object, which a query is rewritten to +/// call where it unwraps one or wraps the wrapped type's result in one. +/// +/// +/// Reached through with the value object as a type +/// argument, from the converter's expressions, whose methods the compiler names: nothing is looked up by name and +/// nothing is made with MakeGenericType, which the trimmer and Native AOT may break (IL2075, IL3050). Both +/// are public because a compiled model writes a rewritten query filter out as C# in the application's assembly. +/// +internal static class ConverterMethods +{ + private static readonly ConcurrentDictionary Methods = new(); + + public static MethodInfo ProviderValue(GeneratedValueObjectAttribute valueObject) => Of(valueObject).ProviderValue; + + public static MethodInfo Materialize(GeneratedValueObjectAttribute valueObject) => Of(valueObject).Materialize; + + private static (MethodInfo ProviderValue, MethodInfo Materialize) Of(GeneratedValueObjectAttribute valueObject) => + Methods.GetOrAdd(valueObject.ValueObjectType, static (_, description) => description.Accept(MethodsOf.Instance), valueObject); + + private sealed class MethodsOf : IValueObjectVisitor<(MethodInfo, MethodInfo)> + { + public static readonly MethodsOf Instance = new(); + + public (MethodInfo, MethodInfo) Visit() + where TValueObject : IValueObject, IValueObjectMaterializer + where T : notnull => + (((MethodCallExpression)ValueObjectConverter.ToProvider.Body).Method, + ((MethodCallExpression)ValueObjectConverter.FromProvider.Body).Method); + } +} diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/README.md b/src/CodoMetis.TypeKit.EntityFrameworkCore/README.md index c356598e..24478f38 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/README.md +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/README.md @@ -221,15 +221,22 @@ instead, which works everywhere and may add a cast: (`OrderBy(o => o.Priority.Value)`), or order in memory after the query. Equality isn't ordering, and stays allowed. A query filter (`HasQueryFilter`, named or not) that does the same fails when the model is built, naming the entity type, since it applies to every query of that type. +- **A value object's arithmetic runs as the underlying type's.** The operators TypeKit generates + ([Add, subtract and scale](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/docs/value-objects.md#add-subtract-and-scale)) work in a query. + `o.Price * 2m > limit` keeps the rows memory keeps, and `o.Tax / o.Price` reads a `decimal`. The + database computes them, on SQLite, PostgreSQL and SQL Server alike. - **Your own operator can't run in the database.** EF reads `o.Price + o.Tax` as the database's own addition of the two columns and never calls your `+`. Whatever your operator does beyond that, such as rounding, would be skipped. So a query that uses an arithmetic or bitwise operator you wrote on a value object, `&&` and `||` included, throws an `InvalidOperationException` naming the value object. Compute it from `.Value` where the underlying type's arithmetic is what you mean (`o.Price.Value + o.Tax.Value`), or do it in memory after the query. A query filter that uses one fails when the model is built. An operator over - values you pass in, which EF computes before the query, stays allowed. The operators TypeKit - generates ([Add, subtract and scale](https://github.com/CaffeinatedCoder/CodoMetis.TypeKit/blob/main/docs/value-objects.md#add-subtract-and-scale)) - are the underlying type's, and work in a query. + values you pass in, which EF computes before the query, stays allowed. +- **Over a `byte`, `short` or `char`,** an operator throws as well, naming the value object. C# + computes their arithmetic as an `int`'s, and EF has no expression for theirs. +- **Over a column your own converter stores,** an operator isn't translated, as `.Value` isn't: the + column doesn't hold the underlying value, so EF's "could not be translated" is the answer. As the + query's last step, a `Select` still works: EF computes it in memory. One exception: a filter may order values your context holds, without a lambda, as in `HasQueryFilter(o => o.Priority == Limits.Max())`. EF works that out in memory, by your `CompareTo`. With a diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/TypeKitConventionSetPlugin.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/TypeKitConventionSetPlugin.cs index 8df12b4d..cdc6e1ed 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/TypeKitConventionSetPlugin.cs +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/TypeKitConventionSetPlugin.cs @@ -6,9 +6,9 @@ namespace CodoMetis.TypeKit.EntityFrameworkCore; /// /// Adds TypeKit's conventions to every model UseTypeKit() builds: the key generation of /// and , -/// the rewrite of , -/// the generator of , and the refusals of -/// and . +/// the rewrites of and , +/// the generator of , and the refusal of +/// . /// internal sealed class TypeKitConventionSetPlugin : IConventionSetPlugin { @@ -27,7 +27,7 @@ public ConventionSet ModifyConventions(ConventionSet conventionSet) // A query filter's .Value read through the value-object interface, before the model is final. finalizing.Add(new ValueThroughInterfaceQueryFilterConvention()); - // A query filter's value-object operator written by hand, refused. + // A query filter's value-object arithmetic, rewritten or, written by hand, refused. finalizing.Add(new ValueObjectArithmeticQueryFilterConvention()); // Last, once whether a property is generated is final: the provider's generator for the wrapped type. diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs index 00e619c6..c496c5b0 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs @@ -1,4 +1,5 @@ using System.Linq.Expressions; +using System.Numerics; using System.Reflection; using CodoMetis.TypeKit.CompilerServices; using Microsoft.EntityFrameworkCore; @@ -9,13 +10,14 @@ namespace CodoMetis.TypeKit.EntityFrameworkCore; /// -/// Refuses an operator a value object declares by hand (), before EF translates -/// the query. +/// Rewrites a value object's generated arithmetic operator to the wrapped type's arithmetic over what it wraps, and +/// refuses an operator written by hand +/// (), before EF translates the query. /// /// /// QueryCompilationStarting sees the LINQ tree once per query shape, after EF has evaluated what reads no /// database, so what is left reads a column. It does not see the query filters EF adds later, which -/// checks. +/// rewrites. /// internal sealed class ValueObjectArithmeticInterceptor : IQueryExpressionInterceptor { @@ -23,11 +25,12 @@ public Expression QueryCompilationStarting(Expression queryExpression, QueryExpr ValueObjectArithmetic.Rewrite(queryExpression); } -/// Refuses the same in the model's query filters, named ones included, when the model is built. +/// Rewrites and refuses the same in the model's query filters, named ones included, when the model is built. /// /// EF adds an entity type's query filters to a query after QueryCompilationStarting, so /// never sees them. A filter applies to every query of its entity type, -/// so a hand-written operator in one is refused where the model is built, as an invalid configuration is. +/// so a hand-written operator in one is refused where the model is built, as an invalid configuration is. A +/// compiled model writes the rewritten filter out, which calls public methods. /// internal sealed class ValueObjectArithmeticQueryFilterConvention : IModelFinalizingConvention { @@ -50,20 +53,37 @@ public void ProcessModelFinalizing(IConventionModelBuilder modelBuilder, IConven } } -/// A value object's operator in a query: one written by hand refused. +/// +/// A value object's operator in a query: its generated arithmetic translated as the wrapped type's, and one written +/// by hand refused. +/// /// /// /// EF translates an operator as the database's arithmetic over the operands' columns, and never calls its method. -/// For a generated operator that is what it does: the generators mark the ones they write -/// , and each calls the wrapped type's through -/// . +/// Between two value objects that is what a generated operator does, where the column holds the wrapped value +/// (ef_add("p"."Price", "p"."Tax") on SQLite). Where the application's own converter stores it, as +/// "M:" + value, it added the texts: Price + Tax > 4 kept no row where memory keeps two, and +/// -Price < -4 all three where memory keeps two. With an operand or a result of another type it applied the +/// value object's converter to the other side: o.Price * 2m threw "Invalid cast from 'System.Decimal' to +/// 'Money'", and o.Tax / o.Price, a , "No coercion operator is defined between types +/// 'Money' and 'Nullable<Decimal>'" (measured with EF Core 10.0.12 on SQLite). Unary + was not +/// translated at all. +/// +/// +/// So every generated operator is rewritten to what it does: the wrapped type's operator over +/// of each value-object operand, which translates +/// only where TypeKit's converter stores the column (), the other operand +/// as it is, and a result of the value object's type wrapped by +/// , which the method translator turns into the +/// wrapped type's result re-typed as the value object. Unary + of a number is the number. Only an operator +/// marked is: the generators mark the ones they write, which +/// call the wrapped type's through . /// /// /// An operator the value object declares without the mark was written by hand, and its body is what the database -/// cannot run: with From(Math.Round(left.Value + right.Value, 2)), Price + Tax > 1.006 became -/// ef_compare(ef_add("l"."Price", "l"."Tax"), @limit) > 0 and kept no row where memory keeps one -/// (measured with EF Core 10.0.12 on SQLite). It is refused, an arithmetic, bitwise or shift operator alike, naming -/// the value object and .Value, as a hand-written CompareTo is (). A +/// cannot run: From(Math.Round(left.Value + right.Value, 2)) added the columns and never rounded, so a filter +/// kept the rows of the plain sum. It is refused, an arithmetic, bitwise or shift operator alike, naming the value +/// object and .Value, as a hand-written CompareTo is (). A /// conversion operator is left to EF: implicit operator string(Name n) => n.Value is common and /// translates to what it does. /// @@ -76,32 +96,76 @@ public static Expression Rewrite(Expression query) => /// /// In a query filter, where EF has not yet evaluated what reads nothing of the database: an operator over the - /// context's own values runs in memory, as written, and is not refused. + /// context's own values runs in memory, as written, and is neither rewritten nor refused. /// public static Expression RewriteFilter(LambdaExpression filter, string where) => new Rewriter(where, "", skipsWhatRunsInMemory: true).Visit(filter); private sealed class Rewriter(string where, string orElse, bool skipsWhatRunsInMemory) : ExpressionVisitor { + /// + /// What each null-checked rewrite returned stands for: its value, not Nullable, and the check that it is + /// there. An operator over one takes both over, so its operands are read once and checked once. + /// + private readonly Dictionary _checked = []; + protected override Expression VisitBinary(BinaryExpression node) { - if (Operator(node.NodeType) is not null) Check(node.Method, node); + if (Operator(node.NodeType) is not { } symbol || ValueObjectOf(node.Method, node) is not { } valueObject) + return base.VisitBinary(node); + + var (left, leftTest) = Present(Visit(node.Left)); + var (right, rightTest) = Present(Visit(node.Right)); + + var valueObjectType = valueObject.ValueObjectType; + var leftValue = Unwrapped(left, valueObject); + var rightValue = right.Type == valueObjectType ? Unwrapped(right, valueObject) : right; + + Expression result; + try + { + result = Expression.MakeBinary(node.NodeType, leftValue, rightValue); + } + catch (InvalidOperationException) + { + // byte, short, char: C# computes their arithmetic as int's, and an expression tree has none of theirs. + throw Untranslatable(valueObject, $"{valueObjectType.Name} {symbol} {right.Type.Name}"); + } - return base.VisitBinary(node); + return Wrapped(result, node.Type, valueObject, $"{valueObjectType.Name} {symbol} {right.Type.Name}", Both(leftTest, rightTest)); } protected override Expression VisitUnary(UnaryExpression node) { - if (Operator(node.NodeType) is not null) Check(node.Method, node); + if (Operator(node.NodeType) is not { } symbol || ValueObjectOf(node.Method, node) is not { } valueObject) + return base.VisitUnary(node); - return base.VisitUnary(node); + var visited = Visit(node.Operand); + var valueObjectType = valueObject.ValueObjectType; + + if (node.NodeType == ExpressionType.UnaryPlus) + { + // A number's unary + is the number, which EF does not translate. Of the value object, it is the operand, + // which reads nothing of the column: it compares or is read through its converter, whoever's it is. + if (!IsNumber(valueObject.WrappedType)) throw Untranslatable(valueObject, $"{symbol}{valueObjectType.Name}"); + if (Unlifted(node.Type) == valueObjectType) return visited; + } + + var (operand, test) = Present(visited); + var value = Unwrapped(operand, valueObject); + var result = node.NodeType == ExpressionType.UnaryPlus ? value : Expression.MakeUnary(node.NodeType, value, value.Type); + + return Wrapped(result, node.Type, valueObject, $"{symbol}{valueObjectType.Name}", test); } - /// Refuses where it is an operator of a value object written by hand, and reads the database. - private void Check(MethodInfo? method, Expression node) + /// + /// The value object whose generated operator is, or for any + /// other operator and one that runs in memory; a hand-written operator of a value object is refused here. + /// + private GeneratedValueObjectAttribute? ValueObjectOf(MethodInfo? method, Expression node) { - if (method?.DeclaringType is not { } declaringType || ValueObjectTypes.Describe(declaringType) is not { } valueObject) return; - if (skipsWhatRunsInMemory && !DatabaseReads.In(node)) return; + if (method?.DeclaringType is not { } declaringType || ValueObjectTypes.Describe(declaringType) is not { } valueObject) return null; + if (skipsWhatRunsInMemory && !DatabaseReads.In(node)) return null; if (method.GetCustomAttribute() is null || method.GetParameters() is not [{ ParameterType: var first }, ..] @@ -110,9 +174,70 @@ private void Check(MethodInfo? method, Expression node) $"{declaringType.Name}'s operator {Operator(node.NodeType)} is used {where}, but it is written by hand, and the database cannot run it: " + $"there it is the {valueObject.WrappedType.Name} arithmetic of the column, which need not be what the operator does. " + $"Compute it from .Value where the {valueObject.WrappedType.Name} arithmetic is meant{orElse}."); + + return valueObject; } + + /// + /// The wrapped type's result as the operator's: re-typed as the value object by + /// , or as it is for any other result type; where + /// an operand is Nullable, null unless finds every one there, as the lifted operator is. + /// + /// + /// EF computes a query's last Select in memory where it cannot translate it, such as over a column the + /// application's own converter stores. Unwrapped by a conversion alone, an optional operand threw "Nullable + /// object must have a value" there on a null row, where the operator answers null. Translated, the check is a + /// CASE whose one branch is the arithmetic, which re-types like the arithmetic. + /// + private Expression Wrapped(Expression result, Type type, GeneratedValueObjectAttribute valueObject, string what, Expression? test) + { + var valueObjectType = valueObject.ValueObjectType; + var expected = Unlifted(type) == valueObjectType ? valueObject.WrappedType : Unlifted(type); + + if (result.Type != expected) throw Untranslatable(valueObject, what); + + var wrapped = Unlifted(type) == valueObjectType ? Expression.Call(ConverterMethods.Materialize(valueObject), result) : result; + + if (test is null) return wrapped.Type == type ? wrapped : Expression.Convert(wrapped, type); + + var lifted = Expression.Condition(test, Expression.Convert(wrapped, type), Expression.Constant(null, type)); + _checked[lifted] = (wrapped, test); + + return lifted; + } + + /// + /// as a value, and the check that it is there where it is Nullable: an + /// operator this rewrote already hands over its own. + /// + private (Expression Value, Expression? Test) Present(Expression operand) + { + if (_checked.TryGetValue(operand, out var rewritten)) return rewritten; + + return Nullable.GetUnderlyingType(operand.Type) is { } underlying + ? (Expression.Convert(operand, underlying), Expression.NotEqual(operand, Expression.Constant(null, operand.Type))) + : (operand, null); + } + + private static Expression? Both(Expression? left, Expression? right) => + left is null ? right : right is null ? left : Expression.AndAlso(left, right); + + /// A value-object operand as of it, which translates as its column. + private static Expression Unwrapped(Expression operand, GeneratedValueObjectAttribute valueObject) => + Expression.Call(ConverterMethods.ProviderValue(valueObject), operand); + + private Exception Untranslatable(GeneratedValueObjectAttribute valueObject, string what) => + new InvalidOperationException( + $"{what} is used {where}, but {valueObject.WrappedType.Name} has no such operator EF can translate. " + + $"Compute it from .Value{orElse}."); } + private static Type Unlifted(Type type) => Nullable.GetUnderlyingType(type) ?? type; + + /// The types whose unary + is the value itself. + private static bool IsNumber(Type type) => + type.IsPrimitive || type == typeof(decimal) || type == typeof(Half) || type == typeof(Int128) || type == typeof(UInt128) || type == typeof(BigInteger); + /// /// The operators a value object can declare and EF translates as the database's own: arithmetic, bitwise and /// shift. Comparison is 's, equality is the record's, and a conversion is left to EF. diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs index 4cbf7979..9dfd9f55 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs @@ -1,3 +1,4 @@ +using System.Linq.Expressions; using System.Reflection; using CodoMetis.TypeKit.CompilerServices; using CodoMetis.TypeKit.ValueObjects; @@ -73,6 +74,10 @@ private sealed class Translator(ISqlExpressionFactory sqlExpressionFactory, IRel ) { if (instance is not null || arguments.Count != 1) return null; + + if (WrappedResult(method) is { } wrapping) + return WrappedValue.AsValueObject(arguments[0], wrapping, sqlExpressionFactory, typeMappingSource); + if (method.GetCustomAttribute() is null) return null; var parameterType = method.GetParameters()[0].ParameterType; @@ -84,6 +89,19 @@ private sealed class Translator(ISqlExpressionFactory sqlExpressionFactory, IRel return WrappedValue.Of(arguments[0], valueObject, valueType, sqlExpressionFactory, typeMappingSource); } + + /// + /// The value object is + /// of, which a generated operator's result is rewritten to (); + /// for any other method. + /// + private static GeneratedValueObjectAttribute? WrappedResult(MethodInfo method) => + method.DeclaringType is { IsGenericType: true } converter + && converter.GetGenericTypeDefinition() == typeof(ValueObjectConverter<,>) + && ValueObjectTypes.Describe(converter.GetGenericArguments()[0]) is { } valueObject + && method == ConverterMethods.Materialize(valueObject) + ? valueObject + : null; } } @@ -93,9 +111,10 @@ internal static class WrappedValue /// /// A column is re-typed rather than cast: it already holds the wrapped value, and a cast is not a /// no-op everywhere (on SQL Server, CAST(Code AS nvarchar(max)) can stop an index seek). - /// The mapping keeps the column's store type, so its facets survive. Any other operand, such as a - /// parameter or a function result, is converted, and so is the element column of a primitive - /// collection (). + /// The mapping keeps the column's store type, so its facets survive. So is the database's arithmetic + /// over such columns, (o.Price * 2m).Value, which a cast to the column's store type would round + /// on SQL Server. Any other operand, such as a parameter or a function's result, is converted, and so + /// is the element column of a primitive collection (). /// /// /// , so EF refuses the query, where the operand's converter is not TypeKit's: @@ -116,15 +135,114 @@ IRelationalTypeMappingSource typeMappingSource var mapping = (operand.TypeMapping?.StoreType is { } storeType ? typeMappingSource.FindMapping(valueType, storeType) : null) ?? typeMappingSource.FindMapping(valueType); - if (operand.TypeMapping is { } operandMapping - && (operandMapping.Converter is not { } converter || !ValueObjectMapping.For(valueObject).StoresTheWrappedValue(converter, mapping?.Converter))) - return null; + if (!HoldsTheWrappedValue(operand, valueObject, mapping)) return null; + + return Retyped(operand, valueType, mapping, sqlExpressionFactory) ?? sqlExpressionFactory.Convert(operand, valueType, mapping); + } + + /// + /// Whether holds the wrapped value, as a column TypeKit's converter stores does; each + /// branch of a CASE or a COALESCE on its own, since EF gives the whole the first branch's mapping and + /// leaves each column its own. + /// + private static bool HoldsTheWrappedValue(SqlExpression operand, GeneratedValueObjectAttribute valueObject, RelationalTypeMapping? mapping) => + Branches(operand) is { } branches + ? branches.All(branch => HoldsTheWrappedValue(branch, valueObject, mapping)) + : operand.TypeMapping is not { } operandMapping + || (operandMapping.Converter is { } converter && ValueObjectMapping.For(valueObject).StoresTheWrappedValue(converter, mapping?.Converter)); + + /// + /// The wrapped type's as the value object over it, which is what the database holds + /// for one: the wrapped type's arithmetic over a value object's columns, re-typed, with the value object's + /// mapping for the same store type, so it compares with and is read as a value object. + /// + /// + /// Re-typed rather than cast, as re-types: a cast to the column's store type is no identity on + /// SQL Server, where CAST([p].[Price] / 3.0 AS decimal(18,2)) rounds what memory does not. A conversion + /// made is undone; any other operand is converted. + /// + public static SqlExpression AsValueObject( + SqlExpression operand, + GeneratedValueObjectAttribute valueObject, + ISqlExpressionFactory sqlExpressionFactory, + IRelationalTypeMappingSource typeMappingSource + ) + { + var type = valueObject.ValueObjectType; + var mapping = (operand.TypeMapping?.StoreType is { } storeType ? typeMappingSource.FindMapping(type, storeType) : null) + ?? typeMappingSource.FindMapping(type); + + if (operand is SqlUnaryExpression { OperatorType: ExpressionType.Convert } conversion && conversion.Operand.Type == type) return conversion.Operand; - return operand is ColumnExpression column && !IsCollectionElement(column) - ? new ColumnExpression(column.Name, column.TableAlias, valueType, mapping, column.IsNullable) - : sqlExpressionFactory.Convert(operand, valueType, mapping); + return Retyped(operand, type, mapping, sqlExpressionFactory) ?? sqlExpressionFactory.Convert(operand, type, mapping); } + /// + /// as , with , where it holds that value + /// already: a table's column, the database's arithmetic over columns, or a CASE or COALESCE of such, + /// each branch re-typed as it would be alone; for anything else. + /// + /// + /// A CASE (?:) or a COALESCE (??) of the database's arithmetic was cast whole, which + /// rounded on SQL Server: (o.Optional / 32m ?? o.Price / 32m).Value read 0.09 where memory reads 0.09375, a + /// filter on it lost its row, and so did arithmetic over a conditional (measured on Azure SQL Edge). A branch + /// that holds the value some other way, such as a parameter, is converted alone. So are a subquery of one value and + /// the MAX or MIN it reads. + /// + private static SqlExpression? Retyped(SqlExpression operand, Type type, RelationalTypeMapping? mapping, ISqlExpressionFactory sqlExpressionFactory) + { + SqlExpression Branch(SqlExpression branch) => + branch is SqlConstantExpression { Value: null } + ? new SqlConstantExpression(null, type, mapping) + : Retyped(branch, type, mapping, sqlExpressionFactory) ?? sqlExpressionFactory.Convert(branch, type, mapping); + + return operand switch + { + ColumnExpression column when !IsCollectionElement(column) => new ColumnExpression(column.Name, column.TableAlias, type, mapping, column.IsNullable), + SqlBinaryExpression binary when IsArithmetic(binary.OperatorType) => new SqlBinaryExpression(binary.OperatorType, binary.Left, binary.Right, type, mapping), + SqlUnaryExpression { OperatorType: ExpressionType.Negate } negation => new SqlUnaryExpression(ExpressionType.Negate, negation.Operand, type, mapping), + + // SQLite's decimal arithmetic: ef_add, ef_multiply, ef_divide, ef_mod, ef_negate. + SqlFunctionExpression { Instance: null, Schema: null, IsNiladic: false, Name: "ef_add" or "ef_multiply" or "ef_divide" or "ef_mod" or "ef_negate" } function => + sqlExpressionFactory.Function(function.Name, function.Arguments, function.IsNullable, function.ArgumentsPropagateNullability, type, mapping), + + CaseExpression @case => + new CaseExpression(@case.Operand, [.. @case.WhenClauses.Select(clause => new CaseWhenClause(clause.Test, Branch(clause.Result)))], + @case.ElseResult is { } elseResult ? Branch(elseResult) : null), + + SqlFunctionExpression { Arguments: { } arguments, ArgumentsPropagateNullability: { } propagates } coalesce when IsCoalesce(coalesce) => + sqlExpressionFactory.Function(coalesce.Name, [.. arguments.Select(Branch)], coalesce.IsNullable, propagates, type, mapping), + + // MAX and MIN are one of the values they read, as a COALESCE is. + SqlFunctionExpression { Instance: null, Schema: null, Arguments: [var argument], ArgumentsPropagateNullability: { } propagates } aggregate + when aggregate.Name.Equals("MAX", StringComparison.OrdinalIgnoreCase) || aggregate.Name.Equals("MIN", StringComparison.OrdinalIgnoreCase) => + sqlExpressionFactory.Function(aggregate.Name, [Branch(argument)], aggregate.IsNullable, propagates, type, mapping), + + // A subquery of one value, such as a navigation's largest quotient: cast whole, it rounded as a CASE did. + ScalarSubqueryExpression { Subquery: { Projection: [var projection] } select } scalar + when Retyped(projection.Expression, type, mapping, sqlExpressionFactory) is { } projected => + scalar.Update(select.Update(select.Tables, select.Predicate, select.GroupBy, select.Having, [projection.Update(projected)], select.Orderings, select.Offset, select.Limit)), + + _ => null, + }; + } + + /// The results of a CASE and the arguments of a COALESCE, each of which is the whole's value; for anything else. + private static IEnumerable? Branches(SqlExpression operand) => + operand switch + { + CaseExpression @case => [.. @case.WhenClauses.Select(clause => clause.Result), .. @case.ElseResult is { } elseResult ? [elseResult] : Array.Empty()], + SqlFunctionExpression { Arguments: { } arguments } coalesce when IsCoalesce(coalesce) => arguments, + _ => null, + }; + + /// ??, which every relational provider translates as COALESCE. + private static bool IsCoalesce(SqlFunctionExpression function) => + function is { Instance: null, Schema: null, IsNiladic: false } && string.Equals(function.Name, "COALESCE", StringComparison.OrdinalIgnoreCase); + + private static bool IsArithmetic(ExpressionType type) => + type is ExpressionType.Add or ExpressionType.Subtract or ExpressionType.Multiply or ExpressionType.Divide or ExpressionType.Modulo; + /// /// The element of a primitive collection is a column of the collection's table expression /// (unnest, json_each, OPENJSON, EF's VALUES), and EF infers the diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueThroughInterface.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueThroughInterface.cs index 865cab7e..65c6b95f 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueThroughInterface.cs +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueThroughInterface.cs @@ -1,4 +1,3 @@ -using System.Collections.Concurrent; using System.Linq.Expressions; using System.Reflection; using CodoMetis.TypeKit.CompilerServices; @@ -60,10 +59,8 @@ public void ProcessModelFinalizing(IConventionModelBuilder modelBuilder, IConven /// /// /// Not to the value object's own Value property: finding it from a is a member lookup the -/// trimmer may break (IL2075). The unwrap is reached through -/// with the value object as a type argument, from an expression whose method the compiler names, so nothing is -/// looked up by name and nothing is made with MakeGenericType. It is public because a compiled model writes -/// a query filter out as C# in the application's assembly. +/// trimmer may break (IL2075). The unwrap is ', reached with the value object as a type +/// argument, so nothing is looked up by name and nothing is made with MakeGenericType. /// /// /// Only the conversion of a value object to its own interface is rewritten: the interface's type argument is the @@ -73,14 +70,8 @@ public void ProcessModelFinalizing(IConventionModelBuilder modelBuilder, IConven /// internal static class ValueThroughInterface { - private static readonly ConcurrentDictionary Unwraps = new(); - public static Expression Rewrite(Expression expression) => Rewriter.Instance.Visit(expression); - /// of . - private static MethodInfo Unwrap(GeneratedValueObjectAttribute valueObject) => - Unwraps.GetOrAdd(valueObject.ValueObjectType, static (_, description) => description.Accept(UnwrapOf.Instance), valueObject); - private sealed class Rewriter : ExpressionVisitor { public static readonly Rewriter Instance = new(); @@ -94,19 +85,9 @@ protected override Expression VisitMember(MemberExpression node) && contract.GetGenericArguments()[0] == conversion.Operand.Type && ValueObjectTypes.Describe(conversion.Operand.Type) is { } valueObject && valueObject.WrappedType == property.PropertyType) - return Expression.Call(Unwrap(valueObject), Visit(conversion.Operand)); + return Expression.Call(ConverterMethods.ProviderValue(valueObject), Visit(conversion.Operand)); return base.VisitMember(node); } } - - private sealed class UnwrapOf : IValueObjectVisitor - { - public static readonly UnwrapOf Instance = new(); - - public MethodInfo Visit() - where TValueObject : IValueObject, IValueObjectMaterializer - where T : notnull => - ((MethodCallExpression)ValueObjectConverter.ToProvider.Body).Method; - } } diff --git a/src/CodoMetis.TypeKit/CompilerServices/TranslatedAsWrappedArithmeticAttribute.cs b/src/CodoMetis.TypeKit/CompilerServices/TranslatedAsWrappedArithmeticAttribute.cs index 899c92e9..23b0e720 100644 --- a/src/CodoMetis.TypeKit/CompilerServices/TranslatedAsWrappedArithmeticAttribute.cs +++ b/src/CodoMetis.TypeKit/CompilerServices/TranslatedAsWrappedArithmeticAttribute.cs @@ -11,7 +11,9 @@ namespace CodoMetis.TypeKit.CompilerServices; /// CodoMetis.TypeKit.Generators puts it on every arithmetic operator it generates, which calls the /// wrapped type's through : an operand of the value object's type is its /// Value, any other operand is passed as it is, and a result of the value object's type wraps the -/// wrapped type's result. +/// wrapped type's result. CodoMetis.TypeKit.EntityFrameworkCore translates the operator so, also where +/// an operand or the result is not the value object (o.Price * 2m, o.Tax / o.Price as a +/// decimal). /// /// /// An operator without it, one written by hand, is refused in a query, naming the value object: EF diff --git a/test/CodoMetis.TypeKit.EntityFrameworkCore.Tests/ArithmeticQueryTests.cs b/test/CodoMetis.TypeKit.EntityFrameworkCore.Tests/ArithmeticQueryTests.cs index 4220a677..e88567e5 100644 --- a/test/CodoMetis.TypeKit.EntityFrameworkCore.Tests/ArithmeticQueryTests.cs +++ b/test/CodoMetis.TypeKit.EntityFrameworkCore.Tests/ArithmeticQueryTests.cs @@ -1,70 +1,471 @@ using CodoMetis.TypeKit.Generators.Probes; +using CodoMetis.TypeKit.ValueObjects; using Microsoft.Data.Sqlite; using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Diagnostics; +using Npgsql; +using Testcontainers.PostgreSql; namespace CodoMetis.TypeKit.EntityFrameworkCore.Tests; /// -/// The arithmetic operators a value object declares, in a query: between two value objects EF translates them as -/// it translates the wrapped type's, and the database computes them, so a filter keeps the rows memory keeps. +/// The arithmetic operators a value object declares, in a query, on every site a query reads one: each query +/// answers as the same query over the objects in memory, which runs the generated operators. /// /// -/// EF reads the operator as SQL arithmetic and never runs it (ef_add("p"."Price", "p"."Tax") on SQLite, -/// measured 2026-10-30, as for two columns). With an operand or a result of another type -/// (Price * 2m, Tax / Price as a ) it applies the value object's converter to the -/// other side, and the query throws: .Value * 2m is the form that translates. +/// +/// EF translates an operator between two value objects as the database's arithmetic over the columns +/// (ef_add("p"."Price", "p"."Tax") on SQLite). With an operand or a result of another type +/// (Price * 2m, Tax / Price as a ) EF applied the value object's converter to +/// the other side, and the query threw: "Invalid cast from 'System.Decimal' to 'ProbeMoney'" and "No coercion operator +/// is defined between types 'ProbeMoney' and 'Nullable<Decimal>'" (measured with EF Core 10.0.12 on SQLite, +/// #186). Unary + was "could not be translated". rewrites every +/// generated operator to the wrapped type's arithmetic. +/// +/// +/// The values divide exactly, so a database's decimal scale cannot tell its answer from memory's. SQLite orders and +/// aggregates no , .Value neither, so those cases run on the servers alone. +/// /// +public static class ArithmeticQueries +{ + private static readonly ProbeMoney Ten = ProbeMoney.From(10m); + private static readonly ProbeMoney Four = ProbeMoney.From(4m); + + /// One query, run against a database and against the same rows in memory; ordered, so the lists compare. + public sealed record Case(Func, IQueryable, IEnumerable> Query, bool OrdersDecimals = false); + + public static readonly Dictionary Cases = new() + { + // Between two value objects into one, which EF translated right before, over TypeKit's converter. + ["Price + Tax, filtered"] = Ids(q => q.Where(p => p.Price + p.Tax > Four)), + ["Price - Tax, filtered"] = Ids(q => q.Where(p => p.Price - p.Tax < Four)), + ["-Price, filtered"] = Ids(q => q.Where(p => -p.Price < -Four)), + ["Price + Tax, projected"] = Values(q => q.Select(p => p.Price + p.Tax)), + + // Another operand, or another result, which EF could not translate. + ["Price * 2m, filtered"] = Ids(q => q.Where(p => p.Price * 2m > Ten)), + ["Price / 2m, filtered"] = Ids(q => q.Where(p => p.Price / 2m >= ProbeMoney.From(2.5m))), + ["Price % 2m, filtered"] = Ids(q => q.Where(p => p.Price % 2m == ProbeMoney.From(1m))), + ["Tax / Price, filtered"] = Ids(q => q.Where(p => p.Tax / p.Price < 0.3m)), + ["checked Price * 2m, filtered"] = Ids(q => q.Where(p => checked(p.Price * 2m) > Ten)), + ["+Price, filtered"] = Ids(q => q.Where(p => +p.Price > Four)), + ["Price * a column, filtered"] = Ids(q => q.Where(p => p.Price * p.Rate > Ten)), + ["Price * a captured factor"] = Ids(q => q.Where(p => p.Price * Factor() > Ten)), + ["(Price + Tax) * 2m, filtered"] = Ids(q => q.Where(p => (p.Price + p.Tax) * 2m > ProbeMoney.From(20m))), + ["Price * 2m + Tax, filtered"] = Ids(q => q.Where(p => p.Price * 2m + p.Tax > ProbeMoney.From(12m))), + ["(Price * 2m).Value, filtered"] = Ids(q => q.Where(p => (p.Price * 2m).Value > 10m)), + + // Past the column's scale, SQL Server's default decimal(18,2): cast to the column's store type, 5 / 32 = 0.15625 read + // 0.16 there, and the filter lost the row (measured on Azure SQL Edge before the re-typing). + ["Price / 32m past the scale"] = Ids(q => q.Where(p => p.Price / 32m < ProbeMoney.From(0.15626m))), + ["(Price / 32m).Value past the scale"] = Ids(q => q.Where(p => (p.Price / 32m).Value < 0.15626m)), + // Composed past the scale: a CASE (?:) or a COALESCE (??) of the database's arithmetic was cast whole, and read + // 0.09 for 3 / 32 = 0.09375 on SQL Server; arithmetic over one, and .ValueOrNull() of an optional one's null + // check, the same (measured on Azure SQL Edge, an outside review of #190). + ["?: of two quotients, .Value"] = Values(q => q.Select(p => (p.Rate > 1m ? p.Price / 32m : p.Tax / 32m).Value)), + ["?: of two quotients * 2m"] = Values(q => q.Select(p => (p.Rate > 1m ? p.Price / 32m : p.Tax / 32m) * 2m)), + ["?? of two quotients, .Value"] = Values(q => q.Select(p => (p.Optional / 32m ?? p.Price / 32m).Value)), + ["?? of two quotients, filtered"] = Ids(q => q.Where(p => (p.Optional / 32m ?? p.Price / 32m).Value < 0.048m)), + ["?? of two quotients * 2m, filtered"] = Ids(q => q.Where(p => (p.Optional / 32m ?? p.Price / 32m) * 2m < ProbeMoney.From(0.1m))), + ["?? of ?: and a quotient, .Value"] = Values(q => q.Select(p => ((p.Rate > 1m ? p.Optional / 32m : null) ?? p.Tax / 32m).Value)), + ["an optional quotient, ValueOrNull"] = Values(q => q.Select(p => (p.Optional / 32m).ValueOrNull())), + ["an optional quotient, filtered"] = Ids(q => q.Where(p => (p.Optional / 32m).ValueOrNull() < 0.048m)), + + ["an optional one * 2m, filtered"] = Ids(q => q.Where(p => p.Optional * 2m > Four)), + ["an optional one / Price"] = Ids(q => q.Where(p => p.Optional / p.Price > 1m)), + ["a JSON property * 2m, filtered"] = Ids(q => q.Where(p => p.Doc.Price * 2m > Ten)), + ["over a navigation"] = new((_, b) => b.Where(x => x.Items.Any(i => i.Price * 2m > Ten)).OrderBy(x => x.Id).Select(x => (object?)x.Id)), + + ["Price * 2m, projected"] = Values(q => q.Select(p => p.Price * 2m)), + ["Price / 2m, projected"] = Values(q => q.Select(p => p.Price / 2m)), + ["Price % 2m, projected"] = Values(q => q.Select(p => p.Price % 2m)), + ["Tax / Price, projected"] = Values(q => q.Select(p => p.Tax / p.Price)), + ["+Price, projected"] = Values(q => q.Select(p => +p.Price)), + ["(Price * 2m).Value, projected"] = Values(q => q.Select(p => (p.Price * 2m).Value)), + ["(Price / 32m).Value, projected"] = Values(q => q.Select(p => (p.Price / 32m).Value)), + ["an optional one * 2m, projected"] = Values(q => q.Select(p => p.Optional * 2m)), + ["an optional one * 2m + Tax"] = Values(q => q.Select(p => p.Optional * 2m + p.Tax)), + ["-an optional one, projected"] = Values(q => q.Select(p => -p.Optional)), + ["Price * an optional factor"] = Values(q => q.Select(p => p.Price * p.Share)), + ["Price * an optional factor, filtered"] = Ids(q => q.Where(p => p.Price * p.Share > Four)), + + ["ordered by Tax / Price"] = Ids(q => q.OrderBy(p => p.Tax / p.Price).ThenBy(p => p.Id), ordersDecimals: true), + ["ordered by Price * a column"] = Ids(q => q.OrderBy(p => p.Price * p.Rate).ThenBy(p => p.Id), ordersDecimals: true), + ["the maximum of Tax / Price"] = new((q, _) => [q.Max(p => p.Tax / p.Price)], OrdersDecimals: true), + ["a basket's largest quotient, .Value"] = new((_, b) => b.OrderBy(x => x.Id).Select(x => (object?)x.Items.Max(i => i.Price / 32m).Value), OrdersDecimals: true), + }; + + public static TheoryData Names => [.. Cases.Keys]; + + /// A factor the query captures, so it reaches the database as a parameter. + private static decimal Factor() => 2m; + + private static Case Ids(Func, IQueryable> query, bool ordersDecimals = false) => + new((q, _) => ordersDecimals ? query(q).Select(p => (object?)p.Id) : query(q).OrderBy(p => p.Id).Select(p => (object?)p.Id), ordersDecimals); + + private static Case Values(Func, IQueryable> query) => + new((q, _) => query(q.OrderBy(p => p.Id)).AsEnumerable().Select(value => (object?)value)); + + /// The rows, and their baskets, as both the database and memory hold them. + public static (List Items, List Baskets) Rows() + { + Priced Row(int id, decimal price, decimal tax, decimal? optional, decimal rate, decimal? share, int basket) => new() + { + Id = id, Price = ProbeMoney.From(price), Tax = ProbeMoney.From(tax), Rate = rate, Share = share, BasketId = basket, + Optional = optional is { } value ? ProbeMoney.From(value) : null, + Doc = new PricedDoc { Price = ProbeMoney.From(price) }, + }; + + List items = [Row(1, 2m, 0.5m, 3m, 3m, 0.5m, 1), Row(2, 10m, 2m, null, 0.5m, null, 1), Row(3, 5m, 2m, 1.5m, 2.5m, 2m, 2)]; + List baskets = [new() { Id = 1, Items = [items[0], items[1]] }, new() { Id = 2, Items = [items[2]] }]; + + return (items, baskets); + } + + public static async Task Seed(PricedDb db) + { + await db.Database.EnsureDeletedAsync(); + await db.Database.EnsureCreatedAsync(); + db.Baskets.AddRange(Rows().Baskets); + await db.SaveChangesAsync(); + db.ChangeTracker.Clear(); + } + + /// The case's answer from the database, as memory gives it. + public static void AnswersAsMemory(string name, PricedDb db) + { + var @case = Cases[name]; + var (items, baskets) = Rows(); + + var memory = @case.Query(items.AsQueryable(), baskets.AsQueryable()).ToList(); + + @case.Query(db.Priced, db.Baskets).ToList().ShouldBe(memory, name); + } + + public sealed class Priced + { + public int Id { get; set; } + + public ProbeMoney Price { get; set; } + + public ProbeMoney Tax { get; set; } + + public ProbeMoney? Optional { get; set; } + + public decimal Rate { get; set; } + + public decimal? Share { get; set; } + + public PricedDoc Doc { get; set; } = new(); + + public int BasketId { get; set; } + } + + public sealed class PricedDoc + { + public ProbeMoney Price { get; set; } + } + + public sealed class SmallRow + { + public int Id { get; set; } + + public SmallLevel Level { get; set; } + } + + public sealed class Basket + { + public int Id { get; set; } + + public List Items { get; set; } = []; + } + + public class PricedDb(DbContextOptions options) : DbContext(options) + { + public DbSet Priced => Set(); + + public DbSet Baskets => Set(); + + public DbSet Small => Set(); + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity().Property(p => p.Id).ValueGeneratedNever(); + modelBuilder.Entity().OwnsOne(p => p.Doc, doc => doc.ToJson()); + modelBuilder.Entity().Property(b => b.Id).ValueGeneratedNever(); + } + } +} + +/// on SQLite. public sealed class ArithmeticQueryTests : IDisposable { private readonly SqliteConnection _connection = new("Data Source=:memory:"); + [Theory] + [MemberData(nameof(ArithmeticQueries.Names), MemberType = typeof(ArithmeticQueries))] + public async Task On_SQLite_an_operator_answers_as_memory(string name) + { + if (ArithmeticQueries.Cases[name].OrdersDecimals) Assert.Skip("SQLite orders and aggregates no decimal, .Value neither"); + + await using var db = await Seeded(); + + ArithmeticQueries.AnswersAsMemory(name, db); + } + + /// The SQL is the wrapped type's: the value object's column re-typed, the result never cast. [Fact] - public void An_operator_between_value_objects_runs_in_the_database() + public async Task The_SQL_is_the_wrapped_types_arithmetic_over_the_columns() { - using var db = Seeded(); + await using var db = await Seeded(); + + db.Priced.Where(p => p.Price + p.Tax > ProbeMoney.From(3m)).ToQueryString().ShouldContain("ef_add(\"p\".\"Price\", \"p\".\"Tax\")"); + db.Priced.Where(p => p.Price * 2m > ProbeMoney.From(3m)).ToQueryString().ShouldContain("ef_multiply(\"p\".\"Price\", '2.0')"); + db.Priced.Select(p => p.Tax / p.Price).ToQueryString().ShouldContain("ef_divide(\"p\".\"Tax\", \"p\".\"Price\")"); + db.Priced.Where(p => +p.Price > ProbeMoney.From(3m)).ToQueryString().ShouldContain("ef_compare(\"p\".\"Price\", "); + db.Priced.Where(p => (p.Price * 2m).Value > 3m).ToQueryString().ShouldNotContain("CAST"); - db.Priced.Where(p => p.Price + p.Tax > ProbeMoney.From(3m)).Select(p => p.Id).ToList().ShouldBe([2]); - db.Priced.Where(p => p.Price - p.Tax < ProbeMoney.From(2m)).Select(p => p.Id).ToList().ShouldBe([1]); - db.Priced.Where(p => -p.Price < ProbeMoney.From(-5m)).Select(p => p.Id).ToList().ShouldBe([2]); - db.Priced.OrderBy(p => p.Id).Select(p => (p.Price + p.Tax).Value).ToList().ShouldBe([2.0m, 12.0m]); - db.Priced.Where(p => p.Price + p.Tax > ProbeMoney.From(3m)).ToQueryString().ShouldContain("ef_add("); + // An optional operand: the arithmetic in one CASE that checks it, which a ?? or .Value re-types, never casts. + db.Priced.Select(p => p.Optional * 2m + p.Tax).ToQueryString() + .ShouldContain("WHEN \"p\".\"Optional\" IS NOT NULL THEN ef_add(ef_multiply(\"p\".\"Optional\", '2.0'), \"p\".\"Tax\")"); + db.Priced.Select(p => (p.Optional / 32m ?? p.Price / 32m).Value).ToQueryString().ShouldNotContain("CAST"); + db.Priced.Select(p => (p.Rate > 1m ? p.Price / 32m : p.Tax / 32m) * 2m).ToQueryString().ShouldNotContain("CAST"); } + /// + /// byte, short and char have no arithmetic of their own in an expression tree, where C# + /// computes it as an 's: such an operator is refused, naming the value object, between two value + /// objects too. + /// [Fact] - public void An_operator_with_another_operand_throws_and_Value_translates() + public async Task An_operator_the_wrapped_type_has_no_expression_for_is_refused_naming_the_value_object() { - using var db = Seeded(); + await using var db = await Seeded(); - Should.Throw(() => db.Priced.Where(p => p.Price * 2m > ProbeMoney.From(10m)).ToList()); - Should.Throw(() => db.Priced.Select(p => p.Tax / p.Price).ToList()); - db.Priced.Where(p => p.Price.Value * 2m > 10m).Select(p => p.Id).ToList().ShouldBe([2]); + var scaled = Should.Throw(() => db.Small.Where(s => s.Level * (byte)2 > SmallLevel.From(4)).ToList()); + var added = Should.Throw(() => db.Small.Where(s => s.Level + s.Level > SmallLevel.From(4)).ToList()); + + scaled.Message.ShouldStartWith("SmallLevel * Byte is used in a query"); + added.Message.ShouldStartWith("SmallLevel + SmallLevel is used in a query"); + scaled.Message.ShouldContain(".Value"); } public void Dispose() => _connection.Dispose(); - private PricedDb Seeded() + private async Task Seeded() { _connection.Open(); - var db = new PricedDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); - db.Database.EnsureCreated(); - db.Priced.AddRange(new Priced { Id = 1, Price = ProbeMoney.From(1.5m), Tax = ProbeMoney.From(0.5m) }, - new Priced { Id = 2, Price = ProbeMoney.From(10m), Tax = ProbeMoney.From(2m) }); - db.SaveChanges(); - db.ChangeTracker.Clear(); + var db = new ArithmeticQueries.PricedDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); + await ArithmeticQueries.Seed(db); return db; } +} - public sealed class Priced +/// A query filter that scales a value object, and one with a ratio: rewritten where the model is built. +public sealed class ArithmeticQueryFilterTests : IDisposable +{ + private readonly SqliteConnection _connection = new("Data Source=:memory:"); + + [Fact] + public async Task Named_query_filters_with_an_operator_answer_as_memory() { - public int Id { get; set; } + _connection.Open(); + await using var db = new FilteredDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); + await ArithmeticQueries.Seed(db); - public ProbeMoney Price { get; set; } + var memory = ArithmeticQueries.Rows().Items.Where(p => p.Price * 2m > ProbeMoney.From(5m) && p.Tax / p.Price < 0.3m).Select(p => p.Id); - public ProbeMoney Tax { get; set; } + db.Priced.OrderBy(p => p.Id).Select(p => p.Id).ToList().ShouldBe(memory); } - public sealed class PricedDb(DbContextOptions options) : DbContext(options) + [Fact] + public async Task An_anonymous_query_filter_with_an_operator_answers_as_memory() { - public DbSet Priced => Set(); + _connection.Open(); + await using var db = new AnonymousFilteredDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); + await ArithmeticQueries.Seed(db); + + var memory = ArithmeticQueries.Rows().Items.Where(p => p.Optional * 2m > ProbeMoney.From(4m)).Select(p => p.Id); + + db.Priced.OrderBy(p => p.Id).Select(p => p.Id).ToList().ShouldBe(memory); + } + + public void Dispose() => _connection.Dispose(); + + private sealed class AnonymousFilteredDb(DbContextOptions options) : ArithmeticQueries.PricedDb(options) + { + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + base.OnModelCreating(modelBuilder); + modelBuilder.Entity().HasQueryFilter(p => p.Optional * 2m > ProbeMoney.From(4m)); + } + } + + private sealed class FilteredDb(DbContextOptions options) : ArithmeticQueries.PricedDb(options) + { + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + base.OnModelCreating(modelBuilder); + modelBuilder.Entity() + .HasQueryFilter("Scaled", p => p.Price * 2m > ProbeMoney.From(5m)) + .HasQueryFilter("Ratio", p => p.Tax / p.Price < 0.3m); + } + } +} + +/// on PostgreSQL. +[Trait("Needs", "Docker")] +public sealed class PostgreSqlArithmeticQueryTests(PostgreSqlArithmeticQueryTests.Database database) : IClassFixture +{ + [Theory] + [MemberData(nameof(ArithmeticQueries.Names), MemberType = typeof(ArithmeticQueries))] + public async Task On_PostgreSQL_an_operator_answers_as_memory(string name) + { + await using var db = database.Create(); + + ArithmeticQueries.AnswersAsMemory(name, db); + } + + public sealed class Database : IAsyncLifetime + { + private readonly PostgreSqlContainer _container = new PostgreSqlBuilder("postgres:17-alpine").Build(); + private DbContextOptions? _options; + + public ArithmeticQueries.PricedDb Create() => new(_options ?? throw new InvalidOperationException("The database is not started.")); + + public async ValueTask InitializeAsync() + { + await _container.StartAsync(); + + var connectionString = new NpgsqlConnectionStringBuilder(_container.GetConnectionString()) { Database = "typekit_arithmetic" }.ConnectionString; + _options = new DbContextOptionsBuilder().UseNpgsql(connectionString).UseTypeKit().Options; + + await using var db = Create(); + await ArithmeticQueries.Seed(db); + } + + public async ValueTask DisposeAsync() => await _container.DisposeAsync(); + } +} + +/// on SQL Server. +[Collection(SqlServerCollection.Name)] +[Trait("Needs", "Docker")] +public sealed class SqlServerArithmeticQueryTests(SqlServer server) +{ + private static readonly SemaphoreSlim Seeding = new(1, 1); + private static bool _seeded; + + [Theory] + [MemberData(nameof(ArithmeticQueries.Names), MemberType = typeof(ArithmeticQueries))] + public async Task On_SQL_Server_an_operator_answers_as_memory(string name) + { + await using var db = await CreateAsync(); + + ArithmeticQueries.AnswersAsMemory(name, db); + } + + private async Task CreateAsync() + { + var options = new DbContextOptionsBuilder().UseSqlServer(await server.DatabaseAsync("typekit_arithmetic")).UseTypeKit() + .ConfigureWarnings(warnings => warnings.Ignore(CoreEventId.ManyServiceProvidersCreatedWarning)) + .Options; + var db = new ArithmeticQueries.PricedDb(options); + + await Seeding.WaitAsync(TestContext.Current.CancellationToken); + try + { + if (!_seeded) await ArithmeticQueries.Seed(db); + _seeded = true; + } + finally + { + Seeding.Release(); + } + + return db; + } +} + +/// Added and scaled by a , which an expression tree cannot do: C# computes two as s. +public readonly partial record struct SmallLevel : IValue, + System.Numerics.IAdditionOperators, + System.Numerics.IMultiplyOperators; + +/// +/// A value object the application stores with a converter of its own, here as "M:" + value: its column does +/// not hold the value, so an operator over it is not translated, as .Value is not (decision 56). +/// +/// +/// Measured on SQLite with EF's own translation of an operator between two value objects: Price + Tax > 4 +/// became "p"."Price" + "p"."Tax" > 'M:4' and kept no row where memory keeps two, and -Price < -4 +/// kept all three where memory keeps two. +/// +public sealed class ForeignConvertedArithmeticTests : IDisposable +{ + private readonly SqliteConnection _connection = new("Data Source=:memory:"); + + [Fact] + public async Task An_operator_over_a_column_another_converter_stores_is_not_translated() + { + _connection.Open(); + await using var db = new ForeignDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); + await ArithmeticQueries.Seed(db); + + foreach (var query in new Func[] + { + () => db.Priced.Where(p => p.Price + p.Tax > ProbeMoney.From(4m)).ToList(), + () => db.Priced.Where(p => -p.Price < ProbeMoney.From(-4m)).ToList(), + () => db.Priced.Where(p => p.Price * 2m > ProbeMoney.From(4m)).ToList(), + () => db.Priced.Where(p => p.Tax / p.Price < 0.3m).ToList(), + }) + Should.Throw(query).Message.ShouldContain("could not be translated"); + + // As the query's last step EF computes it in memory, after the application's converter read the value objects. + db.Priced.OrderBy(p => p.Id).Select(p => p.Tax / p.Price).ToList().ShouldBe([0.25m, 0.2m, 0.4m]); + db.Priced.Count(p => p.Price == ProbeMoney.From(10m)).ShouldBe(1); + } + + /// + /// Computed in memory, an optional operand's null is the operator's: unwrapped by a conversion alone, the null row + /// threw "Nullable object must have a value" (an outside review of #190). + /// + [Fact] + public async Task Computed_in_memory_an_operator_over_an_optional_one_answers_null_as_memory_does() + { + _connection.Open(); + await using var db = new ForeignDb(new DbContextOptionsBuilder().UseSqlite(_connection).UseTypeKit().Options); + await ArithmeticQueries.Seed(db); + var items = ArithmeticQueries.Rows().Items.AsQueryable(); + + void AnswersAsMemory(Func, IQueryable> query) => + query(db.Priced.OrderBy(p => p.Id)).ToList().ShouldBe(query(items.OrderBy(p => p.Id)).ToList()); + + AnswersAsMemory(q => q.Select(p => p.Optional * 2m)); + AnswersAsMemory(q => q.Select(p => p.Optional / p.Price)); + AnswersAsMemory(q => q.Select(p => -p.Optional + p.Tax)); + AnswersAsMemory(q => q.Select(p => p.Optional * 2m * 3m)); + AnswersAsMemory(q => q.Select(p => p.Price * p.Share)); + AnswersAsMemory(q => q.Select(p => (p.Optional / 32m).ValueOrNull())); + } + + public void Dispose() => _connection.Dispose(); + + private sealed class ForeignDb(DbContextOptions options) : ArithmeticQueries.PricedDb(options) + { + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + base.OnModelCreating(modelBuilder); + + var entity = modelBuilder.Entity(); + foreach (var property in new[] { entity.Property(p => p.Price), entity.Property(p => p.Tax) }) + property.HasConversion(money => "M:" + money.Value.ToString(System.Globalization.CultureInfo.InvariantCulture), + text => ProbeMoney.From(decimal.Parse(text.Substring(2), System.Globalization.CultureInfo.InvariantCulture))); + + entity.Property(p => p.Optional) + .HasConversion(money => "M:" + money!.Value.Value.ToString(System.Globalization.CultureInfo.InvariantCulture), + text => ProbeMoney.From(decimal.Parse(text.Substring(2), System.Globalization.CultureInfo.InvariantCulture))); + } } } diff --git a/test/oracles/axes.md b/test/oracles/axes.md index 257d2f05..427cccb6 100644 --- a/test/oracles/axes.md +++ b/test/oracles/axes.md @@ -23,6 +23,8 @@ footprint, it runs with the suite. - **Arithmetic parity (differential).** A value object's generated arithmetic operator gives what the wrapped type's own gives, value or exception, unchecked and checked, for every numeric wrapped type and both record shapes. `ArithmeticParityTests`. +- **Arithmetic in a query (differential).** A value object's generated operator in an EF Core query keeps the rows, + and reads the values, the same operator keeps in memory, on SQLite, PostgreSQL and SQL Server. `ArithmeticQueryTests`. - **No refusal quotes the input (exception).** A sentinel in refused input never reaches a message. `UnreadableInputTests`. - **Off the wire (exception).** `Option` and `Result` refuse System.Text.Json in both directions. `NotWireTypeTests`. @@ -43,6 +45,7 @@ footprint, it runs with the suite. | EF | the query | `QueryShapes` | `QueryTranslationVariantTests` | | | Analyzers | where a type argument is written, and what it is | `TypeArgumentSites`, `TypeArgumentKinds` (`test/CodoMetis.TypeKit.Analyzers.Tests/Variants/`) | `TypeArgumentConstraintVariantTests` | | | Generation | the declaration's shape | record struct, record class, struct, class; sealed or not; over each wrapped kind | `BuildOutcomeTests`, `GeneratedSurfaceTests` | | +| EF | an operator in a query | the operator, the other operand (the value object, the wrapped type as a literal, a parameter or a column), the result (the value object or the wrapped type), the site (a column, a nullable column, a JSON property, a navigation, a query filter, named or not), the use (a filter, a projection, an ordering, an aggregate, `.Value` of it), who stores the column (TypeKit's converter, the application's own), generated or written by hand; on SQLite, PostgreSQL and SQL Server | `ArithmeticQueryTests`, `ArithmeticQueryFilterTests`, `ForeignConvertedArithmeticTests`, `HandWrittenArithmeticTests` | an operator over a primitive collection's element; a `GroupBy` key | | Generation | the operator interfaces a plain value object declares | each arithmetic interface, the other operand and the result as the value object or another type, declared directly or through another interface, written by hand (public, explicit, a base record's explicit, an interface's explicit) or not; over every numeric wrapped type and both record shapes | `ArithmeticParityTests`, `ArithmeticTests`, `ArithmeticBuildTests` | an operator interface beside a hand-written `CompareTo` | An empty last column means nobody has looked yet, not that nothing is missing. Fill it when a feature lands or a diff --git a/test/oracles/claims.json b/test/oracles/claims.json index ba7123b2..212af7c8 100644 --- a/test/oracles/claims.json +++ b/test/oracles/claims.json @@ -1834,19 +1834,6 @@ "ArithmeticBuildTests" ] }, - { - "id": "C28-ef-between-value-objects", - "source": "docs/value-objects.md#add-subtract-and-scale", - "quote": "An operator between two value objects runs in the database, as it would for the underlying type: `o.Price + o.Tax > limit` keeps the rows memory keeps.", - "claim": "An operator between two value objects runs in the database, as it would for the underlying type: `o.Price + o.Tax > limit` keeps the rows memory keeps.", - "oracle": "differential", - "surfaces": [ - "GeneratedArithmetic.*" - ], - "heldBy": [ - "ArithmeticQueryTests" - ] - }, { "id": "C28-ef-own-operator-rejected", "source": "src/CodoMetis.TypeKit.Generators/README.md#write-your-own-members", @@ -1873,6 +1860,22 @@ "HandWrittenArithmeticTests" ] }, + { + "id": "C28-ef-runs-as-underlying-arithmetic", + "source": "docs/value-objects.md#add-subtract-and-scale", + "quote": "Each operator runs in the database as the underlying type's arithmetic: `o.Price + o.Tax > limit`, `o.Price * 2m > limit` and `o.Tax / o.Price < 0.3m` keep the rows memory keeps.", + "claim": "Each operator runs in the database as the underlying type's arithmetic: `o.Price + o.Tax > limit`, `o.Price * 2m > limit` and `o.Tax / o.Price < 0.3m` keep the rows memory keeps.", + "oracle": "differential", + "surfaces": [ + "GeneratedArithmetic.*", + "TranslatedAsWrappedArithmeticAttribute.*" + ], + "heldBy": [ + "ArithmeticQueryTests", + "PostgreSqlArithmeticQueryTests", + "SqlServerArithmeticQueryTests" + ] + }, { "id": "C28-generated-operators-marked", "source": "src/CodoMetis.TypeKit.Generators/README.md#write-your-own-members", @@ -2660,6 +2663,52 @@ "SqlServerQueryVariantTests" ] }, + { + "id": "E18-arithmetic-on-every-provider", + "source": "src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries", + "quote": "The database computes them, on SQLite, PostgreSQL and SQL Server alike.", + "claim": "The database computes them, on SQLite, PostgreSQL and SQL Server alike.", + "oracle": "differential", + "surfaces": [ + "GeneratedArithmetic.*", + "TranslatedAsWrappedArithmeticAttribute.*" + ], + "heldBy": [ + "ArithmeticQueryTests", + "PostgreSqlArithmeticQueryTests", + "SqlServerArithmeticQueryTests" + ] + }, + { + "id": "E18-arithmetic-rows-as-memory", + "source": "src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries", + "quote": "`o.Price * 2m > limit` keeps the rows memory keeps, and `o.Tax / o.Price` reads a `decimal`.", + "claim": "`o.Price * 2m > limit` keeps the rows memory keeps, and `o.Tax / o.Price` reads a `decimal`.", + "oracle": "differential", + "surfaces": [ + "GeneratedArithmetic.*", + "TranslatedAsWrappedArithmeticAttribute.*" + ], + "heldBy": [ + "ArithmeticQueryTests", + "PostgreSqlArithmeticQueryTests", + "SqlServerArithmeticQueryTests" + ] + }, + { + "id": "E18-foreign-converter-not-translated", + "source": "src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries", + "quote": "an operator isn't translated, as `.Value` isn't: the column doesn't hold the underlying value, so EF's \"could not be translated\" is the answer.", + "claim": "an operator isn't translated, as `.Value` isn't: the column doesn't hold the underlying value, so EF's \"could not be translated\" is the answer.", + "oracle": "exception", + "surfaces": [ + "GeneratedArithmetic.*", + "TranslatedAsWrappedArithmeticAttribute.*" + ], + "heldBy": [ + "ForeignConvertedArithmeticTests" + ] + }, { "id": "E18-own-operator-over-passed-values", "source": "src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries", @@ -2699,6 +2748,20 @@ "HandWrittenArithmeticTests" ] }, + { + "id": "E18-small-types-throw", + "source": "src/CodoMetis.TypeKit.EntityFrameworkCore/README.md#queries", + "quote": "an operator throws as well, naming the value object.", + "claim": "an operator throws as well, naming the value object.", + "oracle": "exception", + "surfaces": [ + "GeneratedArithmetic.*", + "TranslatedAsWrappedArithmeticAttribute.*" + ], + "heldBy": [ + "ArithmeticQueryTests" + ] + }, { "id": "G01-a-version-7-guid", "source": "src/CodoMetis.TypeKit.Generators/README.md#declare-a-value-object", diff --git a/test/oracles/oracles.json b/test/oracles/oracles.json index 401bd38c..8eb4c3f6 100644 --- a/test/oracles/oracles.json +++ b/test/oracles/oracles.json @@ -11,6 +11,12 @@ "kind": "differential", "against": "the wrapped type's own operator of the same interface, called by the same generic code in the same checked or unchecked context, for every numeric wrapped type and both record shapes" }, + { + "class": "ArithmeticQueryTests", + "project": "CodoMetis.TypeKit.EntityFrameworkCore.Tests", + "kind": "differential", + "against": "the same query over the same rows in memory, which runs the generated operators: filters, projections, orderings and an aggregate, over a column, a nullable column, a JSON property and a navigation, with another operand or result than the value object" + }, { "class": "JsonParityTests", "project": "CodoMetis.TypeKit.Generators.Tests", diff --git a/test/oracles/seeds/arithmetic-left-to-ef.patch b/test/oracles/seeds/arithmetic-left-to-ef.patch new file mode 100644 index 00000000..af7192ba --- /dev/null +++ b/test/oracles/seeds/arithmetic-left-to-ef.patch @@ -0,0 +1,18 @@ +Seed: A query is not rewritten: a generated operator with another operand or result is left to EF, and a hand-written one runs as the database's arithmetic +Oracles: ArithmeticQueryTests +Fails: every query whose operator has another operand or result than the value object, which EF translates with the value object's converter on the other side and throws, and unary +, which EF does not translate + +diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs +index b1c03eb..153d54d 100644 +--- a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs ++++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectArithmetic.cs +@@ -88,8 +88,7 @@ internal sealed class ValueObjectArithmeticQueryFilterConvention : IModelFinaliz + internal static class ValueObjectArithmetic + { + /// In a query's LINQ tree, which EF has already stripped of what it evaluates in memory. +- public static Expression Rewrite(Expression query) => +- new Rewriter("in a query", ", or do it in memory after the query", skipsWhatRunsInMemory: false).Visit(query); ++ public static Expression Rewrite(Expression query) => query; + + /// + /// In a query filter, where EF has not yet evaluated what reads nothing of the database: an operator over the diff --git a/test/oracles/seeds/column-cast-not-re-typed.patch b/test/oracles/seeds/column-cast-not-re-typed.patch index 1b0c3dc7..9721cc4c 100644 --- a/test/oracles/seeds/column-cast-not-re-typed.patch +++ b/test/oracles/seeds/column-cast-not-re-typed.patch @@ -3,16 +3,14 @@ Oracles: SqlServerQueryVariantTests Fails: the SQL of every query that reads a table column through an accessor, which is no longer the wrapped type's diff --git a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs -index 4cbf797..074e67d 100644 +index 9dfd9f5..eb363d6 100644 --- a/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs +++ b/src/CodoMetis.TypeKit.EntityFrameworkCore/ValueObjectTranslators.cs -@@ -120,9 +120,7 @@ internal static class WrappedValue - && (operandMapping.Converter is not { } converter || !ValueObjectMapping.For(valueObject).StoresTheWrappedValue(converter, mapping?.Converter))) - return null; +@@ -137,7 +137,7 @@ internal static class WrappedValue -- return operand is ColumnExpression column && !IsCollectionElement(column) -- ? new ColumnExpression(column.Name, column.TableAlias, valueType, mapping, column.IsNullable) -- : sqlExpressionFactory.Convert(operand, valueType, mapping); + if (!HoldsTheWrappedValue(operand, valueObject, mapping)) return null; + +- return Retyped(operand, valueType, mapping, sqlExpressionFactory) ?? sqlExpressionFactory.Convert(operand, valueType, mapping); + return sqlExpressionFactory.Convert(operand, valueType, mapping); } diff --git a/tools/ConsumerSmokeTest/Consumers/aot/Data/Program.cs b/tools/ConsumerSmokeTest/Consumers/aot/Data/Program.cs index c3895dd6..8385e53c 100644 --- a/tools/ConsumerSmokeTest/Consumers/aot/Data/Program.cs +++ b/tools/ConsumerSmokeTest/Consumers/aot/Data/Program.cs @@ -36,6 +36,15 @@ await Check("ef-json-column", async () => db.ChangeTracker.Clear(); await Check("ef-value", async () => (await db.Orders.Where(o => o.Code.Value.StartsWith("AB")).SingleAsync()).Quantity.Value.ToString()); db.ChangeTracker.Clear(); +// Generated operators, rewritten to the wrapped type's arithmetic before EF's precompiler translates the query: +// one with another operand, and one between two value objects read through .Value. +await Check("ef-arithmetic", async () => +{ + var scaled = await db.Orders.Where(o => o.Quantity * 3 > Quantity.From(5)).Select(o => o.Quantity * 3).SingleAsync(); + var summed = await db.Orders.Where(o => o.Quantity + o.Quantity > Quantity.From(3)).Select(o => (o.Quantity + o.Quantity).Value).SingleAsync(); + return $"{scaled.Value} {summed}"; +}); +db.ChangeTracker.Clear(); await Check("ef-parameter", async () => (await db.Orders.Where(o => o.Id == id).SingleAsync()).Code.Value); db.ChangeTracker.Clear(); await Check("ef-stored", async () => diff --git a/tools/ConsumerSmokeTest/Consumers/aot/Domain/Shop.cs b/tools/ConsumerSmokeTest/Consumers/aot/Domain/Shop.cs index 5e22ec85..b0b61726 100644 --- a/tools/ConsumerSmokeTest/Consumers/aot/Domain/Shop.cs +++ b/tools/ConsumerSmokeTest/Consumers/aot/Domain/Shop.cs @@ -34,7 +34,7 @@ public enum Rule { Reserved } // fallback with a built-in converter (Uri, an enum), a string, and parsing through a TypeConverter. // Quantity and Amount declare arithmetic, whose operators call the wrapped type's through a type parameter. public readonly partial record struct OrderId : IValue; -public readonly partial record struct Quantity : IValue, System.Numerics.IAdditionOperators; +public readonly partial record struct Quantity : IValue, System.Numerics.IAdditionOperators, System.Numerics.IMultiplyOperators; public readonly partial record struct Amount : IValue, System.Numerics.IAdditionOperators, System.Numerics.IMultiplyOperators; public readonly partial record struct Link : IValue; public readonly partial record struct Weekday : IValue; diff --git a/tools/ConsumerSmokeTest/Program.cs b/tools/ConsumerSmokeTest/Program.cs index 9ed7cd87..cc2c7607 100644 --- a/tools/ConsumerSmokeTest/Program.cs +++ b/tools/ConsumerSmokeTest/Program.cs @@ -454,6 +454,7 @@ private void Aot() AssertContains("""openapi-patch={"type":["null","string"],"description":"A JSON Merge Patch field: absent leaves the value as it is, null clears it, a value sets it."} required=none""", "aot.out", "a patch property is its wrapped type's schema, inlined and nullable, and not required"); AssertContains("ef-insert=1", "aot.out", "EF Core writes value objects through the compiled model"); AssertContains("ef-value=2", "aot.out", ".Value translates in a precompiled query"); + AssertContains("ef-arithmetic=6 4", "aot.out", "a generated operator translates in a precompiled query, with another operand and between two value objects"); AssertContains("ef-parameter=ABC", "aot.out", "a value-object parameter is mapped at run time"); AssertContains("ef-stored=lower", "aot.out", "a stored value the rules refuse is read without them"); AssertContains("ef-identity=1,2", "aot.out", "a key over an integer is generated by the database");