Specialized row decoders per type and pinned byte arrays

Runfile

@@ -73,7 +73,7 @@ tests:
         if [ -n "$NIX" ]; then
             nix-build --no-out-link -A "testsPg${pg}" --argstr hspecArgs "$TARGS"
         else
-            cabal build hpgsql-tests hpgsql-simple-compat-tests
+            cabal build hpgsql-tests # hpgsql-simple-compat-tests
             nix-shell -A "shellPg${pg}" ./default.nix --run "./scripts/run-tests-db-internal.sh $TARGS"
         fi
     done

TODO.md

@@ -0,0 +1,14 @@
+- Check that users can define their own types and create FromPgField instances that derive performant instances. Do they override the specialized methods? How do they do that?
+- Expose a `PinnedByteArray` with `toByteString` to users, move current module to PinnedByteArray.Internal
+- Test both `singleField fieldDecoder` and `singleFieldRowDecoder` for every type in our tests.
+- Do _not_ expose new FromPgField methods. Add new EncodingInternal module, instead.
+  - Check that the non-exposed methods are safe wrt bytearray bounds access by construction, and users can't break that. If that's true, we can omit bounds checks in our row decoding, making row decoders smaller and maybe faster.
+- Some types might still not derive specialized row decoders
+- "Oh no! No colInfo here.. what do we do!?" in hpgsql-simple-compat. This might require a big rethinking of things..
+- Double-check which row encoders we want to use the inlined versions for and which we don't. Tuples?
+- Text internals might be easier to use now?
+- Expose in the FromPgField class two new methods.. inlined and non inlined row decoders with/without bounds checks. Use with-bounds-checks for MonadicRowDecoder, and without-bounds-checks for regular row decoder, because the latter checks type oids
+  - The specialized row decoders are already a problem here! They don't check type OIDs and can read bytes partially. We should ensure this mismatch is not possible.
+- Is `notInlinedSingleFieldRowDecoder` worth keeping? The Generically derived decoder is almost as fast. Maybe for types that aren't records it's a different story, though?
+- Check that we're not holding on to internal buffers when Record fields being materialized into aren't strict
+- Write property-based tests for PinnedByteArray functions

hpgsql-benchmarks/hpgsql-benchmarks.cabal

@@ -72,6 +72,7 @@ executable hpgsql-benchmarks
     , hspec-expectations
     , postgresql-simple
     , resourcet
+    , scientific
     , statistics
     , stm
     , streaming

hpgsql-simple-compat/src/Database/PostgreSQL/Simple/FromField.hs

@@ -177,9 +177,13 @@ class FromField a where
     let dec = Hpgsql.fieldDecoder
      in \f ->
           if Hpgsql.allowedPgTypes dec f
-            then \mbs -> Conversion $ \_encCtx -> case Hpgsql.fieldValueDecoder dec f mbs of
-              Right v -> Ok v
-              Left err -> Errors [toException $ userError err]
+            then \mbs -> Conversion $ \_encCtx -> case mbs of
+              Nothing -> case dec.decodesSqlNullTo of
+                Left err -> Errors [toException $ userError err]
+                Right v -> Ok v
+              Just bs -> case Hpgsql.fieldValueDecoder dec f bs of
+                Right v -> Ok v
+                Left err -> Errors [toException $ userError err]
             else \_ -> Conversion $ \_encCtx -> Errors [toException $ userError "Invalid type OID for FromField instance"]
 
 instance FromField ()

hpgsql-simple-compat/src/Database/PostgreSQL/Simple/HpgsqlUtils.hs

@@ -95,18 +95,29 @@ type FieldParser a = Field -> Maybe ByteString -> Conversion a
 toHpgsqlFieldDecoder :: FieldParser a -> FieldDecoder a
 toHpgsqlFieldDecoder fp =
   FieldDecoder
-    { fieldValueDecoder = \colInfo mbs ->
-        let valConv = fp colInfo mbs
+    { fieldValueDecoder = \colInfo bs ->
+        let valConv = fp colInfo (Just bs)
          in case runConversion valConv colInfo.encodingContext of
               Ok v -> Right v
               Errors errs -> Left (show errs),
+      decodesSqlNullTo =
+        let valConv = fp (error "Oh no! No colInfo here.. what do we do!?") Nothing
+            encCtx = error "We could fake an EncodingContext, at least. TODO."
+         in case runConversion valConv encCtx of
+              Ok v -> Right v
+              Errors errs -> Left (show errs),
       allowedPgTypes = const True -- No way to check if types are valid ahead of time
     }
 
 fromHpgsqlFieldDecoder :: FieldDecoder a -> FieldParser a
-fromHpgsqlFieldDecoder dec = \f mbs -> Conversion $ \_encCtx -> case dec.fieldValueDecoder f mbs of
-  Right v -> Ok v
-  Left err -> Errors [toException $ userError $ show err]
+fromHpgsqlFieldDecoder dec = \f mbs -> Conversion $ \_encCtx ->
+  case mbs of
+    Nothing -> case dec.decodesSqlNullTo of
+      Left err -> Errors [toException $ userError $ show err]
+      Right v -> Ok v
+    Just bs -> case dec.fieldValueDecoder f bs of
+      Right v -> Ok v
+      Left err -> Errors [toException $ userError $ show err]
 
 -- | Given a Hpgsql query, returns the text format with question marks
 -- for query arguments and a row object. With both, you can call