Skip to content

fix: consistent xsd:decimal/xsd:integer datatype across index and novelty (#1329)#1386

Open
bplatz wants to merge 3 commits into
mainfrom
fix/decimal-jsonld-render-1329
Open

fix: consistent xsd:decimal/xsd:integer datatype across index and novelty (#1329)#1386
bplatz wants to merge 3 commits into
mainfrom
fix/decimal-jsonld-render-1329

Conversation

@bplatz

@bplatz bplatz commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #1329: an xsd:decimal served from the binary index rendered with an empty @type ({"@value":"19.99","@type":""}) while the same value in novelty rendered as a bare string. Also fixes a sibling mislabel where indexed > i64 integers came back as xsd:decimal.

Root cause

OType::NUM_BIG_OVERFLOW names no single datatype from o_type alone — its arena holds both overflow xsd:integer (BigInt) and xsd:decimal (BigDecimal). So resolve_datatype_sid(o_type) returns None, and every flake/binding builder fell back to Sid::new(0, "") → empty @type. Novelty copies still carry o_type = XSD_DECIMAL, so they resolved correctly — hence the index-vs-novelty asymmetry.

A second, related issue: the late-materialized EncodedLit for the NumBig arena hardcodes dt_id = DECIMAL (it has no decoded value at creation time), so an indexed overflow integer materialized as xsd:decimal.

Fix

  • New BinaryIndexStore::resolve_datatype_sid_for_value(o_type, &val) derives the datatype from the decoded FlakeValue (Decimalxsd:decimal, BigIntxsd:integer) when o_type can't name it. Wired into every flake/binding builder that has the decoded value in scope: binary_range (x2), binary_history, object_binding, fast_group_count_firsts (x2), block_fetch.
  • binary_scan::matches_datatype_constraint is a pre-decode filter — it dropped indexed big numerics from datatype-constrained patterns (e.g. {"@value":"?p","@type":"xsd:decimal"}) while novelty copies passed. It now decodes to disambiguate only in the rare ambiguous case, keeping the common path decode-free.
  • New FlakeValue::overflow_numeric_datatype_sid() recovers the datatype at materialization time (the EncodedLit can't be fixed at creation). Consulted before the dt_id lookup in both the query materializer and the API formatter; resolve_datatype_sid_for_value delegates to it too.

Because xsd:decimal is an inferable datatype, the consistent output is the bare-string form across crawl and tabular lanes (matching the existing convention). export.rs already had per-variant fallbacks, so it needed no change.

Tests

  • jsonld_decimal_renders_consistently_across_index_and_novelty — crawl + tabular SPARQL + datatype-constrained pattern, asserting the indexed and novelty decimals render identically and both match a xsd:decimal-constrained query.
  • indexed_overflow_integer_reports_xsd_integer_not_decimal — pins the late-materialization datatype recovery.

Both confirmed to fail before the fix and pass after. Full fluree-db-api suite, fluree-db-query/fluree-db-core unit tests, and clippy (all-features) are green.

bplatz added 2 commits June 26, 2026 22:09
Arena-served xsd:decimal rendered with an empty `@type` while the same
value in novelty rendered as a bare string. NUM_BIG_OVERFLOW names no
single datatype from o_type alone (its arena holds both overflow
xsd:integer/BigInt and xsd:decimal/BigDecimal), so resolve_datatype_sid
returned None and callers fell back to Sid::new(0, ""); novelty copies
still carry o_type=XSD_DECIMAL and resolved correctly.

Add BinaryIndexStore::resolve_datatype_sid_for_value, which derives the
datatype from the decoded FlakeValue (Decimal->xsd:decimal,
BigInt->xsd:integer) when o_type cannot name it. Wire it into every
flake/binding builder that has the decoded value in scope: binary_range,
binary_history, object_binding, fast_group_count_firsts, block_fetch.

Also fix the pre-decode datatype filter in binary_scan: it ran on o_type
alone and dropped indexed big numerics from datatype-constrained patterns
(e.g. {"@value":"?p","@type":"xsd:decimal"}) while novelty copies passed.
It now decodes to disambiguate only in the rare ambiguous case, keeping
the common path decode-free.
…erics

BigInt (overflow xsd:integer) and BigDecimal share the NUM_BIG arena and
the same late-materialized EncodedLit, whose dt_id is hardcoded to decimal.
Materializing by dt_id alone mislabeled an indexed > i64 integer as
xsd:decimal.

Add FlakeValue::overflow_numeric_datatype_sid, which recovers the datatype
from the decoded value variant, and use it when materializing EncodedLit
in both the query materializer and the API formatter. resolve_datatype_sid_for_value
now delegates to it as well.
@bplatz bplatz requested review from aaj3f and zonotope June 27, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistent JSON-LD rendering of xsd:decimal values: empty @type from arena decode vs plain string from flake path

1 participant