Skip to content

fix: parse on-file JSON floats exactly - #1605

Merged
thekevinbot merged 4 commits into
mainfrom
bot/1586-float-roundtrip-1104
Nov 4, 2026
Merged

thekevinbot merged 4 commits into
mainfrom
bot/1586-float-roundtrip-1104

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1586

serde_json's default float parser is fast but not correctly rounded, so a JSON float from an on-file hook could be stored one ULP off (123456789.12345679 became 123456789.1234568). This PR enables serde_json's float_roundtrip feature in the workspace Cargo.toml. Every JSON number dirsql reads now parses to the exact double: hook rows, inferred path tables, and worker-function replies.

Red

The test-only commit a6df2e9c failed CI for the reason the test asserts: https://github.com/thekevinscott/dirsql/actions/runs/41879125994/job/126746720725
(left: [Real(123456789.1234568), ...] / right: [Real(123456789.12345679), ...])

Tests:

  • packages/rust/tests/on_file_float.rs: integration test with a declared v REAL table
  • packages/rust/tests/on_file_float_e2e.rs: CLI e2e test of the issue's repro with an inferred path table

Checked by hand with the release binary: the issue's repro prints [{"v":123456789.12345679,"eq":1}]. A run of 20,000 random doubles from Python json.dumps came back with 0 mismatches.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass (impacted subset)
  • TypeScript SDK e2e: pass (impacted subset)
  • Rust core e2e (if applicable): pass
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts)
  • Command(s) run:
    • testing-conventions e2e attest 'uv run python -m pytest tests/e2e/on_file_query_test.py tests/e2e/jsonl_parse_speed_test.py tests/e2e/use_case_transcripts_test.py tests/e2e/query_contract_test.py -x -q' (after maturin develop --release and cargo build --release --features cli)
    • testing-conventions e2e attest 'pnpm exec vitest run --dir tests/e2e query-contract query-params' (after pnpm build)
    • cargo test -p dirsql --features cli --test on_file_float_e2e --test on_file_e2e --test path_table_on_file_e2e --test json_text_e2e --test functions_e2e (plus the on-file, schema-inference and functions integration tests)
  • Result summary: Python 14 passed (this includes the JSONL parse speed test). TypeScript 10 passed. All Rust tests listed passed.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment with the same slug added under <root>/<pkg>/migrations.d/ for each changed package (every subsection but Summary _None._ when nothing breaks)

@thekevinbot
thekevinbot merged commit 3f086b7 into main Nov 4, 2026
89 checks passed
@thekevinbot
thekevinbot deleted the bot/1586-float-roundtrip-1104 branch November 4, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parse on-file JSON floats exactly (enable serde_json float_roundtrip)

1 participant