Skip to content

feat(sql): load local WASM UDFs at startup - #2542

Open
Xgoyeezy wants to merge 1 commit into
getdozer:mainfrom
Xgoyeezy:bounty/1653-wasm-udf
Open

Xgoyeezy wants to merge 1 commit into
getdozer:mainfrom
Xgoyeezy:bounty/1653-wasm-udf

Conversation

@Xgoyeezy

Copy link
Copy Markdown

Closes #1653.

/claim #1653

This adds project-local AssemblyScript/WebAssembly scalar functions behind the optional wasm feature. A configured my_function can be called with SELECT my_function(a, b) FROM numbers. Configuration accepts the issue's udfs.wasm mapping, the existing tagged list, and explicit type: wasm entries.

All configured WASM exports load and validate before connector setup, including unused functions and configurations without SQL. Schema planning and execution share an immutable compiled-module snapshot. Missing or invalid modules, disallowed imports, unsupported signatures, duplicate registrations and built-in name conflicts fail explicitly. Replacing a file after loading cannot silently change a running query.

The Wasmi runtime uses bounded fuel, linear memory, tables and stack depth, permits only AssemblyScript's abort import, and creates a fresh guest instance for every non-NULL call. SQL argument types are checked, return types are inferred, narrowing conversions warn during validation, and integer overflow becomes an error. AssemblyScript Boolean inputs use 0/1; Boolean results have the binary's integer result type and can be cast in SQL.

Validation on macOS arm64 with Rust 1.79.0 and protoc 25.3:

  • WASM enabled: 335 tests passed; 2 existing tests ignored.
  • WASM disabled: 302 tests passed; 2 existing tests ignored.
  • All three YAML forms ran through the actual webhook → SQL → compiled AssemblyScript → sink pipeline, producing 7, 8 and NULL.
  • Six invalid-startup regressions now exit with the expected error before any listener is observed. The previous implementation reached a listener in all six cases.
  • Tests cover streamed inserts/updates/deletes/batches, NULL, filtering, aggregates, conversions, resource limits, concurrent initialization, file replacement/removal and application isolation.
  • Both pinned AssemblyScript fixtures rebuild byte for byte. CLI build, formatting and scoped Clippy pass; pre-existing Clippy warnings remain.

Demo video · Test evidence and source hashes · Configuration and reproduction instructions

The 30-second demo is a clearly labeled replay of actual CLI output, with timing adjusted for readability. The evidence contains raw test results with local filesystem roots redacted.

The primitive scope follows the original maintainer review, which deferred non-primitive types. Managed strings/objects, pointers, WASI and multiple results are outside this change. Wasmi was retained for its explicit execution controls; I welcome maintainer feedback on that choice versus the historically suggested Deno reuse. Full workspace/all-feature CI, other platforms, throughput and total process resource bounds have not been verified.

Implementation and verification used OpenAI Codex assistance. This submission does not presume bounty eligibility, an award or payment.

@CLAassistant

CLAassistant commented Oct 25, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@Xgoyeezy Xgoyeezy mentioned this pull request Oct 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WASM UDF support

2 participants