Add WebAssembly code generation - #23584
Conversation
Comparing two floating point values of different types (e.g. `float` and `double`) cast both operands to `creal`, even when neither is complex or imaginary. Pick the common floating point type instead, so a plain float/double comparison doesn't drag in the complex types. Needed for targets without complex support, and a small improvement on its own.
Wire WebAssembly in as a DMD target: OS/arch predefines (WebAssembly, WASI, WASIp1, CRuntime_WASI), Target configuration, wasm argtype classification (argtypes_wasm.d), driver flags (-os=wasm and friends), and the backend cdef/config plumbing. Adds the @wasmImportModule / @wasmExportName attributes and the OPthrow/OPmemgrow-family opers used by later commits. No codegen yet; this is the target-definition layer the rest of the branch builds on.
Changes to the DMC-shared backend needed by wasm codegen: rtlsym.d gains the wasm runtime symbol set and signatures; cgelem.d suppresses rewrites that assume x87/native ABI (e.g. OPscale/fscale, OPremquo fusion) on wasm; dout.d routes object emission to the wasm Obj; plus small cgcs/go/elpicpie/cgcod/ outbuffer adjustments, and gloop.d handling the new wasm opers in the loop optimizer. The shared backend/obj.d gets the wasm hook.
Frontend-to-backend glue branches for wasm: e2ir passes non-POD structs by invisible reference (ISX64REF, matching Posix) and adjusts slice/aggregate arg lowering; s2ir lowers throw/try/catch/finally/scope to the exnref EH shape and reuses the EH_NONE flag dispatch for try/finally; tocsym/toctype/todt handle wasm symbol/type/data emission. This is where D semantics meet the wasm codegen.
The core of the new backend (backend/wasm/): - codgen.d: elem-tree -> wasm instruction selection over a shadow-stack model (address-taken values live in a linear-memory frame via __stack_pointer). Includes frame-elision paths that match ldc -O0, slice-as-two-i32 lowering, and the intrinsic opers (bit ops, memoryGrow, throw). - blocks.d: reconstructs structured block/loop/if nesting from DMD's arbitrary basic-block graph via a forward-target frame pre-pass, with a br_table dispatch fallback for irreducible CFGs the structurer can't nest. - enums.d: wasm opcodes / type / section / reloc / symbol-flag constants. - simd.d: generic __vector -> v128 lowering. - util.d: shared helpers.
backend/wasm/wat.d disassembles the emitted code section to WebAssembly text format, so `-vasm` shows readable output for the wasm target the way it shows x86 assembly elsewhere.
backend/wasm/obj.d writes relocatable .wasm objects with a real symbol table and R_WASM_* relocations (LLVM-compatible flag values and encodings), so multi-TU linking, --gc-sections and archives work like any other target. lib/ gains wasm library support: scanwasm.d reads the wasm symbol table, and the ar container/scan/write flow is shared between ELF and WASM (lib/package.d, lib/wasm.d) rather than duplicated -- which also fixes a latent odd-member padding bug in the ELF writer that llvm-ar/wasm-ld rejected.
link.d drives wasm-ld for the wasm target (--gc-sections, shared-memory flags, vendored -L paths ordered before user switches, honoring an explicit empty -defaultlib=), runs an optional best-effort `wasm-opt -O` under -O, and executes `.wasm` output via wasmtime for -run. mars.d/main.d route the wasm target through this path. The shipped dmd.conf/sc.ini files gain an [Environmentwasm32] section pointing at the vendored wasm32 library directory.
Make druntime build and run on wasm. The archive is built from the shared $(SRCS) list with OS-specific modules self-gating and only genuinely incompatible ones filtered/guarded, rather than a curated wasm list. Startup runs through the shared rt.dmain2; module ctor/dtor ordering reuses rt.minfo; the real shared conservative GC replaces the earlier bump allocator (roots are scannable because the backend spills every cross-call value into linear memory). wasm-specific runtime lives in rt/wasm/ (exnref EH hooks, WASI start, error hooks, extra compiler-rt builtins). `real` is treated as double, with *l libm symbols wrapped to their double counterparts. Also ports coverage/profile runtimes and the WASI clock/argv shims. build.d builds the wasm druntime archive.
Teach the test runner about the wasm target (wasm-validate on compilable tests, wasmtime EXEC wrapper with a per-test timeout and a 2 GiB memory cap, reduced job count scoped to wasm, auto-skip of .sh/dshell tests). Add new runnable wasm tests (SIMD, throw/catch, memoryGrow, ...) and mark target-inherent failures with `DISABLED: wasm` directives instead of a separate known-failures list. Add the wasm CI job.
e2ir reverses the argument list for D-linkage calls, so the wasm backend evaluates OPparam.E2 (the earlier source arguments) first, while the global optimizer assumed E1-first for every operator outside the Ertol table. localize() then sank an assignment past its use: arr[$ - n .. $] read an undefined __dollar temporary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The miscompile only reproduces under -O on wasm (ERTOL(OPparam) is true only for EX_WASM), so the case lives in the local runnable/wasm_codegen.d instead, where the wasm suite runs it with PERMUTE_ARGS: -O. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This is not true anymore. This is true of WASIp1, but WASIp2+ fully supports TCP/UDP sockets. |
|
Please split off first commit to a separate PR, we can merge that straight away. |
Oh cool!
Yes, there's more ways to split this up and make this PR more reviewable. But first I'm going to bring this up for a high-level, holistic review at the upcoming DLF meeting. |
DMD perf check
|
|
Do you want reviews at this point? Even just seeing the changelog entry, I already have questions/comments. |
|
Yes please! I wouldn't review individual lines of code yet, but please share everything that strikes you from observing the description, architecture, or trying it out. |
QuantumSegfault
left a comment
There was a problem hiding this comment.
I haven't checked anything in the backend/codegen yet.
| @@ -0,0 +1,85 @@ | |||
| WebAssembly target support added to DMD | |||
|
|
|||
| DMD now recognizes WebAssembly as a compilation target via the `-mwasm32` and `-os=wasm` flags. | |||
There was a problem hiding this comment.
Shouldn't this be -os=wasi? Wasm is not an OS, only an architecture, where Emscripten and WASI are considered OSs (in the target-triple sense; wasm32-unknown-wasi).
Additionally, there are multiple releases of WASI. WASIp1, p2, and p3.
We support both p1 and p2 in LDC.
| Version identifiers defined when targeting WebAssembly: | ||
| $(UL | ||
| $(LI `WebAssembly`) | ||
| $(LI `WASM32`) |
There was a problem hiding this comment.
Why a new version identifier? D_LP64 handles the 64-bit case when applicable.
There was a problem hiding this comment.
It's supposed to be mirroring the X86 and X86_64 version identifiers, but I'm all for removing it since the predefined version list is already ridiculously long.
| $(LI `WASI`) | ||
| $(LI `CppRuntime_LLVM`) | ||
| $(LI `LittleEndian`) | ||
| ) |
There was a problem hiding this comment.
You also need to specify which version of WASI.
Define WASIp1 or WASIp2.
EDIT: I see you define WASIp1
I don't know if you want to try supporting both (as -os=wasip1 vs -os=wasip2; or just support the latest; technically WASIp3 is, but we only support wasip2 in LDC right now).
| $(IMPDIR)\core\sys\hurd\time.d \ | ||
| $(IMPDIR)\core\sys\hurd\unistd.d \ | ||
| \ | ||
| $(IMPDIR)\core\sys\wasi\posix\time.d \ |
There was a problem hiding this comment.
Ah. This this is an oversight from #23475.
These files are unused. I forgot to remove them when upstreaming.
|
|
||
| void initSections() nothrow @nogc | ||
| { | ||
| auto mbeg = cast(immutable ModuleInfo**)&__start___minfo; |
There was a problem hiding this comment.
How come? This was correct for LDC.
| predef("WASI"); | ||
| predef("WASIp1"); | ||
| predef("Posix"); | ||
| predef("WASI_EMULATED_PROCESS_CLOCKS"); |
There was a problem hiding this comment.
Why is WASI_EMULATED_PROCESS_CLOCKS being used?
There was a problem hiding this comment.
In the short term, to shut up the pragma(msg, "WASI lacks process-associated clocks; to enable" .... I still need to check out what that version is about.
There was a problem hiding this comment.
It's exactly as the message says. In order to enable the functionality you need the version (in C it'd be -D_WASI_EMULATED_PROCESS_CLOCKS to silence the header) AND you need to link against an additional object/library. We shouldn't be defining this by default.
When compiling DRuntime, it's going to be impossible to avoid (and this message isn't the only one), without excluding those files from compilation. But they are harmless.
For Phobos though, you shouldn't get any such messages (I've versioned around importing these files).
The message is there for users importing Posix APIs, so they know it's not really supported, and that they need to link against extra emulation/stubs if they really need it.
| */ | ||
| public int runLINK(bool verbose, ErrorSink eSink) | ||
| { | ||
| if (target.isWasm) |
There was a problem hiding this comment.
In LDC, to simplify linking, we use the appropriate cross-compile Clang from wasi-sdk as link driver when compiling for WASI (but use wasm-ld directly for "bare-metal" Wasm)
There was a problem hiding this comment.
This is also still an open question, which wasm toolchain to ship with the dmd installation. As you might have guessed from my earlier comments, I lean towards bare metal, but am open to suggestions
There was a problem hiding this comment.
In what way? How do you mean "wasm toolchain"
If you are talking about the stuff provided by wasi-sdk, I assumed we wouldn't? Users are expected to have a copy of wasi-sdk installed if they want to use the WASI target.
| # The test suite validates every generated module with wasm-validate, and | ||
| # errors out when it is missing. Distro packages predate `try_table`, so | ||
| # install an upstream release. | ||
| - name: Install wabt |
There was a problem hiding this comment.
Another option would be to use wasm-tools validate
| $ dmd -mwasm32 -os=wasm -run hello.d | ||
| ) | ||
|
|
||
| Version identifiers defined when targeting WebAssembly: |
There was a problem hiding this comment.
This list seems outdated? No mention of the defining WASIp1 or Posix.
And there should be a distinction between targetting WebAssembly as an architecture, and WASI as on OS. A lot of these only apply to WASI, not Wasm in general.
And maybe I missed it, but I don't see where CRuntime_WASI is defined.
There was a problem hiding this comment.
There was a problem hiding this comment.
Yeah, I wanted to give us direct WASI support, but I was advised against it.
So yeah, we are tied to Posix, which in particular means we are tied to wasi-libc (or Emscripten's libc) for DRuntime/Phobos to provide Posix APIs in terms of WASI. Which as we're dependent on SOME libc right now, it doesn't much matter.
| $(LI `WASM32`) | ||
| $(LI `CRuntime_WASI`) | ||
| $(LI `WASI`) | ||
| $(LI `CppRuntime_LLVM`) |
There was a problem hiding this comment.
Hmmm...I forgot about C++
I don't think we specify any CppRuntime in LDC? Or maybe there's a default.
I haven't tested any C++ interop. One problem is that as it stands, you can't link against c++abi and libunwind without causing conflicts...so you either get DRuntime (w/ exceptions) or C++ exceptions, not both.
There was a problem hiding this comment.
I haven't done anything with C++ either
import core.stdc.stdio;
extern(C) void main()
{
printf("Hello, world!\n");
}It doesn't like |
|
Another general note, regarding GC. You have to make sure that ALL potential GC pointers exist on the shadow stack before any potential GC collections Otherwise the stack scanner won't see it (and we can't easily dump the locals and introspect the Wasm value stack...at least not yet). Even with unoptimized code, expression trees are emitted in optimized form without intermediate stores, so you might end up with edge cases where a call in one part of an expression collects a pointer produced by a temporary other part of the expression. I developed a whole pass over LLVM IR for this: https://github.com/ldc-developers/ldc/blob/master/gen/passes/WasmPointersSpill.cpp |
|
Thanks for the review! 2 general comments before answering specific questions. First let me state my knowledge gaps: I've been using wasm32 in FireFox and Chrome using custom Secondly, about druntime: There's a mutual dependency between the compiler and druntime. Codegen implementation is guided by the test suite, the test suite needs druntime, druntime needs codegen. This cycle was broken by incrementally building up a minimal druntime in rt/wasm as scaffolding, and once the backend was mature enough, integrating everything in druntime proper. You are still seeing some remnants of that due for cleanup. |
Fixes for bootstrap compilers, GDC, older hosts, Windows, pre-commit, the D-Scanner style gate, the dlang.org ddoc build, and 32-bit dshell linking. Build the test tools with the MODEL of the test run, and drop `-mscrtlib` from the environment's REQUIRED_ARGS for tests that target a non-Windows OS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This will never work. |
Add a
wasm32backend for DMD.You can see it in action here:
https://dkorpel.github.io/dmd-explorer/
On that page, DMD is compiled with DMD from this branch, and you can enter code and run it in the browser, or inspect the code in the WebAssembly tab.
Wasm features used over the wasm32 MVP:
memcpycall)DMD features compatible with
-mwasm32:-vasmprintsWAT(WebAssembly text format)-runlauncheswasmtimeon the linked.wasmresult if available-profile(profiling)-cov(code coverage)-unittestOriginally Phobos support was out of scope, but now most of Phobos works (thanks to @QuantumSegfault! dlang/phobos#11079) but note that the WebAssembly System Interface (WASI) doesn't cover everything (e.g. no sockets)
Not implemented:
creal)finallyis chained onto the exception being unwound, but collisions spanning several frames at once, and anErrorbypassing anExceptionchain, are not reconstructed.real(realis equal todoublefor-mwasm32)-Omostly disabled, locals are always spilled to the stack. (But you can run dmd's output throughwasm-optto get some optimizations, but it still needs more testing and ldc2 is king for generation of efficient code of course)CI checks that the test suite passes with
OS=wasmby running them inwasmtime.Some tests are disabled because they do something incompatible:
TODO: merge wasm make target in Phobos
LLM usage
Claude Sonnet / Opus / Fable wrote the bulk of the code, while I reviewed, refactored, and redirected the output.
This wasm-backend3 branch contains cleaned up commits, for the messy history see the wasm-backend2 and wasm-backend branches in my fork.
Challenges
Most of the new codegen is located at
dmd/backend/wasm/{codgen,obj,blocks,enums,simd,util}.d.A lot of it is straightforward, translating DMD IR opcodes to equivalent WASM opcodes.
However, it also exposed a lot x86-isms wired in the backend.
WebAssembly has gets validated according to formal Operational semantics. You can't just arbitrarily push some values on the stack and jump to a subroutine: types for a call are checked. This is very useful for testing, but is also exposed how DMD's backend is a bit of a wild west where anything goes.
Built in hooks like
__assert(RTLSYM) are given a fakevoid()type and then called with string / int arguments anyways, so backend types had to be added for those.maintakes up multiple forms (int return can be void, args[] can be ommitted) and still be called interchangably. extern declarations don't actually need to match, as long as their x86 registers work out.In many cases, the backend's function types are missing hidden pointers (for delegates, varargs etc.) or completely missing (for indirect calls to function pointer variables), so that had to be fixed in the glue layer as well.
In DMD, 32-bit integers can be assigned to 64-bit registers and their sign extension (or lack thereof) is just implicit. WASM requires strict type checks and opcodes to convert i32 <> i64, even if practically they are a no-op.
More difficult is control flow: DMD's backend has basic blocks that arbitrarily jump to another, while wasm only has structured control flow: nested
block/loop/ifwith relative-depth branches. There's algorithms to do this (see blocks.d), but some control flow graphs are irreducable (e.g. a goto into a loop body), in which case a less than ideal dispatch loop is generated. Well structured code should not hit this case though.DMD's backend IR for slices is really crazy, combining 2 32-bit components into a 64-bit int and extracting with bit shifts, before rewriting that to 'register pairs' (and wasm is a stack machine with no registers). I would have loved to use the 'multi-value' proposal and make slices uniform between local variables and parameters, as well as uniform with small structs, but in LDC's abi, structs are always passed by pointer, while slices are returned by ptr but slice parameters are separate (len, ptr) arguments. That is nice for Javascript glue functions, but annoying for the backend to handle. Still, I thought being consistent with ldc was important for interopability, so I stuck to that. Might be worth investigating what clang does with small structs and if ldc can easily be configured to do something similar.
I'm still reviewing some parts of the code myself but already opened the PR to 'announce' the work, to fix CI, and because I expect several questions and suggestions already. So ask away!