Types integration: binding shapes + Module.Types adaptor (TypeHints facade) by lukaszsamson · Pull Request #334 · elixir-lsp/elixir_sense

lukaszsamson · 2026-06-12T09:53:04Z

Draft PR to run the CI version matrix (1.16–1.20) against the types integration branch — validating progressive enhancement on Elixir versions below 1.20 (native typing capability-gated; structural engine fallback elsewhere).

Branch contents: ElixirTypes adaptor over Module.Types with capability probing, ExCk reader, TypeHints LSP facade (trust levels, flow-sensitive hints, effective params), three-marker map tails, improper-list modeling, ModuleResolver, apply-parity + compiler-parity + failure-mode suites. Tracked in ELIXIR_SENSE_TYPES_FABLE.md.

🤖 Generated with Claude Code

substitute_spec_vars/2 substituted `@spec ... when` guard variables via Macro.prewalk, which RE-TRAVERSES its own substituted output: a self-referential guard (`when opt: [term :: opt]` — present in elixir-ls's ModuleWithTypespecs fixture) re-injected the substituted var forever, exploding the AST exponentially (profiled: pure-CPU loop, 4GB->6.8GB->OOM; the three elixir-ls locator tests hung >240s and OOM-killed CI runners). Only surfaced with the types branch because the native spec-sig path is the first consumer of guard-var substitution. Replaced with a manual recursive walk tracking an in-progress set of guard vars: a var already on its own expansion path is left as-is, breaking direct and transitive cycles while fully expanding non-cyclic guards. Locator tests: >240s/OOM -> 0.2s. Regression tests assert metadata build COMPLETES for self-referential and mutually-recursive guards (red without the fix). Suites green on 1.18 and 1.20. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

MapSet is dialyzer-opaque; passing it through do_substitute_spec_vars tripped call_without_opaque in both this repo's dialyzer job and elixir-ls's (which analyzes the dep). Plain map as a set, behavior identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Milestone plans (M1-M3, TYPES_*) and the iterative audit/review documents (FABLE/GPT) served the branch's development process; the durable outcomes live in code, tests, and commit messages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Reuse/simplification/efficiency/altitude review applied: - binding.ex: with_descr_backing/3 combinator unifies the duplicated descr-first gating (enabled? + descr_exact? + rescue + :__fallthrough__) used by covers? and combine_intersection; tail_top?/tail_intersectable? defguards complete the map-tail algebra block (merge/intersect/relations in one place); coalesce_union uses prepend+reverse (was O(n^2) append) - elixir_types.ex: version-probe results (expr/pattern API variants, descr_gradual/disjoint/compatible/only_gradual/bitstring/fun probes) read from the persistent_term capabilities cache instead of re-probing function_exported? per call on coercion/apply hot paths; dead coerce_var_type_public/2 overload removed (closedness lives in the map tail); of_match passthrough inlined; apply_infer/strong mirrors share apply_mirror/3; first-wins list-element generalization renamed honestly - type_hints.ex: one cached/2 pdict helper replaces five copy-pasted get-or-compute blocks; metadata_params computes params once per clause; find_mod_fun_info uses ModFunInfo.get_arities (correctly widens the default-arity window across ALL clause variants — was List.first-narrow for multi-clause defaults; regression test added) - exck_reader.ex: identical :miss/:stale arms collapsed Gates: 1992 tests, compile --warnings-as-errors, format, credo --strict. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

elixir_types_m2_test (M2 was the local-inference/pattern-refinement milestone from the deleted development plan), elixir_types_real_test ("against the real Module.Types" — meaningless now that everything is) and elixir_types_integration_test accreted ~1k lines of duplicated coverage across the fix waves. Verified disposition per describe block: - "remote function integration" (12 tests: ExCkReader surface, maybe_remote_call_sig resolution, chunk round-trip) -> new elixir_types_exck_integration_test.exs - "local function inference" (8 MetadataBuilder-level tests) -> elixir_types_local_inference_test.exs - 7 unique pattern/guard/end-to-end tests -> elixir_types_test.exs; one disabled-feature integration test preserved there too - everything else verified as exact/subset duplicates of the topic files (shape conversions, pattern refinement, enabled/disabled integration, merge_shapes, error handling) and dropped Net: ~42 duplicate tests removed, 28 preserved by relocation, three fossil files deleted. 1950 green on 1.20, 1948-suite green on 1.18 (version gates carried over); format/credo/warnings clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 47 out of 49 changed files in this pull request and generated 3 comments.

+defmodule ElixirSense.Core.TypePresentationTest do
+  use ExUnit.Case, async: true
+


+  # `type_spec` here is a quoted type AST (to_entries renders it with
+  # Macro.to_string). TypePresentation renders to text, so parse it back to an
+  # AST; the rendered forms are always valid type expressions.
+  defp rendered_field_type(value) do


  @type variable_doc :: %{
          name: atom(),
-          kind: :variable
+          kind: :variable,
+          # Rendered inferred type (best-effort), or nil when nothing useful.
+          type: String.t() | nil
        }


…334 review) Two High review findings: map field keys can be {:domain, _} (non-atom), but merge_fields used Keyword.get/2 and the combine_intersection map/struct clauses used fields[k] — both raise for non-atom keys, crashing union coalescing and intersection of domain-keyed maps. - Add field_get/3 (List.keyfind-based), the value-side companion to the existing tolerant field_keys/1 and put_fields/2 - merge_fields and all four combine_intersection_custom map/struct clauses use it instead of Keyword.get / fields[k] - Regression tests: union-merge and intersection of domain-keyed maps, and generic-struct ∩ domain-keyed-map (the union-of-keys branch) Medium finding (not is_map_key narrowing) declined with evidence: in a guard `is_map_key(k, non_map)` raises, so `not is_map_key(k, non_map)` fails the guard and the clause matches only maps lacking the key (empirically verified: non-maps take the no-match clause). The orelse handler keeps a per-var fact only when both disjuncts constrain it, so the map fact never leaks unconditionally. Narrowing to a map is therefore sound; added a comment documenting why. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

lukaszsamson · 2026-06-13T06:08:35Z

Addressed in cfb7090.

Both High findings (domain-key crashes) — fixed. Map field keys can be {:domain, _}, and merge_fields (Keyword.get/2) plus the combine_intersection map/struct clauses (fields[k]) raised on non-atom keys. Added a field_get/3 (List.keyfind-based) companion to the existing tolerant field_keys/1/put_fields/2 and routed all six accesses through it. Regression tests cover union-merge, map∩map, and generic-struct∩domain-map.

Medium finding (not is_map_key/2 narrowing) — respectfully declined, with evidence. The premise "the negated guard is true for non-maps too" doesn't hold in Elixir/Erlang guard semantics. In a guard, is_map_key(k, non_map) raises, which makes not is_map_key(k, non_map) fail the guard — so a clause guarded solely by it matches only maps lacking the key. Verified on the running 1.20:

def f(x) when not is_map_key(x, :k), do: :matched
def f(_), do: :no_match
f(5)        #=> :no_match   (non-map)
f("str")    #=> :no_match
f([1,2])    #=> :no_match
f(%{})      #=> :matched    (map without :k)
f(%{k: 1})  #=> :no_match

So narrowing the variable to a map (with the key :not_set) is sound for the standalone clause. For compound guards, the orelse handler keeps a per-variable fact only when both disjuncts constrain it and then unions them, so the map fact can't leak unconditionally through an or. I added a comment documenting this rather than removing the (sound, precision-adding) refinement. Happy to revisit if there's a specific counterexample I'm missing.

Comment-only cleanup across the type test suite (zero code/behavior change): removed milestone/task/audit/wave/GPT process narrative, RED-before-fix and 'previously' rationale, decorative section headers, and fossil describe/test name suffixes ((from m2), (Task N), (P1 soundness), ...). Kept one-line intent comments, compiler-semantics/version citations, and gating notes. 1953 tests unchanged; format clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> EOF

…1.16/1.17 CI) The consolidation that moved the of_match-with-expected-descriptors describe out of m2_test dropped m2's module-level :requires_expected_type_native umbrella, so its setup — which returned the invalid :skip when native typing is unavailable — started running on 1.16/1.17 and raised (ExUnit setup must return :ok | keyword | map). Use the file's standard @describetag :requires_native_types gate (excluded pre-1.18 by test_helper) and return :ok. 1.16: 1939 tests, 0 failures (was 2); 1.20: 1953 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> EOF

lukaszsamson · 2026-07-04T07:30:53Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53f8879dcc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-04T07:39:44Z

+          :skip | {:ok, hint}
+  def type_hint_at(%Context{} = ctx, position, var_name, opts \\ [])
+      when is_atom(var_name) do
+    cached({__MODULE__, ctx.ref, :hint_at, position, var_name}, fn ->


Include render options in hint cache key

When the same Context asks for the same variable/position with different rendering options, this cache returns the first formatted hint because opts are not part of the key. For example, a call with max_length: 10 or widen_literals: false will poison a later call that expects the full/default rendering, even though the public API passes those options through to render_hint/3.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-04T07:39:44Z

+    cond do
+      Enum.any?(parts, &(&1 in [:binary, :bytes, :utf8, :utf16, :utf32, :float])) -> false
+      Enum.any?(parts, &(&1 in [:bitstring, :bits])) -> true
+      true -> Enum.any?(parts, &(is_integer(&1) and rem(&1, 8) != 0))


Honor segment units before marking binaries as bitstrings

For binary segments with a unit multiplier, this tests the raw size literal instead of the effective bit size. <<1::size(1)-unit(8)>> is an 8-bit binary (is_binary/1 is true), but the flattened parts contain 1, so this path downgrades the whole <<>> expression to bitstring() and produces wrong type hints for byte-aligned size(...)-unit(...) segments.

Useful? React with 👍 / 👎.

…pes delegates Adds descr_dynamic/0, descr_dynamic?/1, descr_union/2, descr_intersection/2, descr_empty?/1, descr_subtype?/2 to ElixirTypes and rewrites the six leaked Module.Types.Descr call sites (binding.ex, type_inference.ex, clauses.ex, state.ex) to use them. ElixirTypes is now the only module coupled to the private compiler internals, keeping a future public-API switch (or internals drift) a single-file change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

@doc

…d truth Adds stream_data (dev/test only) and binding_descr_property_test.exs, which validates Binding's custom (approximate) set operations against Module.Types.Descr as the oracle: - I1 no false disjointness: combine_intersection == :none only when the ground-truth intersection is empty - I2 a non-:none exact intersection result contains the ground truth - U1 an exact union result covers both operands - C1 covers?(a, b) implies subtype?(b, a) in ground truth - T1 totality: no op raises on the full grammar (domain keys, optional fields, all map tail markers, structs, improper lists) Semantic properties restrict operands to descr-exact shapes (lossless coercion); the totality property runs on the broadest generator. The harness forces :use_elixir_types off so the custom algebra (the production path on Elixir < 1.18 and for non-exact shapes) is what gets exercised, with 1.20's Descr as oracle. Gated :requires_native_types; PROP_MAX_RUNS tunes depth. Exposes @doc false __combine_intersection__/__normalize_union__/__covers__?/ __descr_exact__? hooks on Binding. Also adds TYPES_ROADMAP.md capturing the dual-model assessment, the two pre-merge items (this and the Descr funnel), and the post-merge plan (descr-as-carrier, descr_exact? growth, presentation-parity de-emphasis, deletion map). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

Negative results ({:error, :beam_not_found} / :not_found / decode failures) were cached for the full 5-minute TTL. In an editor session a project module is often queried moments before it is (re)compiled, so a stale negative entry suppressed ExCk-backed types for minutes after the BEAM appeared. Failures now expire after 1 second (configurable via :exck_negative_cache_ttl); successes keep the long :exck_cache_ttl. A stale positive is at worst slightly outdated signatures, while a stale negative hides the feature entirely — hence the asymmetry rather than not caching failures at all (repeated code-path probes for chunk-less modules are the common case and worth caching). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

…ract Adds a boundary test asserting that no module outside ElixirSense.Core.ElixirTypes calls Module.Types.* — checked against compiled BEAM import tables (actual remote-call targets), so comments/docs don't count and the funnel refactor can't silently regress. Test-support fixtures (e.g. DescrCompat) are exempt via source-path filtering. TYPES_ROADMAP.md gains the merge-time architecture contract (native descrs authoritative; ElixirTypes the only native adaptor; TypeHints the only LSP-facing API; shapes an editor approximation; lossy translation must degrade to weaker hints) and two new post-merge items: lossy-case property tests and behavior-preserving file splits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

…doc types) - type_inference: honor `unit` when deciding sub-byte binary segments — the effective bit width is size * unit, so <<1::size(1)-unit(8)>> is a byte-aligned binary, not a bitstring. Also handles the 2*4 size*unit shorthand. Regression tests added. - type_hints: include (sorted) render opts in the type_hint_at cache key so a call with e.g. max_length does not poison later calls expecting the default rendering. Regression test added. - type_presentation_test: async: false — it mutates the global :use_elixir_types app env and could race concurrent modules. - hover docs: variable/attribute/keyword doc typespecs said name: atom() but builders produce strings; keyword_doc kind corrected :attribute -> :keyword. - completion_engine: comment no longer claims rendered type forms are always parseable (open-map markers are not; nil fallback is intentional). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

lukaszsamson · 2026-07-04T12:01:17Z

Review comments addressed in 4303494:

Binary segment unit (codex P2): valid — <<1::size(1)-unit(8)>> was classified sub-byte from the raw size literal. The effective bit width is now size * unit (with the 2*4 size*unit shorthand handled); non-literal sizes stay conservative. Regression tests added.
Hint cache key missing opts (codex P2): valid — type_hint_at/4 now includes the (sorted) render opts in the request-scoped cache key; added a regression test that a max_length call doesn't poison the default-rendering entry.
type_presentation_test async env mutation (Copilot): valid — now async: false (the only async module mutating :use_elixir_types).
Hover doc typespecs (Copilot): valid — variable_doc/attribute_doc/keyword_doc name corrected to String.t() (builders use Atom.to_string/1); keyword_doc kind fixed :attribute → :keyword.
completion_engine comment (Copilot): valid — comment now states the parse-back is best-effort with a nil fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

Master replaced the vendored tokenizer/parser with the toxic2 dep and ported the completion engine wholesale; this merge reconciles that with the type integration: - completion_engine: keep the Binding-threaded expand_dot_path signature (no cursor_position param); the attribute-alias clause resolves through expand_with_binding. Struct/map field completions adopt master's call?/summary/metadata enrichment with our inferred-type fallback: type_spec = map_field_spec(...) || rendered_field_type(value), where the fallback renders text and suppresses uninformative term()/none()/dynamic(). - .dialyzer_ignore: union of both sides (completion_engine pattern_match entry slotted into the bucket list). - mix.lock: union (toxic2 + stream_data). - master's ported completion/suggestion tests updated where the enriched type_spec fallback now yields inferred field types instead of nil, and to exclude StreamData (new test-only dep) from load-path module assertions. Gates: 1.20 full suite 2289 passed; 1.16 full suite 0 failures; format, credo --strict, dialyzer clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

PR elixir-lsp/elixir_sense#334 merged; 9788101d is the merge commit on master (verified our previous pin c0991f38 is its ancestor). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

lukaszsamson added 30 commits June 10, 2026 15:21

Plan and m1

f35b379

M1 v2

ed378af

m1

ad52346

m1 done

53bfa85

unwrap dynamic

8bd2f1d

m2 draft

e5ef972

m2 refinement

6c801ab

m2 1, 2, 3

2d5232c

exck reader

74a308f

phase 4

1388017

M 1.5

5f985e8

wip

a155a61

m2 progres

3818f96

wire local signatures into binding

7bf295a

docs

bfc1bcd

thread env

13bcf2b

more refinemnet

bc28969

format

1cd2bef

cleanup

6561a3d

tests

dda6fc2

docs

034e396

m3

dc2c443

audit

622a872

test review

b59d045

update gitignore

8765ed7

wip

db18602

some progres

b0a8e8b

tests added

814bc2d

fix a few basic types

1c1f50b

tests

be5570b

lukaszsamson and others added 2 commits June 12, 2026 15:53

lukaszsamson marked this pull request as ready for review June 12, 2026 20:47

lukaszsamson requested a review from Copilot June 12, 2026 20:47

Copilot AI reviewed Jun 12, 2026

lukaszsamson and others added 2 commits June 12, 2026 23:27

lukaszsamson marked this pull request as draft June 12, 2026 22:05

lukaszsamson and others added 2 commits June 13, 2026 00:18

trim comments

c6f3325

lukaszsamson requested a review from Copilot June 12, 2026 22:45

Copilot started reviewing on behalf of lukaszsamson June 12, 2026 22:46 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

lukaszsamson and others added 2 commits June 13, 2026 08:56

chatgpt-codex-connector Bot reviewed Jul 4, 2026

View reviewed changes

lukaszsamson and others added 5 commits July 4, 2026 10:11

lukaszsamson and others added 2 commits July 4, 2026 14:09

ci: retrigger (no workflow run dispatched for 4303494)

d700303

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvwMWQayfdxowBDrnt5iYM

lukaszsamson marked this pull request as ready for review July 4, 2026 14:22

lukaszsamson merged commit 9788101 into master Jul 4, 2026
18 checks passed

Navigation Menu

Uh oh!

Types integration: binding shapes + Module.Types adaptor (TypeHints facade) - #334

Types integration: binding shapes + Module.Types adaptor (TypeHints facade)#334
lukaszsamson merged 125 commits into
masterfrom
claude/trusting-wu-d1f603

lukaszsamson commented Jun 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

lukaszsamson commented Jun 13, 2026

Uh oh!

lukaszsamson commented Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 4, 2026

Uh oh!

lukaszsamson commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		defmodule ElixirSense.Core.TypePresentationTest do
		use ExUnit.Case, async: true

Uh oh!

Conversation

lukaszsamson commented Jun 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

lukaszsamson commented Jun 13, 2026

Uh oh!

lukaszsamson commented Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 4, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 4, 2026

Choose a reason for hiding this comment

Uh oh!

lukaszsamson commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants