Skip to content

fix(macos): expose the terminal view as an accessibility text area - #23

Merged
kingb merged 1 commit into
mainfrom
fix/ax-text-area
Oct 1, 2026
Merged

kingb merged 1 commit into
mainfrom
fix/ax-text-area

Conversation

@kingb

@kingb kingb commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Problem

Dictation tools such as Speechify insert text through the macOS Accessibility API: they look for a focused text element they can write to. Ember's terminal view exposed nothing to accessibility, so the focused element was the bare window and Speechify fell back to its "copy text" button. In iTerm2 the same dictation goes straight into the prompt.

Fix

The terminal view now presents itself as a focused AXTextArea, matching iTerm2's terminal view:

  • Read: AXValue (the focused pane's screen text), AXSelectedTextRange (the cursor), AXInsertionPointLineNumber, AXNumberOfCharacters, AXVisibleCharacterRange, AXSelectedText (empty).
  • Write: setting AXValue pastes the text the client inserted (the new value minus what it last read) into the focused pane through the normal paste path, bracketed paste included. Text a client removes is ignored, since the terminal's text is read-only. Setting AXSelectedTextRange is accepted and ignored, because the program running in the terminal owns the caret.

How it's installed:

  • The NSAccessibility methods are added to winit's view class at runtime with class_addMethod. Existing methods are never replaced: if the class already implements any of them (a future winit, say), nothing is installed and a warning is logged.
  • The methods override NSView's inherited defaults, as any accessibility-aware view subclass does.
  • Views that aren't registered (the overlay window) answer as plain, non-element views.
  • Screen text is published only after a client has started reading the view, so there is no cost when no assistive tool is running.
  • macOS only. On Linux the module is an inert stub.

Keyboard input, copy, and OSC 52 code paths are untouched.

Verification

Accessibility dump of the focused element (AXFocusedUIElement of the app):

Before (v0.6.0):

-- focused UI element
  role=AXWindow subrole=AXStandardWindow kids=1
    settable=["AXFullScreen", "AXSections", "AXMain", "AXMinimized", "AXSize"]
  (only child: AXStaticText, nothing settable)

After (this branch):

-- focused UI element
  role=AXTextArea subrole=- kids=0
    attrs=["AXSelectedTextRange", "AXNumberOfCharacters", "AXFrame", "AXParent", "AXChildren", "AXSize", "AXFocused", "AXRole", "AXInsertionPointLineNumber", "AXTopLevelUIElement", "AXHelp", "AXChildrenInNavigationOrder", "AXPosition", "AXWindow", "AXRoleDescription", "AXVisibleCharacterRange", "AXValue", "AXSelectedText"]
    settable=["AXSelectedTextRange", "AXFocused", "AXValue"]

iTerm2, for reference:

-- focused UI element
  role=AXTextArea subrole=- kids=0
    settable=["AXSelectedTextRange", "AXValue"]

AXFocused also reads as settable here. That comes from NSView's inherited default, not from this change.

Write path: an Accessibility API client read AXValue, then set it to that value plus echo ax-insert-ok. The text landed at the zsh prompt without being executed (bracketed paste). The next read showed the prompt line, with AXSelectedTextRange at the cursor.

  • cargo test --workspace, cargo clippy, cargo fmt --check; ember-platform checks for x86_64-unknown-linux-gnu
  • Speechify dictation inserts into a prompt in Ember (tested by hand)
  • macOS built-in Dictation: still does not insert in Ember (it does in iTerm2). Out of scope here. The likely cause is winit dropping insertText: calls made outside an IME composition, the same gap that drops emoji-picker input; it's tracked as a separate fix.

🤖 Generated with Claude Code

Dictation tools insert text through the macOS Accessibility API: they
look for a focused text element they can write to. Ember's terminal
view (winit's WinitView) exposed nothing, so the focused element was
the bare window and tools like Speechify fell back to "copy it
yourself". iTerm2's terminal view is a focused AXTextArea with settable
AXValue and AXSelectedTextRange; Ember's now matches.

The NSAccessibility methods are added to winit's view class at runtime
(class_addMethod), never replacing anything: if the class already
implements any of them, nothing is installed. A client writing AXValue
pastes the text it inserted into the focused pane through the normal
paste path (bracketed-paste aware). Setting AXSelectedTextRange is
accepted and ignored. Screen text is published only once a client has
started reading the view, so there is no cost otherwise. macOS only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kingb
kingb merged commit 7cc193d into main Oct 1, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant