Skip to content

Add native touch support on Windows with multi-touch and pen input - #3218

Open
Ahmed Ali (thisAAY) wants to merge 2 commits into
JetBrains:jb-mainfrom
thisAAY:ahmed.ali/desktop-windows-touch
Open

Ahmed Ali (thisAAY) wants to merge 2 commits into
JetBrains:jb-mainfrom
thisAAY:ahmed.ali/desktop-windows-touch

Conversation

@thisAAY

Copy link
Copy Markdown

Issue

Compose for Desktop has no native touch support on Windows. AWT doesn't handle touch, and Windows synthesizes mouse events for it, so a touch screen is treated as a mouse with a single pointer: every finger becomes PointerType.Mouse, only one contact is tracked, there is no touch slop, no multi-touch gestures like pinch-to-zoom, and drag-to-scroll is unreliable. Pens lose their type, pressure and eraser state the same way.

This PR adds a full native touch input pipeline for Windows rather than patching the symptoms:

  • Multiple simultaneous fingers, each tracked as its own pointer
  • Real pointer type mapping: fingers arrive as PointerType.Touch, pens as PointerType.Stylus, and an inverted pen or pressed eraser button as PointerType.Eraser
  • Pen pressure reported through ComposeScenePointer.pressure
  • Proper cancellation when the system takes over a gesture

Fixes CMP-1609
Relates to CMP-2209, CMP-6429, CMP-9125, CMP-5945

Changes

  • Added WindowsTouchBridge that subclasses the window procedure and handles WM_POINTER messages before the mouse event synthesis happens. The pointer input API (Windows 8+) is used instead of the legacy WM_TOUCH, which is emulated on top of it and requires a thread-affine registration call.
  • ComposeSceneMediator keeps track of the active contacts and sends each event to ComposeScene with the complete list of pressed pointers, which is what makes multi-finger gestures work.
  • Pointer types are mapped from the native data: PT_TOUCH to Touch, PT_PEN to Stylus with the pressure from GetPointerPenInfo, and PEN_FLAG_INVERTED/PEN_FLAG_ERASER to Eraser.
  • WM_POINTERCAPTURECHANGED and POINTER_FLAG_CANCELED (for example, the system capturing the pointer for an edge gesture) cancel the ongoing pointer input to avoid stuck pointers.
  • Hovering pointers (a pen in range but not touching) are not intercepted and are still reported as mouse moves, so the hover cursor keeps working.
  • EnableMouseInPointer is intentionally not called: it's process-global and would reroute the mouse input AWT relies on. Touch and pen produce WM_POINTER messages without it.
  • The feature is opt-in behind the compose.windows.nativeTouch system property and has no effect on other operating systems.
  • Since AWT provides no touch API, the implementation uses JNA, which adds a net.java.dev.jna:jna-platform:5.19.1 dependency to the desktop target of compose:ui:ui (declared in both version catalogs).
  • Added a Multitouch example to the mpp demo that visualizes the active pointers.

Implementation notes:

  • AWT dispatches input to the first heavyweight child of the top-level window rather than the window itself, so the bridge subclasses the window procedure of the child of the given windowHandle.
  • Pen tilt and rotation are available from GetPointerPenInfo but not mapped, since ComposeScenePointer has no corresponding fields.
  • The bridge is internal and flag-gated, so if this later moves to a lower layer (JBR or Skiko), only the bridge class changes.

Testing

Tested on Windows 11, Recording attached below.

Compose.MPP.demo.2026-07-12.21-11-33.mp4

./gradlew :compose:mpp:demo:runDesktop -Dcompose.windows.nativeTouch=true
Verified:

  • The Components/Multitouch example shows one pointer per finger with types=Touch (Stylus for a pen). Without the flag it shows a single Mouse pointer.
  • Scrolling a LazyColumn with a finger, touch slop, pinch-to-zoom via detectTransformGestures.
  • Pen pressure in the Components/Pen Input example (this makes the demo introduced in Introduce pen-oriented input demo #3079 functional on Windows).

This should be tested by QA.

Release Notes

Features - Desktop

  • Added experimental native touch and pen support on Windows with multi-touch gestures, pointer type mapping (touch, stylus, eraser) and pen pressure. Enable it with the compose.windows.nativeTouch system property

Windows synthesizes mouse events for touch input, so touch screens are
handled as a mouse with a single pointer. Handle WM_POINTER messages
before the mouse event synthesis happens and send the received touch
and pen pointers to ComposeScene, enabling multi-touch gestures and
pen pressure.

The feature is opt-in via the compose.windows.nativeTouch system
property.
Visualizes the active pointers to test multi-touch input on touch
screen devices.
@thisAAY

Copy link
Copy Markdown
Author

Any updates on this?

@igordmn

Copy link
Copy Markdown

Ahmed Ali (@thisAAY), thank you for your PR.

We are currently investigating how we can provide long-term support for this. When we know the answer, we'll return to your PR.

@petmar20

Copy link
Copy Markdown

Also hoping for better touch screen support on desktop Windows and Linux. Hopefully this gets resolved soon. Igor Demin (@igordmn) Any time frame?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants