Skip to content

Add floating keyboard support - #2226

Open
Bambooin wants to merge 10 commits into
osfans:developfrom
Bambooin:floating-keyborad
Open

Bambooin wants to merge 10 commits into
osfans:developfrom
Bambooin:floating-keyborad

Conversation

@Bambooin

@Bambooin Bambooin commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Pull request

Issue tracker

Fixes will automatically close the related issues

Fix #2094

Feature

Describe features of this pull request

Add floating feature support, this patch is verified with daily usage.

  • feat(core): add floating keyboard preferences
  • feat(core): add floating keyboard geometry helpers
  • feat(core): decide the floating mode from the orientation
  • fix(core): measure wide screens by their portrait width
  • feat(core): lay out the keyboard for the floating card
  • feat(core): add floating keyboard strings and icons
  • feat(core): show the keyboard as a floating card
  • fix(core): report the floating card as an overlay
  • fix(core): keep the composer inside the floating card
  • feat(core): drag and resize the floating card
  • feat(core): add the floating keyboard toggle

Assisted by LLM.

Code of conduct

Code style

Build pass

  • make debug

Manually test

  • Done

Code Review

  1. No wildcards import
  2. Manual build and test pass
  3. GitHub Action CI pass
  4. At least one contributor review and approve
  5. Merged clean without conflicts
  6. PR will be merged by rebase upstream base

Daily build

Login and download artifact at https://github.com/osfans/trime/actions

Additional Info

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resizing or toggling the floating keyboard can unexpectedly commit the active composition and change ASCII mode because it reruns keyboard-switch initialization instead of preserving live input state.

4 open issues
What changed in this PR

Adds a floating-card mode to Trime’s keyboard, addressing the landscape floating-keyboard request in #2094.

Changes:

  • Enables floating mode by default in landscape, with optional portrait mode and saved geometry.
  • Adds dragging, snapping, resizing, collapsing, and docking, with overlay touch bounds and an embedded composer.
  • Adds geometry and mode tests, localized labels, and control icons.
File Description
app/​src/​test/​java/​com/​osfans/​trime/​ime/​keyboard/​FloatingKeyboardModeTest.kt Tests orientation and preference-based mode selection.
app/​src/​test/​java/​com/​osfans/​trime/​ime/​keyboard/​FloatingKeyboardGeometryTest.kt Tests sizing, positioning, insets, and snapping.
app/​src/​main/​res/​values/​strings.xml Adds English floating-control labels.
app/​src/​main/​res/​values/​input_view_ids.xml Adds handle and composer-host IDs.
app/​src/​main/​res/​values-zh-rTW/​strings.xml Adds Traditional Chinese control labels.
app/​src/​main/​res/​values-zh-rCN/​strings.xml Adds Simplified Chinese control labels.
app/​src/​main/​res/​drawable/​ic_baseline_picture_in_picture_alt_24.xml Adds the floating-mode icon.
app/​src/​main/​res/​drawable/​ic_baseline_open_in_full_24.xml Adds the resize icon.
app/​src/​main/​res/​drawable/​ic_baseline_dock_to_bottom_24.xml Adds the docking icon.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​KeyboardWindow.kt Passes floating geometry into keyboard construction and adjusts layout selection.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​KeyboardPrefs.kt Uses portrait width for wide-screen detection.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​Keyboard.kt Scales floating key height and disables split layouts.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​FloatingKeyboardPrefs.kt Stores floating mode, scale, position, and collapse state.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​FloatingKeyboardMode.kt Defines orientation-driven floating-mode selection.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​keyboard/​FloatingKeyboardGeometry.kt Provides card sizing and positioning helpers.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​core/​TrimeInputMethodService.kt Handles mode requests and reports overlay insets.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​core/​InputView.kt Implements floating-card layout, controls, and gestures.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​composition/​PreeditDelegate.kt Repositions the composer’s touch receiver.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​bar/​ui/​AlwaysUi.kt Adds and styles the floating/docking toggle.
app/​src/​main/​java/​com/​osfans/​trime/​ime/​bar/​InputBarDelegate.kt Connects the toggle to mode changes.
app/​src/​main/​java/​com/​osfans/​trime/​data/​prefs/​AppPrefs.kt Registers floating preferences and includes them in resets.

Review effort: Balanced

if (width != keyboardView.layoutParams.width) {
keyboardWindow.floatingScalePercent =
FloatingKeyboardGeometry.percentFor(baseWidth, width)
keyboardView.updateLayoutParams { this.width = width }
floatingToggleButton.apply {
contentDescription = context.getString(R.string.floating_keyboard_float)
setOnClickListener {
service.requestFloatingMode(!service.isFloatingKeyboardActive)
}

MotionEvent.ACTION_MOVE -> {
keyboardView.translationX =
// The layout depends on the floating flag (a card is never split), and
// the card width differs from the docked width. Rebuild the keyboards
// after the new size has been laid out instead of using the old width.
scheduleKeyboardRefresh()
@x4w2nyhvk8

Copy link
Copy Markdown
Contributor

补充审查反馈(在 Copilot 的 4 条意见之上)

整体实现质量不错:FloatingKeyboardGeometry / FloatingKeyboardMode 抽成纯函数并配了单测,用 translation 移动卡片避免引入 SYSTEM_ALERT_WINDOW,onComputeInsets 用 TOUCHABLE_INSETS_REGION 把卡片当 overlay,composer 重挂到 preeditHost 的处理也考虑得很细。但有两处阻断项建议先修再合并。

阻断 1:几何重建会提交正在输入的 composition 并重置 ascii_mode

InputView.scheduleKeyboardRefresh() → KeyboardWindow.refreshKeyboards() → attachKeyboard(),而 attachKeyboard()(KeyboardWindow.kt:229-237)里有:

if (currentMode != targetMode) {
    service.postRimeJob {
        commitComposition()
        setRuntimeOption("ascii_mode", targetMode)
    }
}

拖动缩放手柄、切换悬浮/停靠、onApplyWindowInsets、onSizeChanged 都会走到这条路径。结果是用户正在输入(composition 未上屏)时拖一下卡片或转屏,编码被强制提交;resetAsciiMode 开启时中英状态还会被改回配置默认值。建议拆出 geometry-only 重建路径(只重新测量尺寸),或给 attachKeyboard 加 preserveInputState 参数,把 ascii 对齐严格限制在真正的键盘切换路径(switchKeyboard / onStartInput)。

阻断 2:resize 的每次 ACTION_MOVE 都触发一次完整键盘重建

handleFloatingResize()(InputView.kt:868)在 ACTION_MOVE 里改卡片宽度,会传导到 KeyboardWindow.onKeyboardViewLayoutChangeListener(KeyboardWindow.kt:123-135),后者无条件、无去重地 v.post { refreshKeyboards() }。InputView 自己的 keyboardRefreshPosted 去重覆盖不到这条路径,且 allowedWidth 两个取值口径会让一次宽度变化收敛两轮。手指移动几十帧就要重建几十次全部按键 View,叠加阻断 1 会反复 commit composition。建议拖动期间只做视觉预览,松手后统一重建一次,并给 KeyboardWindow 的 post 加合并去重。

建议一并处理

  • 键盘 ID 错配:refreshKeyboards() 直接用 currentKeyboardId,绕过 evalKeyboard() 的 landscapeKeyboard 映射(KeyboardWindow.kt:303-308)。横屏停靠 → 悬浮仍保留横屏版 ID 却按竖屏尺寸布局,反向也拿不到配置的横屏布局。建议区分逻辑键盘 ID 与方向变体 ID,模式切换时重新 eval。
  • 按键 popup 位置:KeyView.bounds 只在 KeyView.onLayout() 失效(KeyView.kt:70-79, 269-272),卡片用 translation 移动不触发子 View layout,按键预览和长按弹出键盘会画在旧位置;popup.root 是全屏层,超出卡片矩形后还会落在 touchableRegion 之外。建议移动后统一 updateBounds(),并把可见 popup 区域并入 region。
  • 设置入口缺失:FloatingKeyboardPrefs 注册为独立 provider,但 KeyboardSettingsFragment 只接收 AppPrefs.defaultInstance().keyboard,8 个偏好键在设置里完全不可见、不可修改;shouldFloat() 让横屏无条件默认悬浮,停靠只是 session 级、无法持久化关闭。再叠加 hideInputBar=on 时切换按钮随工具栏隐藏,平板/折叠屏用户既不能在设置里关,也找不到停靠入口。建议在键盘设置页加开关,并把停靠控件放到浮动手柄上(不依赖 input bar)。
  • 无障碍:@SuppressLint("ClickableViewAccessibility") 只压掉了 lint 警告,拖拽/缩放手柄没有 isClickable / performClick() / AccessibilityDelegate,TalkBack 用户无法拖动和调整大小;floatingDragHandle 的 contentDescription 还会吞掉内部收起按钮的无障碍焦点。
  • 折叠态:collapsed 跨会话持久化,重启输入法后只剩一条细条,建议改为会话级(内存)或在 onStartInputView 时自动展开。
  • 文案:「将键盘移入悬浮窗口」/ "Move the keyboard into a floating window" 与实现不符 —— 实际是同一 IME 窗口内的卡片 overlay,不需要悬浮窗权限,建议改为「以悬浮卡片显示键盘」/ "Show keyboard as a floating card"。

小问题

  • widthFor(maxWidth) 有注释和 6 个测试用例,但 applyFloatingGeometry()(InputView.kt:626)调用时从未传 maxWidth,该能力实际未启用;建议平板启用或删掉参数与注释。
  • onConfigurationChanged()(InputView.kt:1050)里的调用是死代码:applyFloatingGeometry() 第一行就是 if (!floatingActive || width <= 0) return。
  • 底部 inset 悬浮/停靠两套口径,悬浮路径忽略了 navbar_background 主题项与 ignoreSystemGestureInsets 偏好,建议统一走 getNavBarBottomInset()。
  • KeyboardPrefs.isWideScreen() 改用短边判定是有价值的修复,但属于全局行为变更,会影响所有 landscapeMode=WIDE 的现有用户(横屏布局/分割键盘可能不再触发),建议单独 PR 并在 changelog 说明。
  • 位置用绝对像素持久化而缩放用百分比,换机、改 density、分屏尺寸变化后会漂移或被 clamp 贴边,建议按可用区域比例存储。
  • AlwaysUi 无条件新增 floatingToggleButton 占用一个 rightWidth,窄屏挤压候选区且无开关隐藏,建议可配置或仅悬浮/横屏时显示。
  • Fixes #2094 会关闭一个包含两个诉求(悬浮键盘 + 全面屏模式/两侧功能键)的 issue,本 PR 只实现前者,建议改为 Part of #2094 并另开 issue 跟进;Fixes # 空行也请清理。

验证建议

现有 508 行单测都是纯函数级,建议补:几何重建不触发 commitComposition()、模式切换后的 evalKeyboard 映射、位置归一化往返。真机验收请覆盖:横屏手机(含分屏/小窗)、平板 WIDE 模式、hideInputBar=on、手势导航与三键导航、TalkBack,以及拖动/缩放过程中正在输入中文、旋转时位置保存、resize 帧率、长按弹窗位置、折叠后重启输入法。

Copilot 的 4 条意见我逐条复核过,均成立;阻断 1/2 是对其第 1 条的调用链补充与放大,其余为额外发现。以上供参考,感谢贡献这个功能。
目前来讲,同问需要系统重构。在重构完成之前,不建议改动这么大。

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

This is a good feature but also a large amount of changes, I think it may take longer time to get a closer look.

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

Thanks for the work on this, I review this with my LLM and have some comments.

The geometry helpers are clean, well-documented and well-tested, and splitting currentKeyboardBaseId from the resolved landscape variant ID (plus resetInputState) is a genuinely thoughtful touch. I'd like to propose three concrete changes before this lands. They are mostly about structure and one data-modelling issue, not about the feature itself.

1. Extract the floating-keyboard runtime into a FloatingKeyboardDelegate

InputView.kt grew from 412 to 1311 lines in this PR, and almost all of it is floating-keyboard runtime logic: gesture handling, inset bookkeeping, position persistence, collapse state, mode resolution. Meanwhile the rest of this codebase already uses a delegate pattern for exactly this kind of thing (PopupDelegate, PreeditDelegate, InputBarDelegate, KeyboardWindow), all bound in InputView.di and resolved with by instance().

FloatingKeyboardGeometry already proves the right idea — but it only took the pure math out. The state machine and touch code are still inlined, so InputView now has two unrelated responsibilities. I'd finish the refactor:

New file: ime/keyboard/FloatingKeyboardDelegate.kt

class FloatingKeyboardDelegate(override val di: DI) : DIAware, InputBroadcastReceiver {
    private val service: TrimeInputMethodService by instance()
    private val scope: ThemeScope by instance()
    private val preedit: PreeditDelegate by instance()
    private val inputBar: InputBarDelegate by instance()
    private val popup: PopupDelegate by instance()
    private val keyboardWindow: KeyboardWindow by instance()

    // Injected by InputView after its view tree exists -- see note below.
    private var host: ViewGroup? = null
    private var keyboardView: View? = null

    fun attach(host: ViewGroup, keyboardView: View) {
        this.host = host
        this.keyboardView = keyboardView
    }

    val isActive: Boolean get() = floatingActive

    // ... everything currently floating-related in InputView
}

Why attach() and not DI injection of keyboardView: keyboardView is a val assigned inside InputView.init, while bindSingleton { ... } is lazily evaluated. Injecting the view would depend on construction order; passing it explicitly after the view tree exists is predictable and states plainly that InputView owns the view.

This also lets you delete two workarounds from this PR. InputBroadcaster is populated with allInstances<InputBroadcastReceiver>(), so every DI-bound delegate that implements the interface already receives broadcasts — that is why PreeditDelegate and friends never call addReceiver explicitly. Once the delegate is a singleton and implements InputBroadcastReceiver:

  • drop InputBroadcastReceiver from InputView's declaration,
  • drop broadcaster.addReceiver(this) from init,
  • move onWindowAttached (expand on any non-keyboard window) into the delegate.

What moves into the delegate (state, not just methods):

  • Views: floatingDragPill, floatingCollapseButton, floatingDockButton, floatingDragHandle, floatingResizeHandle, preeditHost, floatingOutlineProvider
  • State: floatingActive, floatingCollapsed, landscapeDockOverride, lastWindowLandscape, floatingSystemInsets, floatingDragging, keyboardRefreshPosted, keyboardRefreshReevaluatesKeyboard, the dragStart* / resizeStart* fields, resizeChanged
  • Buffers: floatingKeyboardBoundsLocation, floatingPopupBounds
  • Methods: getFloatingKeyboardBounds, refreshColors (the setFloatingControlColors part), applyFloatingGeometry, resetFloatingGeometry, applyFloatingCollapsedState, setFloatingCollapsed, expandFloatingKeyboard, positionFloatingCard, updateDragHandlePosition, updatePreeditParent, updatePreeditTranslation, handleFloatingDrag, handleFloatingResize, finishFloatingResize, snapFloatingCard, saveFloatingPosition, savedFloatingPosition, scheduleKeyboardRefresh, clampFloatingX/Y, floatingDefaultTranslationX/Y, floatingBaseWidth, floatingScalePreference, setFloatingScalePreference, shouldUseFloatingMode, applyEffectiveFloatingMode, updateFloatingMode, setFloatingMode, plus the floating half of onSizeChanged / onApplyWindowInsets.

What stays in InputView: the keyboardView ConstraintLayout construction, bottomPaddingSpace and its margin, dockedBottomInset, and the theme padding sizes — those are about the docked layout.

Thin forwarding in InputView keeps TrimeInputMethodService unchanged (it only knows inputView):

val isFloatingKeyboardActive get() = floatingKeyboard.isActive
fun getFloatingKeyboardBounds(out: Rect) = floatingKeyboard.getFloatingKeyboardBounds(out)
fun setFloatingMode(floating: Boolean) = floatingKeyboard.setFloatingMode(floating)
fun updateFloatingMode() = floatingKeyboard.updateFloatingMode()

Two pitfalls worth calling out while doing this:

  1. Insets are the tangle. floatingSystemInsets, dockedBottomInset and updateBottomPaddingMargin() reference each other. The clean split: the delegate owns floatingSystemInsets (it is the card's safe area), InputView owns dockedBottomInset and bottomPaddingSpace, and navBarBottomInset becomes if (floatingKeyboard.isActive) floatingKeyboard.systemInsets.bottom else dockedBottomInset.
  2. updatePreeditParent genuinely needs both. It moves preedit.ui.root between preeditHost and the input view, so the delegate needs host and preeditHost — another reason for attach(host, …).

Also worth guarding host/keyboardView being null at the top of the public entry points, since broadcast callbacks can arrive before attach().

2. Persist the card position as a fraction, not absolute pixels

saveFloatingPosition() stores keyboardView.left + translationX — an absolute pixel coordinate in the input view's space. That is fine on one device in one window configuration, but the coordinate space changes with split-screen resizing, fold/unfold, and display or font scale changes. clampFloatingX/Y keeps the card on screen, so it does not break — but the card visibly jumps to a different place after those events.

Store a normalized position instead. Two more pure functions in FloatingKeyboardGeometry, testable in the same style as the existing ones:

/** Normalized 0..1 position of [position] within [containerStart]..[containerEnd - viewSize]. */
fun positionFraction(
    position: Int,
    containerStart: Int,
    containerEnd: Int,
    viewSize: Int,
): Float

/** Inverse of [positionFraction]; returns containerStart for a degenerate interval. */
fun positionFromFraction(
    fraction: Float,
    containerStart: Int,
    containerEnd: Int,
    viewSize: Int,
): Float

Then switch the four prefs to floats:

val xPortrait = float(X_PORTRAIT, UNSET_FRACTION)   // 0f..1f, UNSET_FRACTION when never saved

saveFloatingPosition() writes the fraction of the current safe interval; positionFloatingCard() restores by converting the fraction back to an absolute coordinate and deriving the translation from it.

One gotcha when switching to float(): in PreferenceDelegateOwner, int() / long() / bool() / string() call .apply { register() }, but float() does not. Since AppPrefs.syncToDeviceEncryptedStorage() iterates preferenceDelegates, an unregistered float delegate would silently be skipped there. Either add .apply { register() } to float() (consistent with the others, probably the right fix) or register explicitly at the call site — but do not leave it unregistered, because this PR already added floatingKeyboard to that sync list.

Since this feature has not shipped in a release, there is no migration obligation, but daily-build users may already have absolute pixels stored. A one-line guard covers it — anything outside 0f..1f is treated as unset:

val fx = raw.takeIf { it in 0f..1f } ?: UNSET_FRACTION

3. Collapse the "unset position" sentinel into one constant

Right now -1 appears as a literal four times in FloatingKeyboardPrefs and again as UNSET_FLOATING_POSITION in InputView's companion. Promote it:

class FloatingKeyboardPrefs(shared: SharedPreferences) : PreferenceDelegateOwner(shared) {
    companion object {
        /** Placeholder value meaning "no saved position yet". */
        const val UNSET_FRACTION = -1f
        // ...
    }
    val xPortrait = float(X_PORTRAIT, UNSET_FRACTION)
    // ... all four use it
}

If you do 2 and 3 together — which is the least work overall — UNSET_FRACTION covers both purposes and InputView's UNSET_FLOATING_POSITION simply disappears.

Suggested order

  1. Extract the delegate first (mechanical, no behaviour change), so 2 and 3 land in a much smaller file and are trivial to review.
  2. Then the fraction-based persistence plus the sentinel constant in the same commit — they touch the same four prefs.
  3. Consider a unit test for positionFraction / positionFromFraction round-tripping, including the degenerate-interval case; the existing FloatingKeyboardGeometryTest gives you the pattern.

Happy to review the refactor PR, and to sketch an empty FloatingKeyboardDelegate.kt skeleton with the full set of method signatures if that would help.

@WhiredPlanck

Copy link
Copy Markdown
Collaborator

@Bambooin One last thing, you can prompt your LLM to "make sure there is no new bug or feature regression introduced, and reduce redundant comments and meaningless unit tests, keep code high quality and efficient". This is a skill that I usually used to avoid AI slops.

@WhiredPlanck WhiredPlanck left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this and get it further polished in the future.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

悬浮键盘和全面屏模式

4 participants