diff --git a/CLAUDE.md b/CLAUDE.md index 5e092d628..a11667b5c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,6 +30,7 @@ The Superpowers plugin is disabled for this repository. Do not invoke or follow - **Daemon**:[src/apps/daemon/cli.cpp](src/apps/daemon/cli.cpp) → `worker.cpp::run_worker`(已近 1000 行上限,装配往外拆),写 `/run/` 运行文件,服务 [src/apps/web/](src/apps/web) 路由与 WebSocket。 - **Headless**(`acecode -p ""`):`command_dispatch.cpp::dispatch_non_tui_command` → [headless_runner.cpp](src/apps/headless/headless_runner.cpp),无 run/ 文件、无心跳、无 Crow,会话正常落盘可 resume。`headless::active()` 两个消费点:权限门(默认自动拒绝并给模型解释,`--yolo` 全放行)与 AskUserQuestion 自动应答。`-c` 接最近会话,`--resume ` 在 print 模式 id 必填,`--session-id` 自定 id,`--output-format json` 成败都出单个 result 对象;未传模型走默认模型、权限固定 default、`--max-turns` 默认无限;`--disable-tools` / `--enable-skills` / `--enable-mcp` 与 `--list-tools` / `--list-skills` / `--list-mcp`,未知名称 exit 64。归档:Current Runtime Surfaces。 - **Desktop**:[src/apps/desktop/](src/apps/desktop),每个 workspace 一个 daemon 子进程(独立端口 / token / Job Object),webview 经同进程 bridge 调用,daemon 内部零改动。见 [docs/desktop-shell/multi-workspace.md](docs/desktop-shell/multi-workspace.md)。 +- **Android**:[android/](android/README.md),APK 内带 Ubuntu 小系统,用 Termux 的 proot 跑原版 Linux `acecode daemon`,全屏 WebView 打开 Web UI;targetSdk 刻意停在 28(只能侧载);daemon 必须带 `--require-local-token`(安卓所有 App 共用回环地址,本机免 token 不成立),所以 Linux 包要用含这个开关的版本。 - 进程级服务(工具重写加载、审计库、记忆运行时、LSP、代理探测、数据目录告警补记)在 worker.cpp / TUI main.cpp / headless_runner.cpp 各自接线;新增进程级服务要逐个入口核对,别只接 daemon。工具注册在三处启动时各做一次,`/model` 中途切换不重建 ToolExecutor。 ## Agent Loop 与工具 @@ -46,7 +47,7 @@ The Superpowers plugin is disabled for this repository. Do not invoke or follow - **GPT 系 apply_patch**:判定只看模型 id(`model_family.cpp::model_prefers_apply_patch`),不看 provider 名;三个编辑工具始终注册、只裁模型侧定义表(`filter_tool_definitions_for_model`),system prompt 按模型族分支,非 GPT 模型输出逐字节不变(`system_prompt_test.cpp::NonGptModelStateIsByteIdenticalToLegacyPrompt`);`apply_patch` 先全量校验再落盘,权限门按路径集合走,Delete / Move 源也要 `track_file_write_before`;表面只扫 header 行。归档:GPT 系模型适配。 - **具体进度提示(工具前言 v4)**:文案全部由 daemon 生成(`engine/tool_preamble/`:推理加粗标题 > 工具模板 > 场景文案),loading 只说正在做什么、不带参数、不依赖模型配合;开关 `agent_loop.tool_preamble.enabled`(设置 > 常规 > 工作模式);只在 `emit_agent_progress` 一个口子替换。前三版(先说一句话 / 必填参数 / `` 标签)都被否,别做回去;历史标签只剥不用。归档:具体进度提示。 - **提问**:`agent_loop.question_policy` ask / deny / timeout(`question_timeout_seconds` 默认 60);YOLO 不改提问策略;active goal 覆盖为 30 秒超时采纳推荐项。**提问期间的插话**走 `AgentLoop::interject_question`(同回合 soft steer,tool_result 标 interjected,`question_closed.reason=interjected`),**不要**改成 `interrupt_turn`。归档:提问期间的插话。 -- **回复中的三种发送**(对标 Claude 桌面版 next / later / now):Enter = 软插话 `/turn/steer`,先画在对话末尾(`SteeredMessages`,排队状态 `lane: 'steer'`),气泡上「立即发送」走 `/turn/steer/send-now`(服务端把这条未读插话提成优先回合并打断,**不要**改成再调 `/turn/interrupt` 重发,会发两遍)、撤回走 `/turn/steer/withdraw`;回合结束仍未读的插话转回排队卡片;Ctrl+Enter = 排到回合结束(原排队卡片),Ctrl+Alt+Enter = `fork` 带 `whole: true` 复制整个会话后在新会话发送。见 [docs/daemon-api.md](docs/daemon-api.md)。 +- **回复中的三种发送**(对标 Claude 桌面版 next / later / now):Enter = 软插话 `/turn/steer`,先画在对话末尾(`SteeredMessages`,排队状态 `lane: 'steer'`),气泡上「立即发送」走 `/turn/steer/send-now`(服务端把这条未读插话提成优先回合并打断,**不要**改成再调 `/turn/interrupt` 重发,会发两遍)、撤回走 `/turn/steer/withdraw` 且**消息退回输入框不删**(已有内容时放前面换行,`steerWithdrawal.js`);回合结束仍未读的插话转回排队卡片;Ctrl+Enter = 排到回合结束(原排队卡片),Ctrl+Alt+Enter = `fork` 带 `whole: true` 复制整个会话后在新会话发送。见 [docs/daemon-api.md](docs/daemon-api.md)。 - **侧边对话只放只读工具**(`side_chat.cpp` 小工具循环):白名单 `file_read` / `grep` / `glob` / `lsp` 与会话策略取交集,MCP 一律不放,**不要**改成按 `is_read_only` 筛;凡要弹确认的一律拒绝;在 `MtimeTracker::DetachedReadScope` 里执行;Web 侧边多一个 `send_to_main_session`,用户确认之前绝不送达,送达走 `send_side_chat_message`(有可插话回合就 steer,否则普通 send)。归档:侧边对话只放只读工具。 - **后台任务**:会话级登记表(`tool/background_tasks`,AgentLoop 持有)管后台命令与后台子代理,结束时经邮箱通知会话(在跑就模型边界插入,空闲就开新回合);bash 前台超时**转后台不杀**、`run_in_background` / `monitor`、开头长 sleep 拒绝;用户插话 / 「转到后台」/ TUI Ctrl+B 让前台工作让路;通知落盘为 user + `metadata.background_task`,按内部上下文处理(不计回合、不当标题)。全文 [docs/background-tasks.md](docs/background-tasks.md)。 - **子代理 / 蜂群**:`spawn_subagent` / `wait_subagent` 双端注册,子会话是普通 SessionRegistry 会话(父 cwd + 权限模式),不能再派子代理;子会话从会话列表隐藏、始终跟随主会话(无单独归档 / 删除,主会话永久删除才级联);蜂群 `off | star | mesh` 两套工具互斥,邮件进 AgentLoop 邮箱,信封落盘为 user 角色 + `metadata.inter_agent`,子 agent 完成不唤醒空闲父 agent,Web 芯片只在与服务端模式不同时才随消息提交 `swarm_mode`。全文 [docs/subagents.md](docs/subagents.md)。 @@ -131,4 +132,4 @@ The Superpowers plugin is disabled for this repository. Do not invoke or follow ## CI / Release - `package.yml` 出 Linux x64/arm64、Windows x64/arm64、macOS x64/arm64,`v*` tag 自动发版;`release-tag.yml` 手动打注解 tag 并显式 `gh workflow run package.yml`,发布门槛仍是该提交的 test.yml 通过。npm 发布暂停(`if: ${{ false }}`),实现保留。归档:CI / Release。 -- **自研 Windows 安装程序**(`src/apps/installer/` → `acecode-setup`,尚未进 package.yml):安装包由 `installer/windows/make_setup.py` 用 `UpdateResource` 嵌进资源区(别拼在文件尾,签名会坏),只装当前用户目录;`acecode_installer_support` 只许链 desktop_support + base_core;本机实测先设 `ACECODE_SETUP_TEST_ROOT` 与 `USERPROFILE` 进沙盒,否则会接管本机已有的 Inno 安装。见 [docs/windows-installer.md](docs/windows-installer.md)。 +- **自研 Windows 安装程序**(`src/apps/installer/` → `acecode-setup`,package.yml 出 x64 / ARM64):安装包由 `installer/windows/make_setup.py` 用 `UpdateResource` 嵌进资源区(别拼在文件尾,签名会坏),只装当前用户目录;`acecode_installer_support` 只许链 desktop_support + base_core;界面模式在工作线程上装,COM 调用(快捷方式)必须自己进套间,开发机测不出;本机实测先设 `ACECODE_SETUP_TEST_ROOT` 进沙盒,否则会接管本机已有的安装。见 [docs/windows-installer.md](docs/windows-installer.md)。 diff --git a/android/.gitignore b/android/.gitignore new file mode 100644 index 000000000..1b9b503c5 --- /dev/null +++ b/android/.gitignore @@ -0,0 +1,9 @@ +.gradle/ +.kotlin/ +build/ +app/build/ +local.properties +*.keystore +keystore.properties +runtime/out/ +runtime/cache/ diff --git a/android/README.md b/android/README.md new file mode 100644 index 000000000..82dbe4205 --- /dev/null +++ b/android/README.md @@ -0,0 +1,140 @@ +# ACECode 安卓 App + +一个 APK 装上就能用:App 自带一个 Ubuntu 小系统,用 proot 在里面跑原版 Linux 版 `acecode daemon`, +界面是全屏 WebView 打开 daemon 自带的 Web UI。不需要 Root,不需要 Termux,也不需要 NDK。 + +``` +MainActivity(WebView) ──http://127.0.0.1:<随机端口>/?token=…──▶ acecode daemon + ▲ │ 运行在 +RuntimeService(前台服务,常驻通知 + 唤醒锁) ▼ + └── DaemonController ── libproot.so ──▶ Ubuntu 24.04(files/rootfs) + /root ← files/home(bind) +``` + +## 构建 + +需要:Docker(buildx,arm64 走 Docker Desktop 自带的 QEMU)、Python 3、JDK 17+、Android SDK(compileSdk 36)。 + +1. 从包含安卓宿主提示、phone MCP 和 `daemon --require-local-token` 的源码构建对应架构的 Linux 包。先构建 Web 前端，再配置 CMake，使设置入口和触屏布局进入内嵌资源。构建脚本会检查 Linux 可执行文件是否包含手机 MCP 入口和 token 开关。 + 布局与发布页的 `acecode-linux-.tar.gz` 相同。 +2. 生成运行环境(每个 ABI 一次;Ubuntu 小系统按 Dockerfile 内容缓存在 `runtime/cache/`): + + ```bash + python android/runtime/build_runtime.py --abi x86_64 --acecode-tar path/to/acecode-linux-x64.tar.gz + python android/runtime/build_runtime.py --abi arm64-v8a --acecode-tar path/to/acecode-linux-arm64.tar.gz + ``` + +3. 打包: + + ```bash + cd android + ./gradlew assembleArm64Release # 真机 + ./gradlew assembleX64Debug # x86_64 模拟器 + ./gradlew testX64DebugUnitTest # JVM 单测 + ``` + + `android/keystore.properties`(不入库,字段 storeFile / storePassword / keyAlias / keyPassword)存在时 + release 用它签名,否则用 debug 签名。换签名的包不能覆盖安装。 + +## 只改前端时的快速预览(调试包) + +调试包启动 daemon 前会看 App 内 `home/.acecode/dev-web/index.html` 是否存在,存在就带 +`--static-dir=/root/.acecode/dev-web` 用这份前端,不必重编 Linux 包: + +```bash +cd web && pnpm build +# 把 web/dist 打成 tar 推到 /data/local/tmp,再用 run-as 解进 files/home/.acecode/dev-web,然后重启 App +``` + +删掉这个目录就回到内嵌前端。正式包(不可调试)不认这个目录。调试包同时开着 WebView 远程调试, +可以用 `chrome://inspect` 或 adb 转发 `webview_devtools_remote_` 看页面。 + +## 网页里与安卓相关的约定 + +- App 在 WebView 的 UA 末尾加 `ACECodeAndroid/<版本>`,前端用 `web/src/lib/mobileShell.js` 的 `isAndroidApp()` + 识别,隐藏桌面专属项(快捷键、桌面宠物、检查更新、退出、自更新、Agent 浏览器 / Computer Use / iOS 模拟器)。 +- 返回键:外壳先执行 `window.__aceHandleBack()`;还有对话框、设置、侧栏浮层、菜单开着就补发一次 Esc + 关最上层,都关完才把 App 退到后台。窄屏设置在分区内容页时先回分区列表。 +- 粘贴:手指长按输入框或已选中文字时,全局右键菜单让路给系统的复制 / 粘贴工具条 + (自定义菜单的「粘贴」要读剪贴板,WebView 不给网页这个权限)。 +- 密钥输入框旁的「粘贴」按钮:vivo 等手机给密码框弹出禁止粘贴的安全键盘,所以 App 用 + `WebViewCompat.addWebMessageListener` 注入 `aceAndroid` 读剪贴板通道(`ClipboardBridge.kt`), + 只注册给当前 daemon 页面的源、只回主框架,对话里内嵌的可视化小框读不到;前端 + `readTextFromSystemClipboard` 在 App 里优先走它,按钮只在有这个通道时出现。不把密码框改成普通文本框, + 否则输入法会学习、记住密钥。 +- 新增 / 编辑模型弹窗在窄屏铺满全屏,分 Provider / 模型 / 高级 三个页签。 +- 窄屏(宽度 < 600,与安卓「紧凑宽度」一致)的布局规则在 `web/src/styles/globals.css` 里搜 `max-width: 599px`。 + +## 目录 + +| 路径 | 内容 | +|---|---| +| `runtime/pins.json` | Termux 的 proot / libtalloc / libandroid-shmem 版本与 sha256,Ubuntu 镜像名 | +| `runtime/rootfs/Dockerfile` | Ubuntu 小系统里装哪些包 | +| `runtime/build_runtime.py` | 下载并校验 proot、导出小系统、重打 acecode,产物写到 `runtime/out//` | +| `app/src/main/java/dev/acecode/android/` | 解压、启动、前台服务、WebView | + +## 设计取舍 + +- **targetSdk 停在 28**:安卓 10 起 targetSdk ≥ 29 的 App 不能执行自己数据目录里的文件, + 小系统里的程序都在数据目录。代价是只能侧载、上不了 Google Play(Termux、dsh-Go 同样如此)。 +- **proot 伪装成 `lib*.so`**:只有 `nativeLibraryDir` 里的文件一定可执行;安卓只解压 `lib*.so`, + 所以 `libtalloc.so.2` 改名并原地改写 proot 的 DT_NEEDED(`build_runtime.py` 的 `ELF_RENAMES`)。 +- **归档扩展名用 `.tgz`**:AGP 打包时会把 `.gz` 资源解压并去掉扩展名。 +- **用户数据在小系统外**:`files/home` bind 到 `/root`,会话、配置、项目都在这里; + 小系统(`rootfs`)和 acecode 分开记版本(`files/runtime-state.json`),升级 App 时只换变了的那份。 +- **安全**:安卓上所有 App 共用 127.0.0.1,「本机请求免 token」不成立。App 总是带 + `--require-local-token` 启动 daemon(实现在 `src/apps/web/auth.cpp`),token 和端口存在 App 私有的 + SharedPreferences 里,WebView 用 `?token=` 交给前端。明文 HTTP 只对 127.0.0.1 放行 + (`res/xml/network_security_config.xml`)。 +- **硬链接**:安卓禁止普通进程建硬链接,解包时复制;proot 用 `--link2symlink` 模拟运行期的硬链接。 +- **通知权限**:安卓 13+ 上 targetSdk 28 的 App 默认没有通知权限,常驻通知(带「停止」)会被藏起来; + 系统补弹的授权框实测不出现,`requestPermissions` 也无效,所以界面第一次就绪时弹一次对话框跳到通知设置。 +- **构建脚本用 Groovy**:本机 Gradle 9.2.1 的 Kotlin DSL 脚本连空项目都编不过,App 代码仍是 Kotlin。 + +## 已知限制 + +- 没有命令沙盒:bubblewrap 在安卓上用不了,Auto 模式下未知命令会改为弹确认。 +- 安卓 12 起系统会杀掉过多的子进程(所有 App 合计约 32 个),长任务里同时开很多 bash / LSP / MCP 可能被杀; + 需要用户在开发者选项或 adb 里关掉这个限制。 +- 部分厂商的省电策略仍会冻结后台,需要用户把 App 加进电池白名单。 +- 暂不能访问手机共享存储(`/sdcard`),项目放在 App 内的 `/root/projects`。 + +## 验证记录 + +2026-11-07 在 android-35 x86_64 模拟器上验证:首次启动约 9 秒(解压 + 起 daemon)到界面; +别的用户不带 token 访问 API 全部 401;带 token 的对话里 bash、git、python3 正常; +覆盖安装只替换 acecode,小系统、会话和项目保留;通知栏「停止」让 daemon 正常退出,「启动」能重新拉起。 +同日 OnePlus 9 Pro(安卓 13,内核 5.4 aarch64)真机:首装约 8 秒到界面;别的用户不带 token 访问 API 全部 401; +内置终端里 bash、git commit、python3 正常,起 100 个进程约 0.43 秒。 +vivo X Fold(V2170A,安卓 15,折叠屏双显示)真机:USB 安装要在手机上解锁后确认;对话里模型调用(HTTPS)、bash、git、curl 访问外网都正常,一轮约 15 秒;别的用户不带 token 访问 API 全部 401。 +2026-11-08 的窄屏适配已覆盖 300 CSS 像素宽的设置页和模型编辑弹窗。该版已装到 vivo X Fold，确认页面宽度为 300、剪贴板通道与返回键通道已加载。密码框实际粘贴仍待使用者复核。 +未验证:附件上传、文件下载、长时间后台运行。 + +## 手机操控与悬浮宠物 + +在「设置 → 工具 → 手机操控」进入系统无障碍设置，开启「ACECode 手机操控」。在对话中输入“打开手机时钟，切到秒表”。默认权限模式会询问是否允许打开应用和点按控件；切到其他应用后，可在该应用上方直接允许一次或拒绝。关闭服务、锁屏、遇到密码框或受保护的系统界面时，手机工具会返回原因。读屏基于安卓无障碍控件树，屏幕文字可能发送给该会话的模型。 + +「设置 → 外观 → 悬浮宠物」可选择鲸鱼娘、大眼回形针或 Rover。首次使用需允许 ACECode 显示在其他应用上层。离开 ACECode 后宠物出现，拖动可贴边并记住位置，点按返回 ACECode。选择「关闭」可移除悬浮宠物。 + +手机操控沿用工具设置的图标行与状态开关；悬浮宠物使用与桌面版同源的角色预览卡，点选即生效。授权未完成时保留目标角色并显示继续授权入口，失败可重试原操作。原生启动状态页与跨应用审批卡复用默认主题的亮暗配色、描边和主次按钮；原生界面跟随系统亮暗模式，暂不实时同步自定义 Web 主题。 + +手机工具通过 APK 随附的 Python stdio 中继连接原生进程，每次 daemon 启动分配独立的本地 socket，校验调用进程的安卓 UID。普通桌面会话不加载该入口。每次点按都会对照最近的屏幕和控件；界面更新后须重新读屏，操作完成后返回观察结果。 + +模拟器触屏验收覆盖 301、412 和 659 CSS 像素宽布局。301 像素宽时，顶栏与输入工具点击高度达到 44 像素，权限、模型、推理强度和发送按钮互不重叠。实测输入法弹出时可视高度为 365 像素，发送按钮仍完整可见。文件面板在 301 和 412 像素宽时用抽屉打开，系统返回可关闭抽屉或模型选择菜单。16 个设置分区在 301 像素宽时均未产生横向滚动。 + +## 接入前的对照记录 + +2026-11-08 核对当前安卓包时，Android 15 模拟器中 WebView 能识别 `ACECodeAndroid/`。同一页面请求 `GET /api/mcp` 得到空的全局配置，请求 `GET /api/config/computer-use` 得到 `supported: false`。分别在该模拟器和 vivo X Fold 上按包名查询系统无障碍服务，两个已安装的 ACECode 包都未声明此服务。 + +| 能力 | 接入前的 ACECode 安卓包 | dsh-Go 本机引擎 | +| --- | --- | --- | +| 模型知道自己在哪运行 | 系统提示显示 Linux，shell 在 Ubuntu 中运行。安卓标记用于 WebView 界面适配。 | 开启手机操控后，引擎加载 phone MCP 和使用说明，模型可以调用手机工具。 | +| 打开应用、发起系统操作 | 网页可把外链交给安卓，模型没有用于打开手机应用、设置闹钟的内置工具。 | `apps`、`open` 和 `intent` 对接安卓。定闹钟等系统意图可独立于无障碍服务使用。 | +| 读取并操作其他应用 | APK 提供网页剪贴板和文件选择器。模型不能通过这些接口读取其他应用的屏幕或点按控件。 | 用户启用无障碍服务后，`screen` 返回带编号的控件，`tap`、`type`、`scroll` 等操作可返回新的屏幕状态，`handoff` 可交还用户处理。 | + +dsh-Go 在启动本机引擎前配置 stdio MCP，由客体内的中继连接安卓侧服务。手机操控关闭时移除该入口。对照版本为 `4cc476d2`。[引擎接入](https://github.com/ccch1mneyyy/dsh-Go/blob/4cc476d29172745ff00255f1035e5de31737af29/app/android/app/src/main/kotlin/io/github/ccch1mneyyy/dsh_app/phone/PhoneGuest.kt) · [工具定义](https://github.com/ccch1mneyyy/dsh-Go/blob/4cc476d29172745ff00255f1035e5de31737af29/app/android/app/src/main/kotlin/io/github/ccch1mneyyy/dsh_app/phone/PhoneToolCatalog.kt) · [权限与接管](https://github.com/ccch1mneyyy/dsh-Go/blob/4cc476d29172745ff00255f1035e5de31737af29/docs/guide/zh/04-phone-control.md) + +2026-11-08 接入后的模拟器验收使用 Android 15 自带 Google 时钟。真实会话调用 `mcp_phone_open`，拒绝失效的控件编号后重新读屏，随后成功调用 `mcp_phone_tap`。工具回读与系统界面均显示 Stopwatch，计时器停在零；另一次通过主界面发送请求，并在时钟上方点击「允许一次」，会话同样完成。 + +同日 vivo X Fold（V2170A，Android 15）真机验收通过。vivo 的闹钟专用入口要求额外权限；ACECode 通过它识别系统时钟，再打开时钟的普通桌面入口。在默认权限模式下，真实会话用 `clock` 打开时钟并从「闹钟」切到「秒表」，再次读取确认选中状态，计时保持为零。Rover 显示在系统时钟上方，点按后回到 ACECode；300 CSS 像素宽的外屏也验证了文件抽屉和返回。 diff --git a/android/app/build.gradle b/android/app/build.gradle new file mode 100644 index 000000000..d54f52b0f --- /dev/null +++ b/android/app/build.gradle @@ -0,0 +1,116 @@ +plugins { + id 'com.android.application' +} + +// 版本号跟随主仓 vcpkg.json,APK 与 acecode 本体同号。 +def vcpkgJson = rootProject.file('../vcpkg.json').text +def versionMatch = (vcpkgJson =~ /"version-semver"\s*:\s*"([^"]+)"/) +def appVersion = versionMatch.find() ? versionMatch.group(1) : '0.0.0' +def versionParts = (appVersion.tokenize('.').collect { it.isInteger() ? it.toInteger() : 0 } + [0, 0, 0]) +def baseVersionCode = versionParts[0] * 1_000_000 + versionParts[1] * 1_000 + versionParts[2] + +def runtimeOut = rootProject.file('runtime/out') + +def petAssets = tasks.register('preparePetAssets', Sync) { + into layout.buildDirectory.dir('generated/phonePetAssets') + from(rootProject.file('../assets/desktop_pet')) { + include 'whale/agent.js', 'whale/map.png', 'whale/LICENSE', + 'classic/*/agent.js', 'classic/*/map.png', 'classic/MIT-LICENSE.txt', 'classic/sources.json', 'whale/sources.json' + into 'pets' + } +} + +// 发布签名:android/keystore.properties(不入库)存在时使用,否则用 debug 签名。 +def keystoreProps = new Properties() +def keystoreFile = rootProject.file('keystore.properties') +if (keystoreFile.exists()) keystoreFile.withInputStream { keystoreProps.load(it) } + +android { + namespace = 'dev.acecode.android' + compileSdk = 36 + + defaultConfig { + applicationId = 'dev.acecode.android' + minSdk = 26 + // 刻意停在 28:安卓 10 起 targetSdk>=29 的 App 不能执行自己数据目录里的文件, + // 而 Ubuntu 小系统里的程序都在数据目录里。代价是只能侧载、上不了 Google Play。 + targetSdk = 28 + versionName = appVersion + versionCode = baseVersionCode * 10 + } + + signingConfigs { + if (!keystoreProps.isEmpty()) { + release { + storeFile = rootProject.file(keystoreProps.getProperty('storeFile')) + storePassword = keystoreProps.getProperty('storePassword') + keyAlias = keystoreProps.getProperty('keyAlias') + keyPassword = keystoreProps.getProperty('keyPassword') + } + } + } + + flavorDimensions += 'abi' + productFlavors { + arm64 { + dimension = 'abi' + ndk { abiFilters += 'arm64-v8a' } + versionCode = baseVersionCode * 10 + 2 + } + x64 { + dimension = 'abi' + ndk { abiFilters += 'x86_64' } + versionCode = baseVersionCode * 10 + 4 + } + } + + sourceSets { + main.assets.srcDir(layout.buildDirectory.dir('generated/phonePetAssets').get().asFile) + arm64 { + assets.srcDirs += new File(runtimeOut, 'arm64-v8a/assets') + jniLibs.srcDirs += new File(runtimeOut, 'arm64-v8a/jniLibs') + } + x64 { + assets.srcDirs += new File(runtimeOut, 'x86_64/assets') + jniLibs.srcDirs += new File(runtimeOut, 'x86_64/jniLibs') + } + } + + buildTypes { + release { + minifyEnabled = false + signingConfig = signingConfigs.findByName('release') ?: signingConfigs.debug + } + } + + packaging { + // proot 等可执行文件伪装成 lib*.so,必须解压到 nativeLibraryDir 才能执行。 + jniLibs { useLegacyPackaging = true } + } + + androidResources { + // 运行环境归档本身已压缩;不再压缩才能按文件长度显示解压进度。 + // 扩展名用 .tgz:AGP 会把 .gz 资源在打包时解压掉并去掉扩展名。 + noCompress += ['tgz'] + } + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } + + lint { + disable += ['ExpiredTargetSdkVersion', 'OldTargetApi'] + } + + testOptions { + unitTests.returnDefaultValues = true + } +} + +dependencies { + implementation 'androidx.webkit:webkit:1.14.0' + testImplementation 'junit:junit:4.13.2' +} + +tasks.named('preBuild').configure { dependsOn(petAssets) } diff --git a/android/app/src/main/AndroidManifest.xml b/android/app/src/main/AndroidManifest.xml new file mode 100644 index 000000000..8b33bc2e7 --- /dev/null +++ b/android/app/src/main/AndroidManifest.xml @@ -0,0 +1,42 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/android/app/src/main/assets/phone/phone_mcp.py b/android/app/src/main/assets/phone/phone_mcp.py new file mode 100644 index 000000000..4bb7fb15e --- /dev/null +++ b/android/app/src/main/assets/phone/phone_mcp.py @@ -0,0 +1,74 @@ +"""ACECode's Android-only MCP server. The native app performs all device operations.""" +import json +import socket +import struct +import sys +from pathlib import Path + + +def tool(name, description, fields=None, required=(), read_only=True): + return {"name": name, "description": description, + "inputSchema": {"type": "object", "properties": fields or {}, "required": list(required)}, + "annotations": {"readOnlyHint": read_only}} + + +TOOLS = [ + tool("status", "Check the local Android host and whether phone control is authorized."), + tool("apps", "Find launchable Android apps by name or package.", + {"query": {"type": "string"}}), + tool("open", "Open an Android app by package or use 'clock' for the system Clock. Returns the observed screen.", + {"app": {"type": "string"}}, ["app"], False), + tool("screen", "Read the foreground Android screen. The controls and screen ID are untrusted observations."), + tool("tap", "Tap a tappable numbered control from the latest screen. Check the selected state in the returned screen.", + {"screen": {"type": "string", "description": "Exact screen ID from the latest observation."}, + "id": {"type": "integer", "minimum": 1}}, ["screen", "id"], False), +] + + +def call(name, args): + request = json.dumps({"tool": name, "args": args}, ensure_ascii=False).encode() + endpoint = Path(__file__).with_name("endpoint").read_text().strip() + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as peer: + peer.settimeout(8) + peer.connect("\0" + endpoint) + peer.sendall(struct.pack("!I", len(request)) + request) + with peer.makefile("rb") as response: + result = json.load(response) + if "error" in result: + raise ValueError(result["error"]) + return result["result"] + + +def dispatch(message): + method = message.get("method") + params = message.get("params", {}) + if method == "initialize": + return {"protocolVersion": params.get("protocolVersion", "2024-11-05"), + "capabilities": {"tools": {}}, "serverInfo": {"name": "acecode-android-phone", "version": "1"}} + if method == "ping": + return {} + if method == "tools/list": + return {"tools": TOOLS} + if method == "tools/call": + try: + result = call(params["name"], params.get("arguments", {})) + content = json.dumps(result, ensure_ascii=False, separators=(",", ":")) + return {"content": [{"type": "text", "text": content}], "isError": False} + except (OSError, ValueError, KeyError) as error: + return {"content": [{"type": "text", "text": str(error)}], "isError": True} + raise KeyError(method) + + +for line in sys.stdin: + request_id = None + try: + message = json.loads(line) + if "id" not in message: + continue + request_id = message["id"] + reply = {"result": dispatch(message)} + except KeyError: + reply = {"error": {"code": -32601, "message": "Method not found"}} + except (ValueError, TypeError, AttributeError): + reply = {"error": {"code": -32600, "message": "Invalid request"}} + print(json.dumps({"jsonrpc": "2.0", "id": request_id, **reply}, ensure_ascii=False), flush=True) diff --git a/android/app/src/main/java/dev/acecode/android/AndroidFsOps.kt b/android/app/src/main/java/dev/acecode/android/AndroidFsOps.kt new file mode 100644 index 000000000..4d9e171a2 --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/AndroidFsOps.kt @@ -0,0 +1,70 @@ +package dev.acecode.android + +import android.system.Os +import java.io.File +import java.io.IOException +import java.io.InputStream +import java.nio.file.FileVisitResult +import java.nio.file.Files +import java.nio.file.LinkOption +import java.nio.file.Path +import java.nio.file.SimpleFileVisitor +import java.nio.file.StandardCopyOption +import java.nio.file.attribute.BasicFileAttributes + +object AndroidFsOps : FsOps { + override fun mkdirs(dir: File) { + Files.createDirectories(dir.toPath()) + } + + override fun writeFile(file: File, input: InputStream) { + Files.copy(input, file.toPath(), StandardCopyOption.REPLACE_EXISTING) + } + + override fun copyFile(from: File, to: File) { + Files.copy(from.toPath(), to.toPath(), StandardCopyOption.REPLACE_EXISTING, LinkOption.NOFOLLOW_LINKS) + } + + override fun symlink(link: File, target: String) { + Os.symlink(target, link.path) + } + + override fun chmod(path: File, mode: Int) { + Os.chmod(path.path, mode) + } + + override fun deleteIfExists(path: File) { + val p = path.toPath() + if (Files.isDirectory(p, LinkOption.NOFOLLOW_LINKS)) deleteTree(path) else Files.deleteIfExists(p) + } + + override fun canonical(path: File): File = path.canonicalFile + + /** 递归删除,不跟随符号链接(小系统里有指向 / 的绝对链接)。 */ + fun deleteTree(dir: File) { + val root = dir.toPath() + if (!Files.exists(root, LinkOption.NOFOLLOW_LINKS)) return + Files.walkFileTree(root, object : SimpleFileVisitor() { + override fun preVisitDirectory(d: Path, attrs: BasicFileAttributes): FileVisitResult { + // 只读目录里的文件删不掉,先补上写权限 + try { Os.chmod(d.toString(), 0x1c0) } catch (_: Exception) {} + return FileVisitResult.CONTINUE + } + + override fun visitFile(f: Path, attrs: BasicFileAttributes): FileVisitResult { + Files.delete(f) + return FileVisitResult.CONTINUE + } + + override fun visitFileFailed(f: Path, exc: IOException): FileVisitResult { + Files.deleteIfExists(f) + return FileVisitResult.CONTINUE + } + + override fun postVisitDirectory(d: Path, exc: IOException?): FileVisitResult { + Files.delete(d) + return FileVisitResult.CONTINUE + } + }) + } +} diff --git a/android/app/src/main/java/dev/acecode/android/ClipboardBridge.kt b/android/app/src/main/java/dev/acecode/android/ClipboardBridge.kt new file mode 100644 index 000000000..caae7c976 --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/ClipboardBridge.kt @@ -0,0 +1,47 @@ +package dev.acecode.android + +/** + * 网页「粘贴」按钮的读剪贴板通道(前端 web/src/lib/systemClipboard.js)。 + * + * 安卓 WebView 不让网页读剪贴板,密码框又会弹出禁止粘贴的安全键盘,所以由 App 代读。 + * 通道只注册给 ACECode 页面自己的源(MainActivity.installClipboardBridge),并且只认主框架, + * 对话里内嵌的可视化小框(srcdoc,源是 null)拿不到。 + * + * 这里只做报文处理,不碰 Android API,便于在电脑上跑单测。 + */ +object ClipboardBridge { + const val OBJECT_NAME = "aceAndroid" + + private val TYPE = Regex("\"type\"\\s*:\\s*\"([^\"]*)\"") + private val ID = Regex("\"id\"\\s*:\\s*\"([A-Za-z0-9_-]{1,64})\"") + + /** 处理网页发来的请求,返回要回给网页的 JSON;看不懂的请求返回 null(不回包)。 */ + fun reply(request: String?, readText: () -> String?): String? { + if (request == null) return null + if (TYPE.find(request)?.groupValues?.get(1) != "readClipboard") return null + val id = ID.find(request)?.groupValues?.get(1) ?: return null + val text = try { + readText() + } catch (e: Exception) { + return """{"id":"$id","ok":false,"error":${quote(e.message ?: "clipboard read failed")}}""" + } + if (text == null) return """{"id":"$id","ok":false,"error":"clipboard empty"}""" + return """{"id":"$id","ok":true,"text":${quote(text)}}""" + } + + fun quote(value: String): String = buildString(value.length + 2) { + append('"') + for (c in value) { + when { + c == '"' -> append("\\\"") + c == '\\' -> append("\\\\") + c == '\n' -> append("\\n") + c == '\r' -> append("\\r") + c == '\t' -> append("\\t") + c < ' ' || c == '
' || c == '
' -> append("\\u%04x".format(c.code)) + else -> append(c) + } + } + append('"') + } +} diff --git a/android/app/src/main/java/dev/acecode/android/DaemonController.kt b/android/app/src/main/java/dev/acecode/android/DaemonController.kt new file mode 100644 index 000000000..3dd9c9c7a --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/DaemonController.kt @@ -0,0 +1,234 @@ +package dev.acecode.android + +import android.content.Context +import android.net.ConnectivityManager +import android.system.Os +import android.system.OsConstants +import android.util.Log +import java.io.File +import java.io.FileOutputStream +import java.net.HttpURLConnection +import java.net.InetAddress +import java.net.ServerSocket +import java.net.URL +import java.security.SecureRandom +import java.util.TimeZone +import java.util.concurrent.CopyOnWriteArraySet +import java.util.concurrent.TimeUnit +import kotlin.concurrent.thread + +sealed class RuntimeState { + data object Idle : RuntimeState() + data class Installing(val fraction: Float) : RuntimeState() + data object Starting : RuntimeState() + data class Running(val port: Int, val token: String) : RuntimeState() + data class Failed(val message: String) : RuntimeState() + data object Stopped : RuntimeState() + + val busy get() = this is Installing || this is Starting || this is Running +} + +/** 进程级单例:管运行环境安装、proot/daemon 进程的启停与状态广播。 */ +object DaemonController { + private const val TAG = "ACECodeRuntime" + private const val READY_TIMEOUT_MS = 90_000L + private const val LOG_LIMIT = 4L shl 20 + + private val lock = Any() + private val listeners = CopyOnWriteArraySet<(RuntimeState) -> Unit>() + @Volatile private var process: Process? = null + @Volatile private var stopping = false + + @Volatile var state: RuntimeState = RuntimeState.Idle + private set + + fun addListener(listener: (RuntimeState) -> Unit) { + listeners += listener + listener(state) + } + + fun removeListener(listener: (RuntimeState) -> Unit) { + listeners -= listener + } + + private fun publish(next: RuntimeState) { + state = next + for (l in listeners) l(next) + } + + fun start(context: Context) { + val app = context.applicationContext + synchronized(lock) { + if (state.busy) return + stopping = false + publish(RuntimeState.Installing(0f)) + } + thread(name = "acecode-runtime", isDaemon = true) { run(app) } + } + + fun stop(context: Context) { + val app = context.applicationContext + val p = synchronized(lock) { + stopping = true + process + } + if (p == null) { + if (!state.busy) publish(RuntimeState.Stopped) + return + } + thread(name = "acecode-runtime-stop", isDaemon = true) { + // 先让 daemon 自己收尾(proot 会忽略 SIGTERM),超时再连同 proot 一起结束 + readDaemonPid(RuntimePaths(app))?.let { pid -> + try { Os.kill(pid, OsConstants.SIGTERM) } catch (e: Exception) { Log.w(TAG, "SIGTERM $pid", e) } + } + if (!p.waitFor(8, TimeUnit.SECONDS)) p.destroyForcibly() + } + } + + private fun run(context: Context) { + val paths = RuntimePaths(context) + var phone: PhoneBridge? = null + try { + val installer = RuntimeInstaller(context, paths) + installer.ensureInstalled { publish(RuntimeState.Installing(it)) } + if (stopping) { + publish(RuntimeState.Stopped) + return + } + publish(RuntimeState.Starting) + installer.writeNetworkFiles(dnsServers(context)) + phone = PhoneBridge(context, paths) + + val prefs = context.getSharedPreferences("runtime", Context.MODE_PRIVATE) + val token = prefs.getString("token", null) ?: newToken().also { prefs.edit().putString("token", it).apply() } + val port = pickPort(prefs.getInt("port", 0)).also { prefs.edit().putInt("port", it).apply() } + + AndroidFsOps.deleteTree(paths.prootTmp) + paths.prootTmp.mkdirs() + paths.logs.mkdirs() + + val spec = ProotCommand.Spec( + nativeLibDir = paths.nativeLibDir.path, + rootfs = paths.rootfs.path, + home = paths.home.path, + prootTmp = paths.prootTmp.path, + port = port, + token = token, + timeZone = TimeZone.getDefault().id, + staticDir = devWebDir(context, paths), + ) + val builder = ProcessBuilder(ProotCommand.argv(spec)) + .directory(paths.files) + .redirectErrorStream(true) + builder.environment().putAll(ProotCommand.hostEnvironment(spec)) + val p = builder.start() + synchronized(lock) { process = p } + pumpLog(p, paths) + + waitUntilReady(p, port, token) + publish(RuntimeState.Running(port, token)) + + val code = p.waitFor() + synchronized(lock) { process = null } + publish(if (stopping) RuntimeState.Stopped else RuntimeState.Failed(exitMessage(code, paths))) + } catch (e: Exception) { + Log.e(TAG, "runtime failed", e) + process?.destroyForcibly() + synchronized(lock) { process = null } + publish(if (stopping) RuntimeState.Stopped else RuntimeState.Failed(e.message ?: e.toString())) + } finally { + phone?.close() + } + } + + private fun waitUntilReady(p: Process, port: Int, token: String) { + val deadline = System.currentTimeMillis() + READY_TIMEOUT_MS + while (System.currentTimeMillis() < deadline) { + if (stopping) throw IllegalStateException("stopped") + if (!p.isAlive) throw IllegalStateException("ACECode exited during startup (${p.exitValue()})") + if (healthy(port, token)) return + Thread.sleep(300) + } + throw IllegalStateException("ACECode did not become ready in ${READY_TIMEOUT_MS / 1000}s") + } + + fun healthy(port: Int, token: String): Boolean = try { + val conn = URL("http://127.0.0.1:$port/api/health").openConnection() as HttpURLConnection + conn.connectTimeout = 1000 + conn.readTimeout = 2000 + conn.setRequestProperty("X-ACECode-Token", token) + try { conn.responseCode == 200 } finally { conn.disconnect() } + } catch (_: Exception) { + false + } + + private fun pumpLog(p: Process, paths: RuntimePaths) { + val log = paths.daemonLog + if (log.exists()) log.renameTo(File(paths.logs, "daemon.prev.log")) + thread(name = "acecode-runtime-log", isDaemon = true) { + var out = FileOutputStream(log, true) + var written = 0L + try { + val buf = ByteArray(8192) + val input = p.inputStream + while (true) { + val n = input.read(buf) + if (n < 0) break + if (written > LOG_LIMIT) { + out.close() + log.renameTo(File(paths.logs, "daemon.prev.log")) + out = FileOutputStream(log, false) + written = 0 + } + out.write(buf, 0, n) + out.flush() + written += n + } + } catch (_: Exception) { + } finally { + out.close() + } + } + } + + private fun exitMessage(code: Int, paths: RuntimePaths): String { + val tail = try { paths.daemonLog.readLines().takeLast(12).joinToString("\n") } catch (_: Exception) { "" } + return "ACECode exited ($code)" + if (tail.isNotBlank()) "\n\n$tail" else "" + } + + /** 调试包且 home/.acecode/dev-web/index.html 存在时,daemon 改用这份前端(见 android/README.md)。 */ + private fun devWebDir(context: Context, paths: RuntimePaths): String? { + val debuggable = context.applicationInfo.flags and android.content.pm.ApplicationInfo.FLAG_DEBUGGABLE != 0 + if (!debuggable || !File(paths.home, ".acecode/dev-web/index.html").isFile) return null + return "/root/.acecode/dev-web" + } + + private fun readDaemonPid(paths: RuntimePaths): Int? = + try { paths.daemonPidFile.readText().trim().toInt() } catch (_: Exception) { null } + + private fun dnsServers(context: Context): List { + val found = try { + val cm = context.getSystemService(ConnectivityManager::class.java) + cm.getLinkProperties(cm.activeNetwork)?.dnsServers.orEmpty() + .mapNotNull { it.hostAddress?.substringBefore('%') } + } catch (_: Exception) { + emptyList() + } + // glibc 最多用 3 个;设备给的放前面,再补公共 DNS 兜底 + return (found.take(2) + listOf("1.1.1.1", "223.5.5.5", "8.8.8.8")).distinct().take(3) + } + + private fun pickPort(saved: Int): Int { + val loopback = InetAddress.getByName("127.0.0.1") + if (saved in 1024..65535) { + try { ServerSocket(saved, 1, loopback).close(); return saved } catch (_: Exception) {} + } + ServerSocket(0, 1, loopback).use { return it.localPort } + } + + private fun newToken(): String { + val bytes = ByteArray(32) + SecureRandom().nextBytes(bytes) + return bytes.joinToString("") { "%02x".format(it) } + } +} diff --git a/android/app/src/main/java/dev/acecode/android/FloatingPet.kt b/android/app/src/main/java/dev/acecode/android/FloatingPet.kt new file mode 100644 index 000000000..104dcb7a0 --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/FloatingPet.kt @@ -0,0 +1,191 @@ +package dev.acecode.android + +import android.content.Context +import android.content.Intent +import android.content.SharedPreferences +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import android.graphics.BitmapRegionDecoder +import android.graphics.Canvas +import android.graphics.Paint +import android.graphics.PixelFormat +import android.graphics.Rect +import android.graphics.RectF +import android.provider.Settings +import android.util.LruCache +import android.view.Gravity +import android.view.MotionEvent +import android.view.View +import android.view.ViewConfiguration +import android.view.WindowManager +import org.json.JSONArray +import org.json.JSONObject +import kotlin.math.abs +import kotlin.math.roundToInt + +/** Owns the touchable pet window for exactly one foreground-service lifetime. */ +class FloatingPet(private val context: Context) : SharedPreferences.OnSharedPreferenceChangeListener { + private val prefs = context.getSharedPreferences("phone-ui", Context.MODE_PRIVATE) + private val wm = context.getSystemService(WindowManager::class.java) + private val size get() = (68 * context.resources.displayMetrics.density).roundToInt() + private var pet: PetSprite? = null + private var character = "" + private var params: WindowManager.LayoutParams? = null + + init { prefs.registerOnSharedPreferenceChangeListener(this); refresh() } + + override fun onSharedPreferenceChanged(prefs: SharedPreferences, key: String?) { refresh() } + + @Suppress("DEPRECATION") + private fun space(): android.graphics.Point { + val screen = android.graphics.Point() + wm.defaultDisplay.getSize(screen) + return android.graphics.Point((screen.x - size).coerceAtLeast(0), (screen.y - size).coerceAtLeast(0)) + } + + fun refresh() { + val selected = prefs.getString("pet", "off") ?: "off" + if (selected !in KINDS || !Settings.canDrawOverlays(context) || prefs.getBoolean("activityVisible", false)) { + hide() + return + } + if (pet == null || character != selected) { + hide() + val sprite = PetSprite(context, selected) + val area = space() + val layout = WindowManager.LayoutParams(size, size, WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY, + WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE or WindowManager.LayoutParams.FLAG_NOT_TOUCH_MODAL, + PixelFormat.TRANSLUCENT).apply { + gravity = Gravity.TOP or Gravity.LEFT + x = if (prefs.getBoolean("petRight", true)) area.x else 0 + y = (prefs.getFloat("petY", .45f) * area.y).roundToInt() + } + sprite.contentDescription = context.getString(R.string.floating_pet) + sprite.setOnClickListener { + context.startActivity(Intent(context, MainActivity::class.java) + .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP)) + } + enableDrag(sprite, layout) + try { wm.addView(sprite, layout) } catch (_: WindowManager.BadTokenException) { sprite.dispose(); return } + pet = sprite + params = layout + character = selected + } + pet?.state(prefs.getString("petActivity", "idle") ?: "idle") + } + + fun reposition() { + val view = pet ?: return + val layout = params ?: return + val area = space() + layout.x = if (prefs.getBoolean("petRight", true)) area.x else 0 + layout.y = (prefs.getFloat("petY", .45f) * area.y).roundToInt() + wm.updateViewLayout(view, layout) + } + + @Suppress("ClickableViewAccessibility") + private fun enableDrag(view: View, layout: WindowManager.LayoutParams) { + var downX = 0f + var downY = 0f + var originX = 0 + var originY = 0 + var moved = false + val slop = ViewConfiguration.get(context).scaledTouchSlop + view.setOnTouchListener { _, event -> + when (event.actionMasked) { + MotionEvent.ACTION_DOWN -> { + downX = event.rawX; downY = event.rawY + originX = layout.x; originY = layout.y; moved = false + } + MotionEvent.ACTION_MOVE -> { + val dx = event.rawX - downX + val dy = event.rawY - downY + moved = moved || abs(dx) > slop || abs(dy) > slop + if (moved) { + val area = space() + layout.x = (originX + dx.roundToInt()).coerceIn(0, area.x) + layout.y = (originY + dy.roundToInt()).coerceIn(0, area.y) + wm.updateViewLayout(view, layout) + } + } + MotionEvent.ACTION_UP -> if (moved) { + val area = space() + prefs.edit().putBoolean("petRight", layout.x > area.x / 2) + .putFloat("petY", layout.y.toFloat() / area.y.coerceAtLeast(1)).apply() + reposition() + } else view.performClick() + MotionEvent.ACTION_CANCEL -> reposition() + } + true + } + } + + private fun hide() { + pet?.let { wm.removeViewImmediate(it); it.dispose() } + pet = null + params = null + } + + fun close() { prefs.unregisterOnSharedPreferenceChangeListener(this); hide() } + + companion object { val KINDS = setOf("whale", "clippy", "rover") } +} + +/** Render the original ACECode atlas and frame timing without loading a second browser engine. */ +@Suppress("DEPRECATION") +private class PetSprite(context: Context, private val kind: String) : View(context) { + private val folder = if (kind == "whale") "pets/whale" else "pets/classic/$kind" + private val agent = context.assets.open("$folder/agent.js").bufferedReader().use { + val js = it.readText() + JSONObject(js.substring(js.indexOf('{'), js.lastIndexOf('}') + 1)) + } + private val dimensions = agent.getJSONArray("framesize") + private val frameW = dimensions.getInt(0) + private val frameH = dimensions.getInt(1) + private val atlas = context.assets.open("$folder/map.png").use { BitmapRegionDecoder.newInstance(it, false)!! } + private val cache = LruCache(16) + private val paint = Paint(Paint.FILTER_BITMAP_FLAG) + private var frames = JSONArray() + private var index = 0 + private var activity = "" + private val advance = Runnable { index = (index + 1) % frames.length(); invalidate() } + + fun state(value: String) { + if (activity == value) return + activity = value + val actions = agent.getJSONObject("animations") + val names = when (value) { + "working" -> listOf("working", "Processing", "Searching", "Thinking") + "waiting" -> listOf("wait", "GetAttention", "Alert") + "done" -> listOf("celebrate", "Congratulate", "Wave") + else -> emptyList() + } + listOf("idle", "Idle1_1", "Idle", "RestPose") + val action = names.first { actions.has(it) } + frames = actions.getJSONObject(action).getJSONArray("frames") + index = 0 + removeCallbacks(advance) + invalidate() + } + + override fun onDraw(canvas: Canvas) { + val frame = frames.optJSONObject(index) ?: return + val images = frame.optJSONArray("images") + val scale = minOf(width.toFloat() / frameW, height.toFloat() / frameH) + val dx = (width - frameW * scale) / 2 + val dy = (height - frameH * scale) / 2 + val dest = RectF(dx, dy, width - dx, height - dy) + for (i in 0 until (images?.length() ?: 0)) { + val pos = images!!.getJSONArray(i) + val x = pos.getInt(0) + val y = pos.getInt(1) + val key = y * atlas.width + x + val image = cache.get(key) ?: atlas.decodeRegion(Rect(x, y, x + frameW, y + frameH), BitmapFactory.Options()) + ?.also { cache.put(key, it) } ?: continue + canvas.drawBitmap(image, null, dest, paint) + } + removeCallbacks(advance) + postDelayed(advance, frame.optLong("duration", 200).coerceIn(50, 10000)) + } + + fun dispose() { removeCallbacks(advance); cache.evictAll(); atlas.recycle() } +} diff --git a/android/app/src/main/java/dev/acecode/android/MainActivity.kt b/android/app/src/main/java/dev/acecode/android/MainActivity.kt new file mode 100644 index 000000000..af784bfe1 --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/MainActivity.kt @@ -0,0 +1,371 @@ +package dev.acecode.android + +import android.annotation.SuppressLint +import android.app.Activity +import android.app.AlertDialog +import android.app.NotificationManager +import android.content.ActivityNotFoundException +import android.content.ClipboardManager +import android.content.Intent +import android.content.pm.ApplicationInfo +import android.content.res.Configuration +import android.net.Uri +import android.os.Build +import android.os.Bundle +import android.provider.Settings +import android.view.View +import android.webkit.ValueCallback +import android.webkit.WebChromeClient +import android.webkit.WebResourceRequest +import android.webkit.WebView +import android.webkit.WebViewClient +import android.widget.Button +import android.widget.ProgressBar +import android.widget.TextView +import org.json.JSONObject +import androidx.webkit.WebSettingsCompat +import androidx.webkit.WebViewCompat +import androidx.webkit.WebViewFeature + +class MainActivity : Activity() { + + private lateinit var web: WebView + private lateinit var statusPanel: View + private lateinit var progress: ProgressBar + private lateinit var statusText: TextView + private lateinit var actionButton: Button + + /** 当前 WebView 加载的是哪一个 daemon(端口 + token),变了才重新加载。 */ + private var loadedOrigin: String? = null + private var fileCallback: ValueCallback>? = null + private var startedFromFocus = false + private var clipboardBridgeOrigin: String? = null + private val approval = PhoneApproval(this) { id, choice -> + val data = JSONObject().put("requestId", id).put("choice", choice) + web.evaluateJavascript("window.dispatchEvent(new CustomEvent('ace-phone-decision',{detail:$data}))", null) + } + + private val listener: (RuntimeState) -> Unit = { state -> runOnUiThread { render(state) } } + private val accessibilityChanged: () -> Unit = { + approval.serviceChanged() + web.evaluateJavascript("window.dispatchEvent(new Event('ace-phone-resume'))", null) + } + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + setContentView(R.layout.activity_main) + web = findViewById(R.id.web) + statusPanel = findViewById(R.id.status_panel) + progress = findViewById(R.id.progress) + statusText = findViewById(R.id.status_text) + actionButton = findViewById(R.id.action_button) + actionButton.setOnClickListener { RuntimeService.start(this) } + + setupWebView() + PhoneAccessibility.subscribe(accessibilityChanged) + } + + override fun onWindowFocusChanged(hasFocus: Boolean) { + super.onWindowFocusChanged(hasFocus) + if (hasFocus && ::web.isInitialized) web.evaluateJavascript("window.dispatchEvent(new Event('ace-phone-resume'))", null) + if (!hasFocus || startedFromFocus) return + startedFromFocus = true + // 等界面真正显示后再建通知渠道、起服务:targetSdk 28 的 App 在安卓 13+ 上, + // 系统只在「前台可见的界面建渠道」时才弹通知授权框。 + RuntimeService.ensureChannel(this) + if (!DaemonController.state.busy) RuntimeService.start(this) + } + + override fun onStart() { + super.onStart() + approval.foreground = true + getSharedPreferences("phone-ui", MODE_PRIVATE).edit().putBoolean("activityVisible", true).apply() + DaemonController.addListener(listener) + } + + override fun onStop() { + approval.foreground = false + getSharedPreferences("phone-ui", MODE_PRIVATE).edit().putBoolean("activityVisible", false).apply() + DaemonController.removeListener(listener) + super.onStop() + } + + override fun onDestroy() { + PhoneAccessibility.unsubscribe(accessibilityChanged) + approval.update(null) + fileCallback?.onReceiveValue(null) + fileCallback = null + web.destroy() + super.onDestroy() + } + + override fun onConfigurationChanged(newConfig: Configuration) { + super.onConfigurationChanged(newConfig) + theme.applyStyle(R.style.Theme_ACECode, true) + // Refresh only the native status panel; keep the running WebView and bridges intact. + val parent = statusPanel.parent as android.view.ViewGroup + val index = parent.indexOfChild(statusPanel) + parent.removeView(statusPanel) + statusPanel = layoutInflater.inflate(R.layout.status_panel, parent, false) + parent.addView(statusPanel, index) + progress = statusPanel.findViewById(R.id.progress) + statusText = statusPanel.findViewById(R.id.status_text) + actionButton = statusPanel.findViewById(R.id.action_button) + actionButton.setOnClickListener { RuntimeService.start(this) } + parent.setBackgroundColor(getColor(R.color.ace_bg)) + window.statusBarColor = getColor(R.color.ace_bg) + window.navigationBarColor = getColor(R.color.ace_bg) + val lightBars = resources.getBoolean(R.bool.ace_light_bars) + val barFlags = View.SYSTEM_UI_FLAG_LIGHT_STATUS_BAR or + (if (Build.VERSION.SDK_INT >= 26) View.SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR else 0) + window.decorView.systemUiVisibility = (window.decorView.systemUiVisibility and barFlags.inv()) or + (if (lightBars) barFlags else 0) + render(DaemonController.state) + approval.serviceChanged() + applyDarkMode() + } + + /** + * 返回键先交给网页:设置、对话框、侧栏浮层、菜单还开着就关最上面那一层 + * (前端 lib/mobileShell.js 的 __aceHandleBack);都关完了才把 App 退到后台,daemon 继续跑。 + */ + @Deprecated("Activity#onBackPressed is fine for targetSdk 28") + override fun onBackPressed() { + if (web.visibility != View.VISIBLE) { + moveTaskToBack(true) + return + } + web.evaluateJavascript("(window.__aceHandleBack ? window.__aceHandleBack() : false)") { handled -> + if (handled == "true") return@evaluateJavascript + if (web.canGoBack()) web.goBack() else moveTaskToBack(true) + } + } + + private fun render(state: RuntimeState) { + when (state) { + is RuntimeState.Running -> { + val origin = "http://127.0.0.1:${state.port}" + if (loadedOrigin != "$origin#${state.token}") { + loadedOrigin = "$origin#${state.token}" + installClipboardBridge(origin) + // 前端从 ?token= 读出后存进 sessionStorage 并从地址栏去掉 + web.loadUrl("$origin/?token=${state.token}") + } + web.visibility = View.VISIBLE + statusPanel.visibility = View.GONE + askNotificationsOnce() + return + } + is RuntimeState.Installing -> { + showStatus(getString(R.string.status_preparing), null) + progress.isIndeterminate = state.fraction <= 0f + progress.progress = (state.fraction * 1000).toInt() + } + is RuntimeState.Idle, is RuntimeState.Starting -> { + showStatus(getString(R.string.status_starting), null) + progress.isIndeterminate = true + } + is RuntimeState.Failed -> showStatus(state.message, getString(R.string.action_retry)) + is RuntimeState.Stopped -> showStatus(getString(R.string.status_stopped), getString(R.string.action_start)) + } + } + + private fun showStatus(text: String, action: String?) { + web.visibility = View.GONE + statusPanel.visibility = View.VISIBLE + statusText.text = text + progress.visibility = if (action == null) View.VISIBLE else View.GONE + actionButton.visibility = if (action == null) View.GONE else View.VISIBLE + if (action != null) { + actionButton.text = action + loadedOrigin = null + } + } + + @SuppressLint("SetJavaScriptEnabled") + private fun setupWebView() { + if (applicationInfo.flags and ApplicationInfo.FLAG_DEBUGGABLE != 0) WebView.setWebContentsDebuggingEnabled(true) + with(web.settings) { + javaScriptEnabled = true + domStorageEnabled = true + databaseEnabled = true + allowFileAccess = false + allowContentAccess = true + mediaPlaybackRequiresUserGesture = false + setSupportMultipleWindows(false) + userAgentString = "$userAgentString ACECodeAndroid/${BuildInfo.versionName(this@MainActivity)}" + } + applyDarkMode() + web.webViewClient = object : WebViewClient() { + override fun shouldOverrideUrlLoading(view: WebView, request: WebResourceRequest): Boolean { + val url = request.url + // 网页里内嵌框(对话内可视化等)的跳转、about:/blob:/data: 留在 WebView 里, + // 只有主页面要去外部网址(或 mailto / tel)时才交给系统浏览器。 + if (!request.isForMainFrame) return false + val external = when (url.scheme?.lowercase()) { + "http", "https" -> url.host != "127.0.0.1" + "mailto", "tel" -> true + else -> false + } + if (external) openExternal(url) + return external + } + + override fun onRenderProcessGone(view: WebView, detail: android.webkit.RenderProcessGoneDetail): Boolean { + // 渲染进程被系统回收:整页重开,daemon 不受影响 + recreate() + return true + } + } + web.webChromeClient = object : WebChromeClient() { + override fun onShowFileChooser( + view: WebView, + callback: ValueCallback>, + params: FileChooserParams, + ): Boolean { + fileCallback?.onReceiveValue(null) + fileCallback = callback + val intent = params.createIntent().apply { + if (params.mode == FileChooserParams.MODE_OPEN_MULTIPLE) putExtra(Intent.EXTRA_ALLOW_MULTIPLE, true) + } + return try { + startActivityForResult(intent, REQUEST_FILES) + true + } catch (_: ActivityNotFoundException) { + fileCallback = null + false + } + } + } + } + + @Deprecated("startActivityForResult is fine without AndroidX") + override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) { + if (requestCode != REQUEST_FILES) return super.onActivityResult(requestCode, resultCode, data) + val callback = fileCallback ?: return + fileCallback = null + if (resultCode != RESULT_OK || data == null) return callback.onReceiveValue(null) + val clip = data.clipData + val uris = if (clip != null) Array(clip.itemCount) { clip.getItemAt(it).uri } else listOfNotNull(data.data).toTypedArray() + callback.onReceiveValue(uris) + } + + @Suppress("DEPRECATION") + private fun applyDarkMode() { + // targetSdk < 33:由 force dark 决定网页看到的 prefers-color-scheme; + // 只用网页自己的深色主题,不让 WebView 算法反色。 + val night = (resources.configuration.uiMode and Configuration.UI_MODE_NIGHT_MASK) == Configuration.UI_MODE_NIGHT_YES + if (WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK_STRATEGY)) { + WebSettingsCompat.setForceDarkStrategy(web.settings, WebSettingsCompat.DARK_STRATEGY_WEB_THEME_DARKENING_ONLY) + } + if (WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK)) { + WebSettingsCompat.setForceDark( + web.settings, + if (night) WebSettingsCompat.FORCE_DARK_ON else WebSettingsCompat.FORCE_DARK_OFF, + ) + } + } + + /** + * 安卓 13+ 上 targetSdk 28 的 App 默认没有通知权限,常驻通知(带「停止」)会被藏起来, + * 而系统自带的补弹授权框实测不出现。界面第一次就绪时问一次,点「以后」就不再打扰。 + */ + private fun askNotificationsOnce() { + if (Build.VERSION.SDK_INT < 33) return + if (getSystemService(NotificationManager::class.java).areNotificationsEnabled()) return + val prefs = getSharedPreferences("ui", MODE_PRIVATE) + if (prefs.getBoolean("notification_prompt_shown", false)) return + prefs.edit().putBoolean("notification_prompt_shown", true).apply() + AlertDialog.Builder(this) + .setMessage(R.string.notification_prompt) + .setPositiveButton(R.string.action_open_settings) { _, _ -> + try { + startActivity( + Intent(Settings.ACTION_APP_NOTIFICATION_SETTINGS) + .putExtra(Settings.EXTRA_APP_PACKAGE, packageName) + ) + } catch (_: ActivityNotFoundException) { + } + } + .setNegativeButton(R.string.action_later, null) + .show() + } + + /** + * 「粘贴」按钮用的读剪贴板通道(见 ClipboardBridge)。只注册给当前 daemon 页面的源, + * 只回主框架;端口变了就换一个源重新注册。 + */ + private fun installClipboardBridge(origin: String) { + if (!WebViewFeature.isFeatureSupported(WebViewFeature.WEB_MESSAGE_LISTENER)) return + if (clipboardBridgeOrigin == origin) return + if (clipboardBridgeOrigin != null) { + WebViewCompat.removeWebMessageListener(web, ClipboardBridge.OBJECT_NAME) + WebViewCompat.removeWebMessageListener(web, "acePhone") + } + WebViewCompat.addWebMessageListener(web, ClipboardBridge.OBJECT_NAME, setOf(origin)) { _, message, _, isMainFrame, replyProxy -> + if (!isMainFrame) return@addWebMessageListener + ClipboardBridge.reply(message.data, ::readClipboardText)?.let { replyProxy.postMessage(it) } + } + clipboardBridgeOrigin = origin + WebViewCompat.addWebMessageListener(web, "acePhone", setOf(origin)) { _, message, _, main, reply -> + if (!main || message.data.orEmpty().length > 2048) return@addWebMessageListener + val request = runCatching { JSONObject(message.data ?: "") }.getOrNull() ?: return@addWebMessageListener + val result = try { phoneSettings(request) } catch (e: Exception) { + JSONObject().put("error", e.localizedMessage ?: "Android settings unavailable") + } + reply.postMessage(result.put("id", request.optString("id")).toString()) + } + } + + private fun phoneSettings(request: JSONObject): JSONObject { + val prefs = getSharedPreferences("phone-ui", MODE_PRIVATE) + when (request.getString("action")) { + "status" -> Unit + "approval" -> approval.update(request.optJSONObject("request")) + "accessibility" -> { + if (PhoneAccessibility.current != null) PhoneAccessibility.current?.disableSelf() + else startActivity(Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS)) + } + "pet" -> { + val kind = request.getString("kind") + require(kind == "off" || kind in FloatingPet.KINDS) + if (kind != "off" && !Settings.canDrawOverlays(this)) { + startActivity(Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION, Uri.parse("package:$packageName"))) + } + prefs.edit().putString("pet", kind).apply() + } + "activity" -> { + val state = request.getString("state") + require(state in setOf("idle", "working", "waiting", "done")) + prefs.edit().putString("petActivity", state).apply() + } + else -> error("Unknown phone setting") + } + val overlay = Settings.canDrawOverlays(this) + return JSONObject().put("accessibility", PhoneAccessibility.current != null) + .put("pet", if (overlay) prefs.getString("pet", "off") else "off") + .put("overlay", overlay) + } + + private fun readClipboardText(): String? { + val clip = getSystemService(ClipboardManager::class.java)?.primaryClip ?: return null + if (clip.itemCount == 0) return null + return clip.getItemAt(0).coerceToText(this)?.toString() + } + + private fun openExternal(url: Uri) { + try { + startActivity(Intent(Intent.ACTION_VIEW, url).addCategory(Intent.CATEGORY_BROWSABLE)) + } catch (_: ActivityNotFoundException) { + } + } + + companion object { + private const val REQUEST_FILES = 1 + } +} + +object BuildInfo { + fun versionName(activity: Activity): String = + try { activity.packageManager.getPackageInfo(activity.packageName, 0).versionName ?: "" } catch (_: Exception) { "" } +} diff --git a/android/app/src/main/java/dev/acecode/android/PhoneAccessibility.kt b/android/app/src/main/java/dev/acecode/android/PhoneAccessibility.kt new file mode 100644 index 000000000..0aaeaa68a --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/PhoneAccessibility.kt @@ -0,0 +1,102 @@ +package dev.acecode.android + +import android.accessibilityservice.AccessibilityService +import android.app.KeyguardManager +import android.graphics.Rect +import android.view.accessibility.AccessibilityEvent +import android.view.accessibility.AccessibilityNodeInfo +import org.json.JSONArray +import org.json.JSONObject +import java.security.MessageDigest + +/** The OS creates this service only after the user grants accessibility access. */ +class PhoneAccessibility : AccessibilityService() { + companion object { + @Volatile var current: PhoneAccessibility? = null + private set + // System callbacks and activity attachment both run on Android's main thread. + private val listeners = mutableSetOf<() -> Unit>() + fun subscribe(listener: () -> Unit) { listeners.add(listener) } + fun unsubscribe(listener: () -> Unit) { listeners.remove(listener) } + } + + override fun onServiceConnected() { + current = this + listeners.toList().forEach { it() } + } + override fun onAccessibilityEvent(event: AccessibilityEvent?) {} + override fun onInterrupt() {} + override fun onDestroy() { + if (current === this) { + current = null + listeners.toList().forEach { it() } + } + super.onDestroy() + } + + data class Screen(val json: JSONObject, val targets: List) + + fun targetLabel(screen: String?, id: Int): String { + val state = runCatching { observe().json }.getOrNull() + if (state?.optString("screen") == screen && state != null) { + val row = state.optJSONArray("controls")?.optJSONObject(id - 1) + val label = row?.optString("text").orEmpty().ifBlank { row?.optString("label").orEmpty() } + if (label.isNotBlank()) return label.take(50) + } + return "#$id" + } + + fun checkUnlocked() { + check(current === this) { "Phone control authorization was revoked." } + check(!getSystemService(KeyguardManager::class.java).isDeviceLocked) { "Unlock the phone first" } + } + + fun observe(): Screen { + checkUnlocked() + val root = rootInActiveWindow ?: error("No accessible window. Unlock the phone and try again.") + val pkg = root.packageName?.toString().orEmpty() + check(pkg.isNotBlank() && pkg !in setOf(packageName, "com.android.systemui", "com.android.settings") && + !pkg.contains("permissioncontroller") && !pkg.contains("packageinstaller")) { + "This protected window requires user interaction." + } + val rows = JSONArray() + val targets = mutableListOf() + var visited = 0 + fun walk(node: AccessibilityNodeInfo, clickableParent: AccessibilityNodeInfo?, depth: Int) { + if (!node.isVisibleToUser) return + check(++visited <= 2400 && depth <= 60) { "Screen is too large to inspect safely." } + check(!node.isPassword) { "A password field is present. Continue on the phone." } + val target = if (node.isClickable && node.isEnabled) node else clickableParent + val text = node.text?.toString().orEmpty().take(200) + val description = node.contentDescription?.toString().orEmpty().take(200) + if ((node.isClickable || text.isNotBlank() || description.isNotBlank()) && rows.length() < 240) { + val bounds = Rect().also(node::getBoundsInScreen) + val row = JSONObject().put("id", rows.length() + 1) + .put("role", node.className?.toString()?.substringAfterLast('.')) + .put("bounds", bounds.flattenToString()).put("tappable", target != null && node.isEnabled) + if (text.isNotBlank()) row.put("text", text) + if (description.isNotBlank()) row.put("label", description) + if (node.isSelected || node.isChecked) row.put("selected", true) + rows.put(row) + targets += target ?: node + } + for (index in 0 until node.childCount) node.getChild(index)?.let { walk(it, target, depth + 1) } + } + walk(root, null, 0) + val state = JSONObject().put("package", pkg).put("window", root.windowId).put("controls", rows) + val hash = MessageDigest.getInstance("SHA-256").digest(state.toString().toByteArray()) + state.put("screen", android.util.Base64.encodeToString(hash, android.util.Base64.NO_WRAP)) + return Screen(state, targets) + } + + fun tap(screen: String, id: Int) { + val current = observe() + check(screen == current.json.getString("screen")) { "Screen changed. Read the screen again before tapping." } + val row = current.json.getJSONArray("controls").optJSONObject(id - 1) + require(row != null && row.getBoolean("tappable")) { "Choose a tappable control on this screen." } + val node = current.targets[id - 1] + check(node.refresh() && node.isEnabled && node.isVisibleToUser && node.performAction(AccessibilityNodeInfo.ACTION_CLICK)) { + "The control did not accept the tap. Read the screen again." + } + } +} diff --git a/android/app/src/main/java/dev/acecode/android/PhoneApproval.kt b/android/app/src/main/java/dev/acecode/android/PhoneApproval.kt new file mode 100644 index 000000000..e0a540495 --- /dev/null +++ b/android/app/src/main/java/dev/acecode/android/PhoneApproval.kt @@ -0,0 +1,85 @@ +package dev.acecode.android + +import android.graphics.PixelFormat +import android.graphics.Point +import android.os.Build +import android.view.LayoutInflater +import android.view.WindowInsets +import android.view.Gravity +import android.view.View +import android.view.WindowManager +import android.widget.Button +import android.widget.TextView +import org.json.JSONObject + +/** A non-focusing approval leaves the observed app in front while the user answers. */ +class PhoneApproval(private val activity: MainActivity, private val answer: (String, String) -> Unit) { + private var pending: JSONObject? = null + private var window: View? = null + private var manager: WindowManager? = null + private var shown = "" + var foreground = true + set(value) { field = value; refresh() } + + fun update(request: JSONObject?) { pending = request; refresh() } + + fun serviceChanged() { hide(); refresh() } + + private fun refresh() { + val request = pending + val service = PhoneAccessibility.current + val id = request?.optString("requestId").orEmpty() + if (foreground || id.isBlank() || service == null) { hide(); return } + if (shown == id) return + hide() + val wm = service.getSystemService(WindowManager::class.java) + val label = if (request!!.optString("tool") == "mcp_phone_tap") { + val args = request.optJSONObject("args") + val name = service.targetLabel(args?.optString("screen"), args?.optInt("id") ?: 0) + activity.getString(R.string.approve_phone_tap, name) + } else activity.getString(R.string.approve_phone_open, request.optJSONObject("args")?.optString("app").orEmpty().take(80)) + val bar = LayoutInflater.from(activity).inflate(R.layout.phone_approval, null) + bar.findViewById(R.id.approval_label).text = label + fun bindButton(viewId: Int, choice: String) { + bar.findViewById @@ -193,6 +193,20 @@ try { assert.ok(await nav.evaluate(e=>e.scrollWidth>e.clientWidth&&e.scrollLeft>0)); await page.screenshot({path:path.join(output,'breadcrumbs-narrow.png')}); await page.locator('main').click({position:{x:395,y:610}});await menu.waitFor({state:'detached'}); + const more=page.getByRole('button',{name:'文件操作'}); + const fileMenu=page.getByRole('menu',{name:'文件操作'}); + await more.focus();await page.keyboard.press('ArrowDown');await fileMenu.waitFor(); + const anchorBox=await more.boundingBox(),popupBox=await fileMenu.boundingBox(); + assert.ok(Math.abs(anchorBox.x+anchorBox.width-popupBox.x-popupBox.width)<=1); + const beforeScroll=anchorBox.x; + await nav.evaluate(e=>{e.scrollLeft=0;}); + assert.equal((await more.boundingBox()).x,beforeScroll); + assert.equal(await fileMenu.getByRole('menuitem',{name:'打开文件夹'}).count(),0); + await page.screenshot({path:path.join(output,'file-actions-narrow.png')}); + await fileMenu.getByRole('menuitem',{name:'在内置浏览器中打开'}).click(); + assert.match(await page.evaluate(()=>window.openedBrowserUrl),/very-long-file-name\.html$/); + await more.click();await page.evaluate(()=>window.fixture.select('docs/guide.md')); + await fileMenu.waitFor({state:'detached'}); console.log('[pass] long path scroll, narrow popup containment and outside dismissal'); assert.deepEqual(errors,[]); console.log(`Screenshots: ${output}`); diff --git a/web/scripts/test-follow-up-prompts.mjs b/web/scripts/test-follow-up-prompts.mjs index 6306c10a6..6f413d24c 100644 --- a/web/scripts/test-follow-up-prompts.mjs +++ b/web/scripts/test-follow-up-prompts.mjs @@ -220,6 +220,17 @@ try { await editor.waitFor(); assert.equal(fullRequests.length,0,'home never generates a session suggestion'); await page.evaluate(()=>window.fullFixture.render('full-a')); await page.waitForFunction(()=>document.querySelector('[data-slate-placeholder]')?.textContent.includes('full-a')); + const originalPrompt = await ghost.textContent(); + await editor.click(); + for (const text of ['My own draft', ' ']) { + await page.keyboard.type(text); + assert.doesNotMatch(await ghostText(), /full-a/); + await page.keyboard.press('Control+A'); + await page.keyboard.press('Backspace'); + await ghost.filter({hasText: originalPrompt}).waitFor({state:'visible'}); + assert.equal(await ghost.textContent(), originalPrompt); + } + assert.equal(fullRequests.filter(r=>r.sid==='full-a').length,1,'clearing reuses the original recommendation'); await editor.click();await page.keyboard.press('Tab');await settle(); assert.match(await editor.textContent(),/Run the parser tests for full-a/); await page.evaluate(()=>window.fullFixture.render('full-b')); diff --git a/web/scripts/test-preview-presentation.mjs b/web/scripts/test-preview-presentation.mjs index 2401200da..b1172bd0e 100644 --- a/web/scripts/test-preview-presentation.mjs +++ b/web/scripts/test-preview-presentation.mjs @@ -21,6 +21,9 @@ import {sessionWorkbench} from '/src/lib/sessionWorkbench.js'; import {PREVIEW_TAB_TYPES} from '/src/lib/previewTabs.js'; import '/src/styles/globals.css'; window.__ACECODE_DESKTOP_SHELL__=true; +window.openedFiles=[]; +window.aceDesktop_openInExplorer=async path=>{window.openedFiles.push(path);return {ok:true};}; +window.aceDesktop_writeClipboardText=async text=>{window.copiedText=text;return {ok:true};}; const code=Array.from({length:130},(_,i)=>'const line'+i+' = "detail text and long source line '.repeat(i===3?12:1)+'";').join('\n'); const markdown='# Preview heading\n\nEditable Markdown text.\n\n'+String.fromCharCode(96).repeat(3)+'js\nconst example = 42;\n'+String.fromCharCode(96).repeat(3)+'\n\n- List item\n'; const textFor=p=>p.endsWith('.md')?markdown:code; @@ -94,6 +97,21 @@ try { const errors=[];page.on('pageerror',error=>{errors.push(error.message);console.error('[pageerror]',error.message);}); await page.goto(`http://127.0.0.1:${server.httpServer.address().port}/__preview-presentation`); const editor=page.locator('.ace-file-text-editor');await editor.waitFor(); + const fileActions=page.getByRole('button',{name:'文件操作',exact:true}); + const fileMenu=page.getByRole('menu',{name:'文件操作',exact:true}); + async function chooseFileAction(name) { + await fileActions.click(); + await fileMenu.getByRole('menuitem',{name,exact:true}).click(); + } + await chooseFileAction('开启自动换行'); + assert.equal(await editor.getAttribute('wrap'),'soft'); + await chooseFileAction('关闭自动换行'); + assert.equal(await editor.getAttribute('wrap'),'off'); + await chooseFileAction('打开文件夹'); + await editor.click({button:'right'}); + await page.getByRole('menuitem',{name:'打开文件夹',exact:true}).click(); + assert.deepEqual(await page.evaluate(()=>window.openedFiles),Array(2).fill('C:\\fixture\\source.js')); + assert.equal(await page.locator('.ace-side-preview-code > .ace-code-actions').count(),0); const settle=()=>page.evaluate(()=>new Promise(r=>requestAnimationFrame(()=>requestAnimationFrame(r)))); const state=()=>page.evaluate(()=>({prefs:window.fixture.prefs,base:window.fixture.base,active:window.fixture.active})); const press=async()=>{await page.keyboard.press('Control+F11');await settle();}; @@ -151,8 +169,10 @@ try { const mdSize=await markdown.evaluate(e=>parseFloat(getComputedStyle(e).fontSize));await markdown.hover();await page.keyboard.down('Control');await page.mouse.wheel(0,-120);await page.keyboard.up('Control');await settle(); assert.ok(await markdown.evaluate((e,s)=>parseFloat(getComputedStyle(e).fontSize)>s,mdSize));assert.equal(await markdown.textContent(),mdBefore); await press();await page.screenshot({path:path.join(output,'presentation-markdown.png')});await press();assert.equal(await markdown.textContent(),mdBefore); - await page.getByRole('button',{name:'查看 Markdown 原文',exact:true}).click();await editor.waitFor();assert.match(await editor.inputValue(),/Draft stays\./); - await page.getByRole('button',{name:'渲染 Markdown',exact:true}).click();await markdown.waitFor(); + await chooseFileAction('查看 Markdown 原文');await editor.waitFor();assert.match(await editor.inputValue(),/Draft stays\./); + await chooseFileAction('复制代码'); + assert.equal(await page.evaluate(()=>window.copiedText),await editor.inputValue()); + await chooseFileAction('渲染 Markdown');await markdown.waitFor(); console.log('[pass] Markdown WYSIWYG/source switches, scale and presentation retain unsaved content'); const savesBefore=await page.evaluate(()=>window.fixture.saves); assert.deepEqual(await page.evaluate(async()=>{const {shortcutStore,captureShortcut}=await import('/src/lib/shortcutBindings.js');return shortcutStore().save('save',captureShortcut({code:'KeyG',ctrlKey:true,altKey:true}));}),{ok:true}); @@ -169,6 +189,10 @@ try { console.log('[pass] read-only Markdown context menu and rendered diff font scaling'); await page.evaluate(()=>window.fixture.select('slides.pptx')); const slide=page.frameLocator('.ace-side-presentation-frame').locator('section');await slide.waitFor(); + await fileActions.click(); + assert.ok(await fileMenu.getByRole('menuitem',{name:'上一张幻灯片'}).isDisabled()); + assert.ok(await fileMenu.getByRole('menuitem',{name:'下一张幻灯片'}).isDisabled()); + await fileMenu.getByRole('menuitem',{name:'放大',exact:true}).click(); await slide.focus();await press();assert.equal((await state()).active,true);await slide.focus();await press();assert.equal((await state()).active,false); await page.frameLocator('.ace-side-presentation-frame').locator('body').click({button:'right',position:{x:150,y:30}});await menu.getByRole('menuitem',{name:/全屏演示/}).click();await settle();assert.equal((await state()).active,true);await press(); await page.evaluate(()=>window.dispatchEvent(new MessageEvent('message',{source:document.querySelector('iframe').contentWindow,data:{source:'acecode:presentation-preview',channel:'forged',status:'presentation'}})));await settle();assert.equal((await state()).active,false); diff --git a/web/src/App.jsx b/web/src/App.jsx index ba5df8cf4..48ea67ba7 100644 --- a/web/src/App.jsx +++ b/web/src/App.jsx @@ -1,4 +1,5 @@ import { useDesktopOffice } from './lib/useDesktopOffice.js'; +import { useAndroidPetActivity, usePhoneApproval } from './lib/androidPhone.js'; // 顶层 App:鉴权 gate(401 → TokenPrompt)+ 主壳。 // // 视觉对齐设计稿方向 C:顶部 41px TopBar + 270px Sidebar + 主区(单会话/4宫格/9宫格) @@ -2105,6 +2106,8 @@ export function App() { [questionReqs, activeId, permissionOwnership], ); // Keep every hook above the authentication early returns. + useAndroidPetActivity(pendingPermissionSessionIdsForSidebar.size > 0 || pendingQuestionSessionIdsForSidebar.size > 0); + usePhoneApproval(permReqs, handlePermissionDecision); const visibleQuestionReq = useMemo(() => { if (visiblePermissionUnresolved) return null; const request = visibleQuestionRequest(questionReqs, activeId, permissionOwnership); diff --git a/web/src/components/AnchoredMenu.jsx b/web/src/components/AnchoredMenu.jsx index 168420bf9..7fb221fcc 100644 --- a/web/src/components/AnchoredMenu.jsx +++ b/web/src/components/AnchoredMenu.jsx @@ -7,7 +7,7 @@ import { notifyNativeSurfaceOverlayChange } from '../lib/agentBrowserSurfaceCoor // Own placement and dismissal together so menus escape clipped ancestors while // still following their trigger as the window, content, or dock changes size. -export function AnchoredMenu({ anchorRef, onClose, children, className = '', width, maxHeightRatio = 1, preferredPlacement = 'below', ...props }) { +export function AnchoredMenu({ anchorRef, onClose, children, className = '', width, maxHeightRatio = 1, preferredPlacement = 'below', align = 'start', ...props }) { const zIndex = useOverlayLayer(); const menuRef = useRef(null); const updatePositionRef = useRef(null); @@ -47,6 +47,7 @@ export function AnchoredMenu({ anchorRef, onClose, children, className = '', wid viewportHeight, maxHeight: viewportHeight * maxHeightRatio, preferredPlacement, + align, }); const key = JSON.stringify(position); if (lastPosition === key) return; @@ -108,7 +109,7 @@ export function AnchoredMenu({ anchorRef, onClose, children, className = '', wid if (menu.contains(document.activeElement)) previousFocus?.focus?.({ preventScroll: true }); notifyNativeSurfaceOverlayChange(); }; - }, [anchorRef, width, maxHeightRatio, preferredPlacement]); + }, [anchorRef, width, maxHeightRatio, preferredPlacement, align]); // A sibling insertion can move the anchor without resizing any observed box. // Recheck after parent renders as well as on resize/scroll notifications. diff --git a/web/src/components/AndroidPhoneSettings.jsx b/web/src/components/AndroidPhoneSettings.jsx new file mode 100644 index 000000000..5d1e07551 --- /dev/null +++ b/web/src/components/AndroidPhoneSettings.jsx @@ -0,0 +1,76 @@ +import { useEffect, useSyncExternalStore } from 'react'; +import { phoneRequest } from '../lib/androidPhone.js'; +import { createPhoneSettingsStore } from '../lib/androidPhoneSettings.js'; +import { VsIcon } from './Icon.jsx'; +import { Toggle } from './Modal.jsx'; +import clippyPreview from '../../../assets/desktop_pet/classic/clippy/preview.png'; +import roverPreview from '../../../assets/desktop_pet/classic/rover/preview.png'; +import whalePreview from '../../../assets/desktop_pet/whale/preview.png'; + +const settings = createPhoneSettingsStore(phoneRequest); +const PETS = [ + { id: 'off', label: '关闭' }, + { id: 'whale', label: '鲸鱼娘', preview: whalePreview }, + { id: 'clippy', label: '大眼回形针', preview: clippyPreview }, + { id: 'rover', label: 'Rover', preview: roverPreview }, +]; + +function PhoneIcon() { + return ; +} + +export function AndroidPhoneSettings({ pet = false }) { + const { status, pending, error, draftPet } = useSyncExternalStore(settings.subscribe, settings.getSnapshot); + const available = !!globalThis.window?.acePhone; + useEffect(() => { + if (!available) return undefined; + const refresh = () => { void settings.refresh(); }; + refresh(); + window.addEventListener('ace-phone-resume', refresh); + return () => window.removeEventListener('ace-phone-resume', refresh); + }, [available]); + if (!available) return null; + const busy = pending > 0; + const selected = draftPet ?? status?.pet ?? 'off'; + const needsOverlay = !!status && !status.overlay && selected !== 'off'; + return ( +
+
+
+ {pet ? : } +
+
+
{pet ? '悬浮宠物' : '手机操控'}
+ {(!pet || busy || !status || needsOverlay) &&
+ {busy ? <>正在更新 : !status ? '状态未加载' : needsOverlay ? '需要悬浮窗权限' + : <>{status.accessibility ? '已开启' : '未开启'}} +
} +
+ {!pet && { void settings.accessibility(); }} />} +
+ {pet &&
+ {PETS.map(item => )} +
} + {needsOverlay &&
+ 允许显示在其他应用上层后生效 + +
} + {error &&
+ {error} + +
} +
+ ); +} diff --git a/web/src/components/ChannelsSettings.jsx b/web/src/components/ChannelsSettings.jsx index b1505f914..f5157b931 100644 --- a/web/src/components/ChannelsSettings.jsx +++ b/web/src/components/ChannelsSettings.jsx @@ -33,6 +33,8 @@ import { savedBotSummary, } from '../lib/channelsSettings.js'; import { copyTextToSystemClipboard } from '../lib/systemClipboard.js'; +import { clipboardPasteButtonAvailable } from '../lib/systemClipboard.js'; +import { ClipboardPasteButton } from './ClipboardPasteButton.jsx'; import { Modal } from './Modal.jsx'; import { VsIcon } from './Icon.jsx'; import { toast } from './Toast.jsx'; @@ -211,11 +213,17 @@ function CredentialFields({ platformKey, platform, draft, setDraft, autoFocus }) ) : ( - setDraft((d) => ({ ...d, [field.key]: e.target.value }))} - autoComplete={field.secret ? 'new-password' : 'off'} spellCheck={false} aria-label={field.label} /> +
+ setDraft((d) => ({ ...d, [field.key]: e.target.value }))} + autoComplete={field.secret ? 'new-password' : 'off'} spellCheck={false} aria-label={field.label} /> + {field.secret && ( + setDraft((d) => ({ ...d, [field.key]: text }))} + className="absolute inset-y-0 right-0 w-12 rounded-r-md flex items-center justify-center px-2.5 text-[12px] text-accent transition hover:bg-surface-hi focus:outline-none focus:ring-1 focus:ring-inset focus:ring-accent disabled:opacity-50" /> + )} +
) )); } diff --git a/web/src/components/ChatView.jsx b/web/src/components/ChatView.jsx index d97a9e053..f26a202a2 100644 --- a/web/src/components/ChatView.jsx +++ b/web/src/components/ChatView.jsx @@ -96,6 +96,7 @@ import { sessionContentLoadingPhase } from '../lib/sessionContentLoading.js'; import { SidePanel } from './SidePanel.jsx'; import { BackgroundTasksView } from './SubagentPanel.jsx'; import { SummaryPanel } from './SummaryPanel.jsx'; +import { mergeWithdrawnContexts, withdrawnSteerComposerState } from '../lib/steerWithdrawal.js'; import { TranscriptItems } from './TranscriptItems.jsx'; import { TranscriptHistoryBoundary } from './TranscriptHistoryBoundary.jsx'; import { useTranscriptHistory } from '../lib/useTranscriptHistory.js'; @@ -343,6 +344,7 @@ import { closeAgentBrowserPage, createAgentBrowserPage, hasNativeAgentBrowser, + reserveDraftBrowserSessionId, runAgentBrowserBridgeAction, selectAgentBrowserPage, waitForAgentBrowserPageReady, @@ -905,6 +907,10 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses const [fileLocateRequest, setFileLocateRequest] = useWorkbenchState(workbenchOwner, 'fileLocate', () => ({ path: '', token: 0 })); const [previewTabState, setPreviewTabState] = useWorkbenchState(workbenchOwner, 'previews', () => ({})); const getPreviewTabState = useStableCallback(() => sessionWorkbench.get(workbenchOwner, 'previews', () => ({}))); + // 新建任务页开浏览器时预留的会话 id:页面先归属它,发送首条消息时交给 daemon 建会话。 + const [draftBrowserSessionId, setDraftBrowserSessionId] = useWorkbenchState(workbenchOwner, 'draftBrowserSessionId', ''); + const workbenchOwnerRef = useRef(workbenchOwner); + workbenchOwnerRef.current = workbenchOwner; const previewContextRef = useRef(null); const previewLeaveRequestRef = useRef(() => true); const [previewCloseConfirm, setPreviewCloseConfirm] = useState(null); @@ -1560,9 +1566,18 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses const options = targetNoWorkspace ? { ...baseOptions, ...expertOptions, no_workspace: true, noWorkspace: true } : { ...baseOptions, ...expertOptions }; - const create = isRealWorkspaceHash(targetHash) - ? api.createWorkspaceSession(targetHash, options) - : api.createSession(options); + const createIn = (createOpts) => (isRealWorkspaceHash(targetHash) + ? api.createWorkspaceSession(targetHash, createOpts) + : api.createSession(createOpts)); + // 新建页已开过浏览器:用预留的 id 建会话,页面就直接归属新会话。id 已被占用 + // (极少见,比如上次建会话成功但响应丢了)时改由 daemon 生成,浏览器页面不跟过去。 + const draftBrowserOwner = !sid && !children ? draftBrowserSessionId : ''; + const create = draftBrowserOwner + ? createIn({ ...options, session_id: draftBrowserOwner }).catch((error) => { + if (error?.code !== 'SESSION_ID_EXISTS') throw error; + return createIn(options); + }) + : createIn(options); setHomeSubmitting(true); try { const r = await create; @@ -1605,12 +1620,18 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses } const nextOwner = sessionWorkbench.ownerFor(next); if (health?.console?.available) await api.transferPtyOwner(workbenchOwner, nextOwner); - sessionWorkbench.transfer(workbenchOwner, nextOwner, (record) => ({ + const browserPagesFollow = !!draftBrowserOwner && id === draftBrowserOwner; + sessionWorkbench.transfer(workbenchOwner, nextOwner, ({ draftBrowserSessionId: _draftBrowser, ...record }) => ({ ...record, previews: sourcePreviewContext ? transferPreviewTabs(record.previews || {}, sourcePreviewContext, { ...sourcePreviewContext, sessionId: id, - }) : {}, + }, { browserPages: browserPagesFollow }) : {}, })); + if (draftBrowserOwner && !browserPagesFollow) { + for (const page of agentBrowserPagesForSession(agentBrowserPageStoreSnapshot(), draftBrowserOwner)) { + void closeAgentBrowserPage(page.pageId); + } + } onSessionPromoted?.(next); notifySessionListChanged({ reason: 'session-created', @@ -3402,12 +3423,38 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses }); }); - // 插话气泡上的撤回:模型还没读到时从当前回合拿掉,不影响回合。 + // 撤回成功的插话退回输入框(照 Claude 桌面版:没发出去的话不丢)。输入框已有内容时 + // 退回的放前面、换行接原内容;附件块与上下文芯片一起回来。 + const returnWithdrawnSteerToComposer = useStableCallback((queuedItem, targetSid) => { + const restored = withdrawnSteerComposerState(queuedItem, { + currentContent: composerContentRef.current, + currentText: composerValueRef.current, + sessionId: targetSid, + }); + if (!restored) return; + if (restored.resources.length > 0) { + // 先登记附件资源:输入框回调里按资源认附件块,认不到会标成「上传不可用」。 + composerAttachmentsRef.current = mergeComposerAttachmentResources(composerAttachmentsRef.current, restored.resources); + setComposerAttachments((current) => mergeComposerAttachmentResources(current, restored.resources)); + } + if (restored.contexts.length > 0) { + setComposerContexts((current) => mergeWithdrawnContexts(restored.contexts, current)); + } + if (inputRef.current?.setComposerContent) inputRef.current.setComposerContent(restored.content); + else handleComposerChange(composerContentText(restored.content), restored.content); + requestAnimationFrame(() => inputRef.current?.focus?.()); + }); + + // 插话气泡上的撤回:模型还没读到时从当前回合拿掉,不影响回合;这条消息退回输入框。 const withdrawSteered = useStableCallback((queuedId) => { const targetSid = sidRef.current; if (!targetSid) return; + const queuedItem = queueStore.getState().items.find((item) => item?.queued?.id === queuedId); api.withdrawSteer(targetSid, queuedId) - .then(() => updateQueueState((prev) => cancelQueuedInput(prev, queuedId))) + .then(() => { + updateQueueState((prev) => cancelQueuedInput(prev, queuedId)); + if (sidRef.current === targetSid) returnWithdrawnSteerToComposer(queuedItem, targetSid); + }) .catch((e) => { if (e?.code !== 'STEER_NOT_PENDING') { toast({ kind: 'err', text: '撤回失败:' + (e?.message || '未知错误') }); @@ -5195,9 +5242,13 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses agentBrowserPageStoreSnapshot, agentBrowserPageStoreSnapshot, ); + // 浏览器页面的 owner:会话页就是会话 id;新建任务页是预留 id(还没开过浏览器时为空)。 + // 定时任务页 / 专家组件页(children)不开浏览器。 + const homeBrowserAvailable = !sid && !children; + const browserSessionId = sid || (homeBrowserAvailable ? draftBrowserSessionId : ''); const sessionBrowserPages = useMemo( - () => agentBrowserPagesForSession(agentBrowserRegistry, sid), - [agentBrowserRegistry, sid], + () => agentBrowserPagesForSession(agentBrowserRegistry, browserSessionId), + [agentBrowserRegistry, browserSessionId], ); const sessionBrowserTargetPageId = useMemo( () => agentBrowserSessionTargetPageId(agentBrowserRegistry, sid), @@ -5753,11 +5804,14 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses } }); + // 浏览器页签挂在预览视图的会话键上:会话页是会话 id,新建任务页是草稿身份; + // 页面本身的 owner 另见 browserSessionId。 + const browserTabSessionId = previewContext.sessionId; const showBrowserPage = useCallback((pageId, title, favicon) => { - if (!sid || !pageId) return; + if (!browserTabSessionId || !pageId) return; return selectPreview((prev) => openBrowserTab(prev, { scopeKey: previewScope, - sessionId: sid, + sessionId: browserTabSessionId, pageId, title, favicon, @@ -5766,19 +5820,30 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses setPreviewPanelHidden(false); void selectAgentBrowserPage(pageId); }); - }, [onRevealPreviewPanel, previewScope, selectPreview, sid]); + }, [browserTabSessionId, onRevealPreviewPanel, previewScope, selectPreview]); const browserOpenPendingRef = useRef(false); const openBrowserPreview = useStableCallback(async (url = '') => { - if (!sid || !hasNativeAgentBrowser() || browserOpenPendingRef.current) return; + if (!(sid || homeBrowserAvailable) || !hasNativeAgentBrowser() || browserOpenPendingRef.current) return; + // 视图换了(切会话、新建页已发出首条消息)就不再往原视图开页。 + const viewOwner = workbenchOwner; + const isCurrentView = () => workbenchOwnerRef.current === viewOwner; + let ownerSessionId = sid; + if (!ownerSessionId) { + ownerSessionId = draftBrowserSessionId || reserveDraftBrowserSessionId(); + if (ownerSessionId !== draftBrowserSessionId) setDraftBrowserSessionId(ownerSessionId); + } browserOpenPendingRef.current = true; try { - if (!await requestActiveFileLeave() || sidRef.current !== sid) return; - const created = await createAgentBrowserPage(agentBrowserOwnerForSession(ref)); + if (!await requestActiveFileLeave() || !isCurrentView()) return; + const created = await createAgentBrowserPage(agentBrowserOwnerForSession({ + sessionId: ownerSessionId, + workspaceHash: ref?.workspaceHash || '', + })); if (created?.ok === false || !created?.page_id) { throw new Error(created?.error || '浏览器操作失败'); } - if (sidRef.current !== sid) { + if (!isCurrentView()) { await closeAgentBrowserPage(created.page_id); return; } @@ -5793,7 +5858,7 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses } if (typeof url === 'string' && url) { const ready = await waitForAgentBrowserPageReady(created.page_id, { - isCurrent: () => sidRef.current === sid, + isCurrent: isCurrentView, }); if (ready.cancelled) { await closeAgentBrowserPage(created.page_id); @@ -5814,9 +5879,9 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses // 切会话时向 Desktop 对账一次 native 页面池;事件流已经在 App 级持续镜像。 useEffect(() => { - if (!sid || !hasNativeAgentBrowser()) return; - void reconcileAgentBrowserPageStore(sid); - }, [sid]); + if (!browserSessionId || !hasNativeAgentBrowser()) return; + void reconcileAgentBrowserPageStore(browserSessionId); + }, [browserSessionId]); // 旧版 Desktop 的状态事件不带 owner:退回按「当前会话有正在执行的浏览器工具」 // 认领,只影响本地登记表镜像,native 已归属的页面不会被抢走。 @@ -5839,18 +5904,18 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses // 同一会话不重复抢焦点。 const revealedBrowserPagesRef = useRef(new Map()); useEffect(() => { - if (!sid) return; + if (!browserSessionId || !browserTabSessionId) return; const restoredPages = visiblePreviewTabs(getPreviewTabState(), previewContext) .filter((tab) => tab.type === PREVIEW_TAB_TYPES.BROWSER).map((tab) => tab.pageId); setPreviewTabState((prev) => syncBrowserTabsForSession(prev, { scopeKey: previewScope, - sessionId: sid, + sessionId: browserTabSessionId, pages: sessionBrowserPages, })); - let revealed = revealedBrowserPagesRef.current.get(sid); + let revealed = revealedBrowserPagesRef.current.get(browserTabSessionId); if (!revealed) { revealed = new Set(restoredPages); - revealedBrowserPagesRef.current.set(sid, revealed); + revealedBrowserPagesRef.current.set(browserTabSessionId, revealed); } for (const page of sessionBrowserPages) { if (revealed.has(page.pageId)) continue; @@ -5862,7 +5927,7 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses revealed.delete(pageId); } } - }, [getPreviewTabState, previewContext, previewScope, sessionBrowserPages, showBrowserPage, sid, workbenchOwner]); + }, [browserSessionId, browserTabSessionId, getPreviewTabState, previewContext, previewScope, sessionBrowserPages, showBrowserPage, workbenchOwner]); // Agent 切换默认目标页(browser_open / 显式选页)且有浏览器工具正在执行时,把 // 那一页的页签激活到前台;目标不属于本会话(显式操作别的会话的页)则不动。 @@ -6900,7 +6965,7 @@ export function ChatView({ titleTarget, actionsTarget, children, sessionRef, ses onToggleMaximize={onToggleSidePanelMaximized} onToggleSidePanelList={toggleVisibleSidePanelList} onOpenFile={sidePanelCwd ? openPreviewFilePicker : null} - onOpenBrowser={sid && hasNativeAgentBrowser() ? openBrowserPreview : null} + onOpenBrowser={(sid || homeBrowserAvailable) && hasNativeAgentBrowser() ? openBrowserPreview : null} onRetargetSimulatorTab={retargetSimulatorPreview} onOpenSideChat={openSideQuestionComposer} onHide={hidePreviewPanel} diff --git a/web/src/components/ClipboardPasteButton.jsx b/web/src/components/ClipboardPasteButton.jsx new file mode 100644 index 000000000..fb6ea68c2 --- /dev/null +++ b/web/src/components/ClipboardPasteButton.jsx @@ -0,0 +1,22 @@ +import { clipboardPasteButtonAvailable, readTextFromSystemClipboard } from '../lib/systemClipboard.js'; +import { toast } from './Toast.jsx'; + +// 密钥输入框旁的「粘贴」。只在安卓 App 里出现:那里密码框会弹出禁止粘贴的安全键盘, +// 网页自己也读不了剪贴板,只能经 App 的剪贴板通道读。粘进来的内容去掉首尾空白。 +export function ClipboardPasteButton({ onPaste, className = '', disabled = false }) { + if (!clipboardPasteButtonAvailable()) return null; + const paste = async () => { + const result = await readTextFromSystemClipboard(); + const text = result.ok ? result.text.trim() : ''; + if (!text) { + toast({ kind: 'err', text: '剪贴板里没有文字' }); + return; + } + onPaste(text); + }; + return ( + + ); +} diff --git a/web/src/components/ComposerSessionControls.jsx b/web/src/components/ComposerSessionControls.jsx index 474753fb2..b29d4ba18 100644 --- a/web/src/components/ComposerSessionControls.jsx +++ b/web/src/components/ComposerSessionControls.jsx @@ -72,6 +72,7 @@ function useAdaptiveComposerControls(rootRef, measureKey) { const isCrowded = () => overflowWidth() > 1; const compactOrder = ['permission', 'expert-pending', 'expert', 'swarm-mode', 'goal', 'reasoning', 'model']; + const minimumWidth = parseFloat(getComputedStyle(root).getPropertyValue('--ace-composer-min-target')) || 28; const nextCompact = new Set(); for (const controlName of compactOrder) { if (!isCrowded()) break; @@ -84,8 +85,8 @@ function useAdaptiveComposerControls(rootRef, measureKey) { nextCompact.add(controlName); element.setAttribute('data-compact', 'true'); const width = controlName === 'model' - ? Math.max(28, Math.floor(element.getBoundingClientRect().width - overflowWidth())) - : 28; + ? Math.max(minimumWidth, Math.floor(element.getBoundingClientRect().width - overflowWidth())) + : minimumWidth; if (controlName === 'model') element.style.width = `${width}px`; element.style.flex = `0 0 ${width}px`; root.getBoundingClientRect(); @@ -99,7 +100,7 @@ function useAdaptiveComposerControls(rootRef, measureKey) { if (!contentClipped) return; nextCompact.add(element.dataset.composerControl); element.setAttribute('data-compact', 'true'); - element.style.flex = '0 0 28px'; + element.style.flex = `0 0 ${minimumWidth}px`; root.getBoundingClientRect(); }); if (isCrowded()) { diff --git a/web/src/components/DesktopContextMenu.jsx b/web/src/components/DesktopContextMenu.jsx index f28196afc..f41214365 100644 --- a/web/src/components/DesktopContextMenu.jsx +++ b/web/src/components/DesktopContextMenu.jsx @@ -15,6 +15,7 @@ import { DESKTOP_CONTEXT_ACTIONS, editableTargetFromElement, SESSION_PIN_TOGGLE_EVENT, + shouldOpenContextMenu, OPEN_DESKTOP_CONTEXT_MENU_EVENT, CLOSE_DESKTOP_CONTEXT_MENU_EVENT, withMenuSeparators, @@ -34,8 +35,10 @@ import { } from '../lib/richComposerContextPaste.js'; import { api } from '../lib/api.js'; import { VsIcon } from './Icon.jsx'; +import { openTargetInExplorer } from './openTargetInExplorer.js'; import { Modal } from './Modal.jsx'; import { toast } from './Toast.jsx'; +import { shouldUseNativeContextMenu } from '../lib/mobileShell.js'; const MENU_WIDTH = 176; const ICON_MENU_WIDTH = 216; @@ -141,14 +144,6 @@ const CONTEXT_ACTION_ICONS = Object.freeze({ [DESKTOP_CONTEXT_ACTIONS.INSPECT]: 'Inspect', }); -function parseDesktopResult(value) { - if (value == null) return value; - if (typeof value !== 'string') return value; - const text = value.trim(); - if (!text || text === 'null') return null; - return JSON.parse(text); -} - function selectedTextForTarget(target) { const editable = editableTargetFromElement(target); if (editable instanceof HTMLInputElement || editable instanceof HTMLTextAreaElement) { @@ -265,30 +260,6 @@ async function pasteIntoTarget(target, richComposerSelection = null) { document.execCommand('paste'); } -async function openTargetInExplorer(openTarget) { - if (!openTarget?.path) { - toast({ kind: 'err', text: '无法打开:路径为空' }); - return; - } - try { - let result; - if (typeof window.aceDesktop_openInExplorer === 'function') { - result = parseDesktopResult(await window.aceDesktop_openInExplorer(openTarget.path)); - } else { - // webapp 兼容模式没有 webview bridge,daemon 就在本机,改走 REST 端点。 - result = await api.openInExplorer(openTarget.path); - } - if (!result?.ok) { - toast({ kind: 'err', text: '打开失败:' + (result?.error || '') }); - return; - } - toast({ kind: 'ok', text: '已打开文件夹' }); - } catch (e) { - const detail = e?.body?.error || e?.message || ''; - toast({ kind: 'err', text: '打开失败:' + detail }); - } -} - function dispatchSessionPinToggle(sessionPinTarget, nextPinned) { if (!sessionPinTarget?.sessionId) return; window.dispatchEvent(new CustomEvent(SESSION_PIN_TOGGLE_EVENT, { @@ -546,6 +517,9 @@ export function DesktopContextMenu() { // Dismiss before mousedown enters the native bridge; leave the press intact. flushSync(close); }; + // 记下最近一次按下来自鼠标还是手指,长按(contextmenu)时据此决定走哪种菜单。 + let lastPointerType = ''; + const rememberPointerType = (event) => { lastPointerType = event.pointerType || ''; }; const onContextMenu = (event) => { // 控制台终端区由 ConsoleDock 自己处理右键(VS Code 式 复制/粘贴),这里放行 // (不 preventDefault / 不 stopPropagation),让事件继续冒泡到终端的 onContextMenu。 @@ -553,6 +527,15 @@ export function DesktopContextMenu() { const rawTarget = explicit?.target || event.target; if (rawTarget instanceof Element && rawTarget.closest('.ace-console-term')) return; + // 手机上手指长按输入框或已选中的文字:交给系统自己的复制 / 粘贴工具条。 + // 自定义菜单的「粘贴」要读剪贴板,安卓 WebView 不给网页这个权限,拦下来就粘贴不了。 + if (!explicit && shouldUseNativeContextMenu({ + pointerType: lastPointerType, + firesTouchEvents: event.sourceCapabilities?.firesTouchEvents === true, + editable: !!editableTargetFromElement(rawTarget), + hasSelection: String(window.getSelection?.() || '').length > 0, + })) return; + // 顶栏等区域右键委托给认领方(会话头部 = 与「会话菜单」按钮左键同一份菜单); // 没人认领(例如首页没有打开会话)时回落到通用菜单。 if (!explicit) { @@ -612,6 +595,11 @@ export function DesktopContextMenu() { [...(explicit?.leadingItems || []), ...contextItems], explicit?.actionOverrides, )); + // 空菜单不弹;上面已 preventDefault,浏览器原生菜单也不会出来。 + if (!shouldOpenContextMenu(items)) { + close(); + return; + } // 侧栏会话行 / 项目行的右键与会话菜单按钮一样带图标;其它区域的右键保持纯文字。 const showIcons = explicit ? explicit.showIcons !== false @@ -703,6 +691,7 @@ export function DesktopContextMenu() { document.addEventListener(OPEN_DESKTOP_CONTEXT_MENU_EVENT, onContextMenu); document.addEventListener(CLOSE_DESKTOP_CONTEXT_MENU_EVENT, close); + window.addEventListener('pointerdown', rememberPointerType, true); document.addEventListener('contextmenu', onContextMenu, true); document.addEventListener('selectionchange', rememberSelection); document.addEventListener('select', rememberSelection, true); @@ -715,6 +704,7 @@ export function DesktopContextMenu() { return () => { document.removeEventListener(OPEN_DESKTOP_CONTEXT_MENU_EVENT, onContextMenu); document.removeEventListener(CLOSE_DESKTOP_CONTEXT_MENU_EVENT, close); + window.removeEventListener('pointerdown', rememberPointerType, true); document.removeEventListener('contextmenu', onContextMenu, true); document.removeEventListener('selectionchange', rememberSelection); document.removeEventListener('select', rememberSelection, true); diff --git a/web/src/components/FilePreviewContent.jsx b/web/src/components/FilePreviewContent.jsx index 9643d504a..efb72ae12 100644 --- a/web/src/components/FilePreviewContent.jsx +++ b/web/src/components/FilePreviewContent.jsx @@ -35,14 +35,14 @@ import { captureFilePreviewScroll, restoredFilePreviewScroll, } from '../lib/filePreviewScroll.js'; -import { CopyableCodeFrame } from './CopyableCodeFrame.jsx'; +import { createPortal } from 'react-dom'; +import { FilePreviewMenu } from './FilePreviewMenu.jsx'; import { ImageLightbox } from './ImageLightbox.jsx'; import { - OfficePreviewControls, + officeZoomActions, useOfficePreviewZoom, } from './OfficePreviewControls.jsx'; import { SelectionAnnotationOverlay } from './SelectionAnnotationOverlay.jsx'; -import { VsIcon } from './Icon.jsx'; import { toast } from './Toast.jsx'; // Office 预览与 Markdown 所见即所得编辑器各自拖着大块第三方库(pptx2html 渲染器 @@ -66,7 +66,7 @@ function escapeHtml(s) { } async function copyWithToast(text, okText) { - const result = await copyTextToSystemClipboard(text); + const result = await copyTextToSystemClipboard(text, window, { allowEmpty: true }); if (result.ok) toast({ kind: 'ok', text: okText }); else toast({ kind: 'err', text: '复制失败:' + (result.error || '') }); } @@ -174,6 +174,7 @@ function HighlightedTextEditor({ } export function FilePreviewContent({ + actionsHost, textZoom = 1, owner, api, @@ -527,21 +528,26 @@ export function FilePreviewContent({ 'data-desktop-preview-copy-image-url': state.kind === 'image' ? state.previewUrl || undefined : undefined, 'data-selection-source-revision': sourceContentRevision || undefined, }; + const renderMenu = (actions = []) => actionsHost && createPortal( + , actionsHost, + ); if (state.status === 'loading') { - return
加载中...
; + return
{renderMenu()}加载中...
; } if (state.status === 'error') { return (
+ {renderMenu()}
{state.error}
{path}
); } - if (state.status !== 'ok') return null; + if (state.status !== 'ok') return renderMenu(); if (state.kind === 'image') { return (
+ {renderMenu()}