Repository navigation
feat: Advanced build that drives agent tabs through the debugger - #28
Merged
Merged
Conversation
One codebase, two build profiles. Standard is unchanged and has no debugger permission or code. DEWEE_PROFILE=advanced adds the debugger permission and a CDP adapter behind the same page-driver interface: accessibility-tree snapshots with the shared redaction rule, trusted click/type/press input and viewport JPEG screenshots under 1.5 MiB. The person turns the debugger on in the side panel, per browser session. It attaches only to agent-owned tabs while a command runs, calls only a fixed CDP method list, re-checks the live policy before every command and detaches on Stop, Pause, revoke, expiry, disconnect and opt-in off. When another debugger holds the tab it answers debugger_unavailable, or reads the standard way when the person allowed that; screenshots never fall back. The side panel now shows agent tabs, site scope and TTL, and a timeline of recent outcomes. Agent tabs get a badge, targets are highlighted before a click, and the person's own input in an agent tab pauses control. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The e2e fixture can load either build. The Advanced spec drives the opt-in, AX snapshot redaction, screenshots, frames, a debugger conflict and detaching mid-action against the loopback relay. The Standard spec now checks the manifest has no debugger permission. bench:browser compares snapshot and click latency of both engines. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Threat model section for the debugger, build profiles and scripts, Chrome Web Store review notes for the debugger permission, the live control UX, and an AGENTS.md security rule for the CDP method list. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…put as unknown Keys are sent only when the real focus, followed through open shadow roots and frames, is in the granted document and not on a sensitive field, in both engines. Once any input event reached the page, a later failure reports outcome_unknown so the agent never repeats a half-done action. Screenshot redaction is measured before and after the capture and refuses the image when the layout moved twice, a box cannot be measured or more than 64 areas need covering. Every cross-origin or unreadable frame is blacked out, and an accessibility node with no DOM record is redacted. A detach requested during an attach takes effect when the attach ends, and the debugger adapter stops when the command is cancelled. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ing attach A detach requested while the debugger is attaching now ends the command with the policy's own reason (device_paused, grant_revoked, grant_expired) or cancelled, and the router falls back to the standard engine only for a real debugger conflict while the command is live and the policy still allows it. Keys in a background agent tab with nothing focused go to the main document. Takeover detection is suppressed only around the input events themselves, an attach Chrome finishes after it timed out is detached, and the threat model records that focus can move between the check and the key. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Chrome leaves cross-site frames out of the frame tree, so finding no focused node does not prove focus is in the main document. When no tree shows focus, keys now go to the main document only if every frame owner on the page, inside closed shadow roots and including object and embed, holds a readable document of the granted main origin. Otherwise the key is refused with origin_not_granted and a message saying why. Object and embed are blacked out in screenshots like frames, and input from the person is ignored only after an input event was actually sent. Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an Advanced build profile that drives agent tabs through the Chrome debugger (CDP) behind the same page-driver interface as the Standard engine. One codebase, two builds:
pnpm build(Standard) is unchanged: nodebuggerpermission and no CDP code in the bundle (checked by an e2e manifest test and a bundle grep).pnpm build:advanced/zip:advancedadds thedebuggerpermission and the CDP adapter.Closes #18. Builds on #23. The Gateway already checks the
advancedcapability of the live attach, so no backend change is needed.What Advanced adds
[eN] role "name"format, with the shared sensitive-field rule; an AX node with no DOM record is redacted.Input.*).objectorembedframe are blacked out; boxes are measured before and after the capture and the image is refused if the layout moved twice, a box cannot be measured or more than 64 areas need covering.Safeguards
["standard","advanced"]; attach announcesadvancedonly while it is on.src/browser/adapters/cdp/cdp-client.ts). NoRuntime.evaluate,Network,Storage, cookies,Target, browser-level target or method/params from the wire (also an AGENTS.md rule).origin_not_granted.outcome_unknown(both engines), so an agent never repeats a half-done action.debugger_unavailable, or the Standard read/act path when the person allowed that fallback. Screenshots never fall back.Benchmark (
pnpm bench:browser, 20 rounds, same build and page)CDP is slower on this small page (attach/detach per command, full AX tree); ~550 ms of each click is the shared highlight and settle wait. Its value is trusted input, the AX tree and screenshots, not speed.
Docs
docs/browser-control-threat-model.md(debugger section, residual risks),docs/architecture.md,docs/development.md,README.md,AGENTS.md(debugger rule,build:advanced), and Chrome Web Store review notes for thedebuggerpermission.Verification
pnpm lint,pnpm typecheck,pnpm test(802/802),pnpm build,pnpm build:advancedpass. The Standard bundle contains no"debugger",Input.dispatchKeyEvent,Accessibility.getFullAXTreeorPage.captureScreenshot.browser-control-advanced(opt-in, AX redaction, screenshots, frames, closed-shadow-root cross-site frame, debugger conflict, detach mid-action, partial input →outcome_unknown, background-tab keys),browser-control,smokeand the live connected-browser spec against a real local Gateway pass 28/28. The same-site frame test passed 8/8 repeats.🤖 Generated with Claude Code