Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ node_modules/
.output/
.output-e2e/
.output-store/
.output-advanced/
.output-e2e-advanced/
.wxt/
dist/
coverage/
Expand Down
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ pnpm typecheck
pnpm test # unit + integration (Vitest)
pnpm test:e2e # builds, then runs Playwright against the unpacked build
pnpm build # .output/chrome-mv3
pnpm build:advanced # .output-advanced/chrome-mv3, adds the `debugger` permission (zip with `pnpm zip:advanced`)
pnpm zip # sideload zip (keeps the pinned extension key)
pnpm zip:store # Chrome Web Store zip (manifest without `key`)
```
Expand Down Expand Up @@ -54,6 +55,7 @@ Rules:
- The side panel never reads the active page or the person's other tabs. There are two opt-in exceptions, both off by default:
- The "Ask Dewee" page script sends the text the person selected (never form fields) when they click its button, and only as an editable draft.
- "Connected browser" (self-hosted only) lets an agent read and act only in tabs it opened itself, only on exact origins the person allowed for that agent in the panel, only until the grant expires or is revoked, and only after the person granted Chrome host permission for those origins. Password, hidden, payment and one-time-code values are never read or typed, page content is never stored, and Pause/Stop/Revoke work locally. See [docs/browser-control-threat-model.md](docs/browser-control-threat-model.md).
- The `debugger` permission exists only in the Advanced build profile. Its adapter attaches only to agent-owned tabs, only while a command runs and the person has Advanced mode on, and calls only the fixed CDP method list in `src/browser/adapters/cdp/cdp-client.ts`. Never add `Runtime.evaluate`, `Network`, `Storage`, cookie or `Target` methods, a browser-level target, a remote debugging port, or a CDP method or params taken from a command. Screenshots are viewport-only, never stored, and never fall back to another path.

## Code layout

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Chat with your Dewee agents, with Super Agent pinned first, in Chrome's side pan
- **Workspaces and agents:** pick a workspace, search agents, and keep separate sessions per agent.
- **Reliable chat:** streaming, stop, retry, history, new chat. If the connection drops, the panel checks the server instead of sending your message twice.
- **Credential cards:** secrets stay in the form and go only to Dewee's respond endpoint. Nothing secret is stored by the extension.
- **Connected browser (self-hosted, opt-in):** turn on *Connected browser* in Settings, register this browser, and allow chosen agents on chosen sites for a limited time. Agents can then open, read, and act in tabs they open themselves, never in your other tabs. Risky clicks wait for your approval, passwords and payment fields are never read or typed, and *Pause*, *Stop* and *Revoke* are always one click away.
- **Connected browser (self-hosted, opt-in):** turn on *Connected browser* in Settings, register this browser, and allow chosen agents on chosen sites for a limited time. Agents can then open, read, and act in tabs they open themselves, never in your other tabs. Risky clicks wait for your approval, passwords and payment fields are never read or typed, and *Pause*, *Stop* and *Revoke* are always one click away. Agent tabs carry an "AI" badge, and using one yourself pauses the agents.
- **Advanced build (sideload, opt-in):** `pnpm build:advanced` adds Chrome's `debugger` permission. After you turn on *Advanced mode* in the browser drawer, agents read pages through the accessibility tree, use real input and can take viewport screenshots, in their own tabs only. It shares the extension ID with the standard build, so install one or the other. See the [threat model](docs/browser-control-threat-model.md#advanced-mode-debugger).
- **Pending badge**, English and Vietnamese, light/dark/system theme, keyboard shortcuts.

## Install (development build)
Expand Down
2 changes: 1 addition & 1 deletion docs/api-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ What the extension adds on its side of the contract:
- Every command is checked again locally before it runs: grant present, not expired or revoked, agent in the grant, capability, exact origin of both the tab's current page and any requested URL, restricted pages (every `about:` page included, so `open about:blank` is refused), owned tab, Chrome host permission for that exact host, and local Pause/Stop.
- `ack` is sent after validation and before any work; exactly one `browser.command.result` follows. A command id is never executed twice while the service worker lives; after a worker restart this relies on the Gateway not resending a dispatched command. Results are not queued across a reconnect; commands in flight when the socket drops are cancelled locally, and the Gateway's deadline rules apply. A command runs for at most 120 s locally, whatever its deadline.
- `type` with `submit`, `press` with `Enter`, and clicks on high-risk elements wait for the person's "Allow once" in the side panel and fail with `approval_denied` otherwise; if the element changed while the person decided, they fail with `stale_ref`.
- `type` is refused (`sensitive_field`) for password, PIN, payment-card and one-time-code fields, detected by input type, `autocomplete` (`cc-*`, `one-time-code`, `current-password`, `new-password`) and name/id hints; the snapshot redacts their values. `screenshot` answers `unsupported_capability` (no advanced build yet).
- `type` is refused (`sensitive_field`) for password, PIN, payment-card and one-time-code fields, detected by input type, `autocomplete` (`cc-*`, `one-time-code`, `current-password`, `new-password`) and name/id hints; the snapshot redacts their values. `screenshot` answers `unsupported_capability` in the Standard build and while Advanced mode is off; in the Advanced build with Advanced mode on it returns a viewport JPEG of at most 1.5 MiB, or `debugger_unavailable` when another debugger holds the tab.
- Grants and devices are read by `grant_id` / `device_id`, also accepting `id`, since v1 does not pin the object shape beyond those fields.
- A command whose grant is revoked while it runs, or that arrives for a revoked grant, fails with `grant_revoked`. Page results (`open`, `navigate`, `snapshot`, `click`, `type`, `screenshot`) always carry the page `url`; a result without a valid one is reported as `internal`, or as `outcome_unknown` after an interaction (the action may have run). A refresh (`attach`, `browser.grant.list`) that no longer lists a grant cancels its in-flight commands, approvals included, with `grant_revoked`.
- The person's last Pause/Stop/Resume is kept per profile and device in `storage.local` and re-sent after every `attach` (`browser.control.state`), so a Chrome restart, a profile switch or a Gateway restart cannot lift a Pause. `active` is re-sent only when the person's last choice was Resume; with no stored choice nothing is sent and the Gateway keeps its state. A stored choice that cannot be read leaves the panel paused locally and sends nothing (see [browser-control-threat-model.md](browser-control-threat-model.md)).
Expand Down
13 changes: 9 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,21 @@ Gateway ──/ws──► service worker
executor.ts policy re-check before every side effect, then the action
policy.ts grant, agent, capability, exact origin, restricted pages, owned tab, host permission
tabs.ts owned tabs t1, t2…, generation per document, single writer, user-input hold
engine-router.ts Standard or debugger per action; refs stay with the engine that issued them
chrome-api.ts ──► chrome.tabs + chrome.scripting.executeScript({ func: pageRun, args })
snapshot.ts (bundled page runtime, isolated world)
side panel ◄─port "browser-control"─ state, approval cards; Pause/Resume/Stop, enroll, grants
adapters/cdp/ ──► chrome.debugger (Advanced build only, opt-in, owned tabs, method whitelist)
tab-marker.ts "AI" badge and title on agent tabs
side panel ◄─port "browser-control"─ state, approval cards, agent tabs, timeline; Pause/Resume/Stop, enroll, grants, Advanced mode
```

- **Lifecycle.** The socket exists only while the "Connected browser" preference is on, the active profile is a self-hosted Gateway and it holds an API key. A profile switch, sign-out or turning the preference off closes it, cancels running commands, denies open approvals and forgets grants and owned tabs. Dewee Cloud shows "not available yet" and opens nothing. A one-minute `chrome.alarms` alarm revives a suspended worker, which reconnects and re-attaches with the stored device id; the 20 s heartbeat keeps it alive while connected (Chrome 116+).
- **Stored.** `storage.local`: the server-generated device id and name, the ids of grants created here and the person's last Pause/Stop/Resume, per profile (not secrets; `storage.local` stays locked to trusted contexts), so Pause/Stop survives a Chrome restart and a profile switch. `storage.session`: owned-tab handles, so a worker restart keeps them. Never stored: the API key outside the credential store, page content, snapshots, typed text, screenshots.
- **Stored.** `storage.local`: the server-generated device id and name, the ids of grants created here and the person's last Pause/Stop/Resume, per profile (not secrets; `storage.local` stays locked to trusted contexts), so Pause/Stop survives a Chrome restart and a profile switch. `storage.session`: owned-tab handles, so a worker restart keeps them, and the Advanced mode opt-in. Never stored: the API key outside the credential store, page content, snapshots, typed text, screenshots.
- **Commands.** Each `browser.command` is validated strictly (unknown keys, actions or keys for `press` are refused), acknowledged, executed and answered with exactly one result. A command id is executed at most once. Results are never queued across a reconnect: a dropped link cancels the commands it carried. An interrupted command that may already have changed the page reports `outcome_unknown`.
- **Pages.** Only `chrome.tabs` and `chrome.scripting.executeScript` with the bundled `pageRun` function (func + args, never a code string), top frame only. Same-origin iframes are read only when their origin is granted; cross-origin iframes are omitted. No `tabs` and no `debugger` permission; screenshots answer `unsupported_capability`.
- **The person stays in charge.** High-risk clicks and `type` with `submit` wait for an "Allow once" card in an open side panel; no panel or a denial gives `approval_denied`. Pause and Stop cancel what is running and refuse new commands (Stop also releases the agent's tabs). A click or key press by the person in an owned tab cancels the agent's command there and holds the tab for 5 s.
- **Pages.** Only `chrome.tabs` and `chrome.scripting.executeScript` with the bundled `pageRun` function (func + args, never a code string), top frame only. Same-origin iframes are read only when their origin is granted; cross-origin iframes are omitted. No `tabs` permission. The Standard build has no `debugger` permission; screenshots answer `unsupported_capability`.
- **Build profiles.** One codebase, two builds. Standard (`pnpm build`) is the default. Advanced (`DEWEE_PROFILE=advanced`, `pnpm build:advanced`) adds the `debugger` permission and the CDP adapter; the `__DEWEE_ADVANCED__` define drops that adapter from Standard bundles. A device enrolls with what the build can do (`["standard"]` or `["standard","advanced"]`) and each attach announces what is usable now: `advanced` only while the person has Advanced mode on.
- **Advanced mode.** Off by default, per browser session, turned on in the drawer. `EngineRouter` sends snapshot, describe, highlight, click, type and press to the CDP adapter (accessibility tree, `Input.dispatch*`) and screenshots only there; scroll, wait, find and tab actions stay on the page script. The adapter attaches to the owned tab when a debugger command starts and detaches when it ends, and on Stop, Pause, revoke, expiry, disconnect, opt-in off and tab close. If attaching fails (DevTools open), it answers `debugger_unavailable`, or uses the Standard path for snapshot and press when the person allowed that fallback. Same-process frames on granted origins are read and driven; cross-site frames are left out and blacked out in screenshots. Details: [browser-control-threat-model.md](browser-control-threat-model.md#advanced-mode-debugger).
- **The person stays in charge.** High-risk clicks and `type` with `submit` wait for an "Allow once" card in an open side panel; no panel or a denial gives `approval_denied`. Pause and Stop cancel what is running and refuse new commands (Stop also releases the agent's tabs). A click or key press by the person in an owned tab cancels the agent's command there, holds the tab for 5 s and pauses control until Resume. Agent tabs carry an "AI" badge; the drawer lists them with agent, site and expiry, shows the active command's agent, channel and site scope, and keeps a short in-memory timeline of outcomes. Before a click the target is highlighted briefly.

## Decisions

Expand Down
Loading
Loading