Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions docs/features/workbot-command-palette/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Workbot: command palette

- Feature key: `workbotCommandPalette`
- Deployments: cloud and self-hosted, off by default
- Author: @benjaminshafii

Members press **Cmd+K** (macOS) or **Ctrl+K** (Windows, Linux), or the search
button in Workbot's header, to search and run what Workbot already does from
its visible controls:

| Group | Entries | Enter | Cmd/Ctrl+Enter |
| --- | --- | --- | --- |
| Actions | Message Workbot, Stop the reply (while answering), New side chat, Switch chat ›, Show background tasks ›, Show files | Run | |
| Chats | Main chat and side chats (four most recent; all when searched) | Open | Open in a new tab |
| Background tasks | Running tasks of this chat (finished ones when searched) | Show the task card | Stop it |
| Go to | Home, Calendar, Documents, Computer (the ones this person has) | Open | |
| Settings | Your dashboard, Your connections (Den pages the header links to) | Open | Open in a new tab |

Recently used entries come first. Tab opens a nested list (Switch chat,
Show background tasks); Backspace on an empty search, Shift+Tab or Esc goes
back. Ctrl+N / Ctrl+P move like the arrow keys. Esc closes and focus returns
where it was (usually the message box).

![The palette open over the conversation](./workbot-command-palette.png)

With the feature off nothing changes: no header button, the shortcut does
nothing, and no extra request is made.

![Feature off: the header without the search button](./workbot-command-palette-off.png)

## How it works

- Den reports `commandPalette` on `GET /v1/workbot/session` from the feature;
Workbot passes it through `GET /v1/workbot/me` (off while Workbot itself is
off) to `WorkbotHost.commandPalette`.
- `ee/packages/workbot-ui/src/command-palette.tsx` renders it with Base UI
`Dialog` and inline `Autocomplete` (combobox, listbox, active descendant,
live result count). Ranking and recents are in `command-palette-search.ts`
(fuzzysort, the desktop palette's approach), tested in
`ee/apps/workbot/scripts/command-palette.test.ts`.
- No new server routes. Chats come from `GET /v1/workbot/chats` (already used
by the Chats drawer), tasks from the conversation already on screen; Stop
calls the task card's existing stop route.
- Recents are kept in this browser's local storage
(`workbot.command-palette.recents`, entry ids only).

## Turning it off

Safe at any moment: the next `/v1/workbot/me` read hides the button and the
shortcut. Nothing is stored on the server, and an open palette simply closes
on the next page load.

## Proof

`evals/proofs/workbot-command-palette.proof.test.ts` (`pnpm evals:e2e
workbot-command-palette --local`) drives the real Workbot page headless:
feature off, Cmd+K from the composer, loading, search with highlights, nested
Chats list, no results, Cmd+Enter stopping a task, Enter opening Calendar,
recents, and a 390px phone.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions ee/apps/den-api/src/workbot/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ const sessionSchema = z.object({
activationChecklist: z.boolean().optional(),
/** The person can start side chats next to their main chat (while Workbot is on). */
sideChats: z.boolean(),
/** The Cmd+K / Ctrl+K command palette (`workbotCommandPalette`); older Dens omit it: off. */
commandPalette: z.boolean().optional(),
/**
* The model Workbot's turns run on: the organization's default model. Null, and
* omitted by older Dens, means the headless runner's default.
Expand Down Expand Up @@ -400,6 +402,7 @@ export function registerWorkbotRoutes<T extends { Variables: object }>(app: Hono
usageRecovery: includedModels,
activationChecklist: features.activationChecklist,
sideChats: true,
commandPalette: features.workbotCommandPalette,
model,
gatewayModels: true,
stepTurns: true,
Expand Down
3 changes: 2 additions & 1 deletion ee/apps/workbot/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
"test:events": "tsx --conditions=development --test scripts/events.test.ts",
"test:steps": "tsx --conditions=development --test scripts/steps.test.ts",
"test:session-cells": "tsx --conditions=development --test scripts/session-cells.test.ts",
"test:activation": "tsx --conditions=development --test scripts/activation.test.ts"
"test:activation": "tsx --conditions=development --test scripts/activation.test.ts",
"test:command-palette": "tsx --conditions=development --test scripts/command-palette.test.ts"
},
"dependencies": {
"@hono/node-server": "2.0.12",
Expand Down
80 changes: 80 additions & 0 deletions ee/apps/workbot/scripts/command-palette.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
import assert from "node:assert/strict"
import { test } from "node:test"
import { Hono } from "hono"
import { highlightRuns, loadPaletteRecents, matchIndexes, rankPaletteItems, recordPaletteRecent, SEARCH_LIMIT, type Rankable } from "../../../packages/workbot-ui/src/command-palette-search.js"
import { createAuth, type AppEnv } from "../src/server/auth.js"
import { createDen, denSessionSchema } from "../src/server/den.js"
import { loadConfig } from "../src/server/config.js"
import { registerWorkbotRoutes } from "../src/server/routes.js"

const PUBLIC = "http://localhost:3090"

async function me(session: Record<string, unknown>) {
const config = loadConfig({ WORKBOT_PUBLIC_URL: PUBLIC, WORKBOT_DEN_API_URL: "http://localhost:3091", WORKBOT_RUNNER_URL: "http://localhost:3092", WORKBOT_RUNNER_TOKEN: "r".repeat(32), WORKBOT_SESSION_SECRET: "s".repeat(32) })
const den = createDen({ apiUrl: config.denApiUrl, publicUrl: config.publicUrl })
const app = new Hono<AppEnv>()
registerWorkbotRoutes(app, { config, den, sameOrigin: createAuth({ config, den }).sameOrigin, member: async (c, next) => {
const who = denSessionSchema.parse({ user: { id: "user_example", name: "Example", email: "member@example.test" }, organization: { id: "org_example", name: "Example workspace", brandAppName: null }, memberId: "member_example", canSchedule: false, ...session })
c.set("member", { session: { id: "session_example", userId: who.user.id, organizationId: who.organization.id, tokens: { accessToken: "fixture-token", expiresAt: Date.now() + 60000 }, createdAt: Date.now() }, accessToken: "fixture-token", den: who })
await next()
} })
const body: unknown = await (await app.request(`${PUBLIC}/v1/workbot/me`, { headers: { origin: PUBLIC } })).json()
assert.ok(typeof body === "object" && body !== null && "commandPalette" in body)
return body.commandPalette
}

test("the page sees the palette only when Den turns it on for a member Workbot is on for", async () => {
assert.equal(await me({ enabled: true }), false, "a Den from before the feature says nothing: off")
assert.equal(await me({ enabled: true, commandPalette: false }), false)
assert.equal(await me({ enabled: false, commandPalette: true }), false, "never while Workbot itself is off")
assert.equal(await me({ enabled: true, commandPalette: true }), true)
})

const items: Rankable[] = [
{ id: "action:new-side-chat", group: "actions", title: "New side chat", keywords: "start create conversation topic" },
{ id: "action:tasks", group: "actions", title: "Show background tasks", keywords: "jobs work progress" },
{ id: "chat:budget", group: "chats", title: "Budget review", keywords: "side chat conversation" },
{ id: "chat:launch", group: "chats", title: "Launch plan", keywords: "side chat conversation" },
{ id: "chat:old", group: "chats", title: "Old notes", keywords: "side chat conversation", preview: false },
{ id: "nav:calendar", group: "navigation", title: "Calendar", keywords: "week meetings schedule automations" },
{ id: "settings:connections", group: "settings", title: "Your connections", keywords: "apps google microsoft calendar connect" },
]

test("with nothing typed, recent entries come first, then each group in order, without search-only entries", () => {
const groups = rankPaletteItems("", items, ["nav:calendar", "gone"])
assert.deepEqual(groups.map((group) => group.value), ["recent", "actions", "chats", "settings"])
assert.deepEqual(groups[0]?.items.map((item) => item.id), ["nav:calendar"], "a recent id that no longer exists is skipped")
assert.ok(!groups.flatMap((group) => group.items).some((item) => item.id === "chat:old"), "older chats appear only when searched")
})

test("search ranks a title match above a keyword match, leads with the best group and finds search-only entries", () => {
const calendar = rankPaletteItems("calendar", items, [])
assert.equal(calendar[0]?.items[0]?.id, "nav:calendar", "the Calendar page beats the connections row that mentions calendars")
assert.equal(calendar[0]?.value, "navigation")
assert.equal(rankPaletteItems("old", items, [])[0]?.items[0]?.id, "chat:old")
assert.deepEqual(rankPaletteItems("xyzzy", items, []), [])
assert.equal(rankPaletteItems("bud", items, [])[0]?.items[0]?.id, "chat:budget")
})

test("a recently used entry wins a tie, and a long list stays short", () => {
const twins: Rankable[] = [{ id: "a", group: "chats", title: "Plan" }, { id: "b", group: "chats", title: "Plan" }]
assert.equal(rankPaletteItems("plan", twins, ["b"])[0]?.items[0]?.id, "b")
const many: Rankable[] = Array.from({ length: 500 }, (_, index) => ({ id: `chat:${index}`, group: "chats", title: `Weekly report ${index}` }))
assert.equal(rankPaletteItems("weekly", many, []).flatMap((group) => group.items).length, SEARCH_LIMIT)
})

test("matched letters are emphasized in order, as plain text runs", () => {
assert.deepEqual(highlightRuns("Budget review", matchIndexes("bud", "Budget review")), [{ text: "Bud", match: true }, { text: "get review", match: false }])
assert.deepEqual(highlightRuns("Calendar", matchIndexes("", "Calendar")), [{ text: "Calendar", match: false }])
})

test("recents keep the newest eight, once each, and survive unreadable storage", () => {
const store = new Map<string, string>()
const storage = { getItem: (key: string) => store.get(key) ?? null, setItem: (key: string, value: string) => { store.set(key, value) } }
for (const id of ["a", "b", "c", "a", "d", "e", "f", "g", "h", "i"]) recordPaletteRecent(id, storage)
assert.deepEqual(loadPaletteRecents(storage), ["i", "h", "g", "f", "e", "d", "a", "c"])
store.set("workbot.command-palette.recents", "{not json")
assert.deepEqual(loadPaletteRecents(storage), [])
const failing = { getItem: () => null, setItem: () => { throw new Error("quota") } }
assert.deepEqual(recordPaletteRecent("x", failing), ["x"], "running an action never depends on storage")
})
3 changes: 3 additions & 0 deletions ee/apps/workbot/src/client/host.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ export const meSchema = z.object({
canSchedule: z.boolean().default(false),
/** Side chats are on for this person (while Workbot is on). */
sideChats: z.boolean().default(false),
/** The Cmd+K / Ctrl+K command palette; older servers omit it. */
commandPalette: z.boolean().default(false),
denUrl: z.string().nullable(),
})
export type Me = z.infer<typeof meSchema>
Expand Down Expand Up @@ -106,6 +108,7 @@ export function createHost(me: Me, recovery: { authorization?: WorkbotHost["docu
&& current.userId === me.userId && current.organizationId === me.organizationId
},
sideChats: me.sideChats,
commandPalette: me.commandPalette,
identity: me.userId && me.organizationId ? { principalId: me.userId, organizationId: me.organizationId } : undefined,
}
}
2 changes: 2 additions & 0 deletions ee/apps/workbot/src/server/den.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ export const denSessionSchema = z.object({
activationChecklist: z.boolean().default(false),
/** Side chats are on for this person. A Den from before side chats doesn't say: off. */
sideChats: z.boolean().default(false),
/** The Cmd+K / Ctrl+K command palette. A Den from before it doesn't say: off. */
commandPalette: z.boolean().default(false),
/** The organization's default model for Workbot's turns. Null, or a Den from before it, means the runner's default. */
model: z.string().nullable().default(null),
verifyModel: z.boolean().default(false),
Expand Down
1 change: 1 addition & 0 deletions ee/apps/workbot/src/server/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ export function registerWorkbotRoutes(app: Hono<AppEnv>, input: { config: Config
activationChecklist: who.enabled && who.activationChecklist,
canSchedule: who.enabled && who.canSchedule,
sideChats: who.enabled && who.sideChats,
commandPalette: who.enabled && who.commandPalette,
denUrl: config.denWebUrl ?? (await den.webUrl().catch(() => null)),
})
})
Expand Down
5 changes: 3 additions & 2 deletions ee/packages/workbot-ui/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@
"@base-ui/react": "^1.4.1",
"@novnc/novnc": "1.7.0",
"@openwork/calendar": "workspace:*",
"@openwork/documents": "workspace:*",
"@openwork/types": "workspace:*",
"@openwork/ui": "workspace:*",
"@openwork/workbook": "workspace:*",
"fuzzysort": "^3.1.0",
"lucide-react": "^0.577.0",
"marked": "^17.0.1",
"zod": "^4.6.6",
"@openwork/documents": "workspace:*"
"zod": "^4.6.6"
},
"peerDependencies": {
"@tanstack/react-query": "^5.96.2",
Expand Down
145 changes: 145 additions & 0 deletions ee/packages/workbot-ui/src/command-palette-search.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
import fuzzysort from "fuzzysort";

/**
* Ranking for Workbot's command palette (feature `workbotCommandPalette`): pure, so it is tested without a page.
* The same idea as the desktop palette's search (apps/app/src/react-app/shell/command-palette-search.ts): fuzzy
* match on the title first, then keywords and detail; recently used entries win ties.
*/

export type PaletteGroupId = "recent" | "actions" | "chats" | "tasks" | "navigation" | "settings";

export const PALETTE_GROUP_LABELS: Record<PaletteGroupId, string> = {
recent: "Recent",
actions: "Actions",
chats: "Chats",
tasks: "Background tasks",
navigation: "Go to",
settings: "Settings",
};

const GROUP_ORDER: readonly PaletteGroupId[] = ["actions", "chats", "tasks", "navigation", "settings"];

/** What the ranking needs from an entry. */
export type Rankable = {
id: string;
group: Exclude<PaletteGroupId, "recent">;
title: string;
detail?: string;
keywords?: string;
/** Shown when nothing is typed. Entries that only appear when searched (every chat, older tasks) set false. */
preview?: boolean;
};

export type RankedGroup<T extends Rankable> = { value: PaletteGroupId; label: string; items: T[] };

/** The most entries a search lists, so a person with hundreds of chats still gets an instant, short list. */
export const SEARCH_LIMIT = 50;
const RECENT_LIMIT = 5;

function group<T extends Rankable>(value: PaletteGroupId, items: T[]): RankedGroup<T> {
return { value, label: PALETTE_GROUP_LABELS[value], items };
}

function normalize(value: string) {
return value.trim().toLowerCase().replace(/\s+/g, " ");
}

export function rankPaletteItems<T extends Rankable>(query: string, items: readonly T[], recentIds: readonly string[]): RankedGroup<T>[] {
const trimmed = query.trim();
if (!trimmed) {
const byId = new Map(items.map((item) => [item.id, item]));
const recent = recentIds.flatMap((id) => {
const item = byId.get(id);
return item ? [item] : [];
}).slice(0, RECENT_LIMIT);
const shown = new Set(recent.map((item) => item.id));
return [
group("recent", recent),
...GROUP_ORDER.map((value) => group(value, items.filter((item) => item.group === value && item.preview !== false && !shown.has(item.id)))),
].filter((entry) => entry.items.length > 0);
}

const wanted = normalize(trimmed);
const tokens = wanted.split(" ");
const haystack = (item: T) => [item.title, item.keywords, item.detail].filter(Boolean).join(" ");
// Every word has to match somewhere, so "side bud" finds the "Budget" side chat but not every side chat.
const candidates = items.filter((item) => tokens.every((token) => fuzzysort.single(token, haystack(item)) !== null));
const recent = new Set(recentIds);
const ranked = fuzzysort.go(trimmed, candidates, {
keys: [(item) => item.title, (item) => item.keywords ?? "", (item) => item.detail ?? ""],
threshold: 0.3,
limit: SEARCH_LIMIT,
scoreFn: (result) => {
const score = Math.max(result[0].score * 2, result[1].score, result[2].score * 0.7);
const title = normalize(result.obj.title);
const tier = title.startsWith(wanted) ? 4 : title.includes(wanted) ? 3 : normalize(result.obj.keywords ?? "").includes(wanted) ? 2 : 1;
return tier * 10 + (recent.has(result.obj.id) ? score * 1.1 : score);
},
}).map((result) => result.obj);
// Tokens that matched across fields but not as one phrase still count, after the phrase matches.
const extra = tokens.length > 1 ? candidates.filter((item) => !ranked.includes(item)).slice(0, Math.max(0, SEARCH_LIMIT - ranked.length)) : [];
const ordered = [...ranked, ...extra];
const best = ordered[0]?.group;
const order = best ? [best, ...GROUP_ORDER.filter((value) => value !== best)] : GROUP_ORDER;
return order.flatMap((value) => {
const entries = ordered.filter((item) => item.group === value);
return entries.length > 0 ? [group(value, entries)] : [];
});
}

/** Character positions of `query` in `text` to emphasize, or none. */
export function matchIndexes(query: string, text: string): ReadonlySet<number> {
const trimmed = query.trim();
if (!trimmed) return new Set();
const whole = fuzzysort.single(trimmed, text);
if (whole) return new Set(whole.indexes);
const marked = new Set<number>();
for (const token of normalize(trimmed).split(" ")) for (const index of fuzzysort.single(token, text)?.indexes ?? []) marked.add(index);
return marked;
}

/** Split `text` into runs, each matched or not, for rendering emphasis without innerHTML. */
export function highlightRuns(text: string, marked: ReadonlySet<number>): Array<{ text: string; match: boolean }> {
const runs: Array<{ text: string; match: boolean }> = [];
for (let index = 0; index < text.length; index++) {
const match = marked.has(index);
const last = runs.at(-1);
if (last && last.match === match) last.text += text[index];
else runs.push({ text: text[index] ?? "", match });
}
return runs;
}

export const COMMAND_PALETTE_RECENTS_KEY = "workbot.command-palette.recents";
const RECENTS_KEPT = 8;

type RecentsStorage = Pick<Storage, "getItem" | "setItem">;

function browserStorage(): RecentsStorage | undefined {
try {
return typeof window === "undefined" ? undefined : window.localStorage;
} catch {
return undefined;
}
}

export function loadPaletteRecents(storage: RecentsStorage | undefined = browserStorage()): string[] {
if (!storage) return [];
try {
const parsed: unknown = JSON.parse(storage.getItem(COMMAND_PALETTE_RECENTS_KEY) ?? "[]");
if (!Array.isArray(parsed)) return [];
return [...new Set(parsed.filter((id): id is string => typeof id === "string"))].slice(0, RECENTS_KEPT);
} catch {
return [];
}
}

export function recordPaletteRecent(id: string, storage: RecentsStorage | undefined = browserStorage()): string[] {
const next = [id, ...loadPaletteRecents(storage).filter((entry) => entry !== id)].slice(0, RECENTS_KEPT);
try {
storage?.setItem(COMMAND_PALETTE_RECENTS_KEY, JSON.stringify(next));
} catch {
// Storage off or full: the action still runs; it just isn't remembered.
}
return next;
}
Loading
Loading