Skip to content

Commit cfda9da

Browse files
feat(void): support custom Workers AI bindings
1 parent 53e5e4a commit cfda9da

14 files changed

Lines changed: 151 additions & 61 deletions

File tree

‎docs/integrations/cloudflare.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -98,12 +98,12 @@ Bindings are [inferred automatically](../reference/resource-inference.md) by sca
9898
```json
9999
{
100100
"inference": {
101-
"bindings": { "db": true, "kv": true, "storage": false, "ai": true }
101+
"bindings": { "db": true, "kv": true, "storage": false, "ai": "MY_AI" }
102102
}
103103
}
104104
```
105105

106-
`db`, `kv`, and `storage` accept a string to customize the binding name (e.g. `"db": "MY_DB"`). `ai` only accepts a boolean.
106+
Every binding accepts a string to customize its name (for example, `"db": "MY_DB"` or `"ai": "MY_AI"`).
107107

108108
See [Configuration](../reference/config.md) for details.
109109

@@ -349,7 +349,7 @@ That's it. The Cloudflare Vite plugin produces a complete build output with a me
349349

350350
`void/ai` works on your own Cloudflare account, along two paths:
351351

352-
- **Workers AI** (`ai.run`, `ai.stream`, `ai.image`) works out of the box. When your app imports `void/ai`, `vite build` infers that you need AI and adds a Workers AI binding (`env.AI`) to the generated `wrangler.json` automatically -- you do **not** add it to `wrangler.jsonc`.
352+
- **Workers AI** (`ai.run`, `ai.stream`, `ai.image`) works out of the box. When your app imports `void/ai`, `vite build` infers that you need AI and adds a Workers AI binding (`env.AI`) to the generated `wrangler.json` automatically -- you do **not** add it to `wrangler.jsonc`. Set `inference.bindings.ai` to a string to use a custom name.
353353
- **Provider models** (`ai.provider("openai").fetch(...)`) route through _your own_ Cloudflare AI Gateway. Set its id in `void.json` and add the provider's API key as a Worker secret.
354354

355355
```jsonc
@@ -372,7 +372,7 @@ import { defineHandler } from 'void';
372372
import { ai } from 'void/ai';
373373

374374
export const POST = defineHandler(async (c) => {
375-
// Workers AI -- uses env.AI directly
375+
// Workers AI -- uses the inferred or configured AI binding directly
376376
const summary = await ai.run('@cf/meta/llama-3.1-8b-instruct', {
377377
prompt: 'Summarize the changelog.',
378378
});
@@ -393,7 +393,7 @@ export const POST = defineHandler(async (c) => {
393393
Notes:
394394

395395
- **No cross-tenant usage metering.** On the Void platform, AI calls are metered and billed through a shared proxy. Self-hosted, there is no metering -- you get your own [Cloudflare AI Gateway analytics](https://developers.cloudflare.com/ai-gateway/) instead.
396-
- **The runtime reads the `env.AI` binding by name** -- a custom-named root AI binding is not supported.
396+
- **Custom AI binding names are supported.** Configure `"inference": { "bindings": { "ai": "MY_AI" } }`, or declare the custom `ai.binding` in your root Wrangler config; the generated worker records the resolved name for `void/ai` automatically.
397397
- **`void dev` has no local Workers AI emulation.** In development, AI still routes through Void, so `void dev` AI requires `void auth login` even when you deploy self-hosted.
398398

399399
### ISR (self-host)

‎docs/reference/config.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -422,17 +422,17 @@ Explicitly control inferred Cloudflare bindings. If omitted, bindings are [autom
422422
Each binding accepts `true` (use default name), `false` (disable), or a string (custom binding name):
423423

424424
```json
425-
{ "inference": { "bindings": { "db": true, "kv": false, "storage": "MY_BUCKET" } } }
425+
{ "inference": { "bindings": { "db": true, "kv": false, "storage": "MY_BUCKET", "ai": "MY_AI" } } }
426426
```
427427

428428
| Key | Default Binding | Type | Custom Name |
429429
| --------- | --------------- | ------------- | ------------------------ |
430430
| `db` | `DB` | `D1Database` | `"db": "MY_DB"` |
431431
| `kv` | `KV` | `KVNamespace` | `"kv": "MY_KV"` |
432432
| `storage` | `STORAGE` | `R2Bucket` | `"storage": "MY_BUCKET"` |
433-
| `ai` | `AI` | `Ai` | Boolean only |
433+
| `ai` | `AI` | `Ai` | `"ai": "MY_AI"` |
434434

435-
Custom names are used in the deploy manifest, Wrangler config generation/sync, remote binding proxying, internal migrations, and generated runtime helpers such as `void/db`, `void/kv`, and `void/storage`.
435+
Custom names are used in generated Wrangler config and runtime helpers such as `void/db`, `void/kv`, `void/storage`, and `void/ai`. Resource binding names also flow through deploy manifests, remote binding proxying, and internal migrations where applicable. Managed Workers AI continues to use Void's service proxy.
436436

437437
#### `inference.build`
438438

‎packages/void/CLAUDE.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,8 @@ src/
9393

9494
Two features need extra wiring on a self-hosted `vite build && wrangler deploy` (no Void platform proxy/dispatch in front of the worker). Both are inert on managed `void deploy`.
9595

96-
- **`ai.gateway` config + `__VOID_AI_GATEWAY` var** — `void.json`'s `ai.gateway` (validated in `config.ts` as `ai?: { gateway?: string }`; non-empty string, `additionalProperties: false`) names a Cloudflare AI Gateway in the user's own account. `mergeBindings` (`index.ts`): (1) injects `{ ai: { binding: 'AI' } }`, but only when `shouldBindWorkersAi()` is true — the app infers `needsAI`, `command === 'build'`, and neither the resolved root wrangler config nor the result already declares an `ai` binding (so it never clobbers a user's own `ai` binding); (2) emits `vars.__VOID_AI_GATEWAY = config.ai.gateway` whenever `config.ai?.gateway` is set. At runtime, `runtime/ai.ts` `resolveBackend()` selects the direct backend (Priority 3) when `env.AI` is present and no `__VOID_PROXY`/`__VOID_TOKEN` exists. `ai.run/stream/image` hit Workers AI directly; `ai.provider().fetch()` routes through the user's own gateway via `directAi.gateway(__VOID_AI_GATEWAY).getUrl(provider)` with the provider key from the user's own secret (`resolveProviderKey` → `env[PROVIDER_KEY_MAP[provider]]`), returning HTTP 501 if `__VOID_AI_GATEWAY` is unset. No cross-tenant metering on this path. The runtime reads `env.AI` **by name** — a custom-named root AI binding is not honored.
97-
- **Internal-binding leak check permits exactly `__VOID_AI_GATEWAY`** — the integration guard (`test/integration/kitchen-sink-build.test.ts`) that asserts no `__VOID_*` var reaches `dist/ssr/wrangler.json` explicitly excludes `__VOID_AI_GATEWAY`, because it is a legitimate, non-credential runtime var (unlike the credential keys in `INTERNAL_ENV_KEYS`, which are stripped).
96+
- **`ai.gateway`, custom AI bindings, and private runtime vars** — `void.json`'s `ai.gateway` (validated in `config.ts` as `ai?: { gateway?: string }`; non-empty string, `additionalProperties: false`) names a Cloudflare AI Gateway in the user's own account. `inference.bindings.ai` accepts a string custom binding name. `mergeBindings` (`index.ts`) injects the resolved binding only when `shouldBindWorkersAi()` is true — the app infers `needsAI`, `command === 'build'`, and neither the resolved root wrangler config nor the result already declares an `ai` binding — so a root Wrangler declaration stays authoritative. It emits `vars.__VOID_AI_BINDING` for a non-default binding and `vars.__VOID_AI_GATEWAY` when configured. At runtime, `runtime/ai.ts` `resolveBackend()` uses that resolved binding when no `__VOID_PROXY`/`__VOID_TOKEN` exists. `ai.run/stream/image` hit Workers AI directly; `ai.provider().fetch()` routes through the user's own gateway via `directAi.gateway(__VOID_AI_GATEWAY).getUrl(provider)` with the provider key from the user's own secret (`resolveProviderKey` → `env[PROVIDER_KEY_MAP[provider]]`), returning HTTP 501 if `__VOID_AI_GATEWAY` is unset. No cross-tenant metering on this path.
97+
- **Internal-binding leak check permits the AI routing markers** — the integration guard (`test/integration/kitchen-sink-build.test.ts`) that asserts no credential-bearing `__VOID_*` var reaches `dist/ssr/wrangler.json` permits `__VOID_AI_BINDING` and `__VOID_AI_GATEWAY`, because they are legitimate, non-credential runtime routing values (unlike the credential keys in `INTERNAL_ENV_KEYS`, which are stripped).
9898
- **`ISR_CACHE` self-host requirement** — self-host ISR (`runtime/isr-cache.ts`, `runtime/isr.ts`) reads `env.ISR_CACHE` (a `KVNamespace`) + `env.__VOID_ISR_DEPLOYMENT_ID`. Unlike D1/KV/R2, `ISR_CACHE` is **not** auto-injected (a KV binding needs a real namespace id), so the user must create the namespace and declare it as `ISR_CACHE` in their root wrangler config. `routing.revalidate` (void.json) and per-page `export const revalidate` are merged by `collectRevalidate` (`isr-config.ts`) and baked as `__VOID_ISR_REVALIDATE`. On-demand `revalidate()` on self-host purges the worker's own KV (global) plus the local-colo `caches.default` edge variants only.
9999

100100
### Package Exports

‎packages/void/schema.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -257,8 +257,8 @@
257257
"description": "Enable R2 bucket binding. `true` uses default name \"STORAGE\", a string sets a custom binding name."
258258
},
259259
"ai": {
260-
"type": "boolean",
261-
"description": "Enable Workers AI binding"
260+
"oneOf": [{ "type": "boolean" }, { "type": "string" }],
261+
"description": "Enable Workers AI binding. `true` uses default name \"AI\", a string sets a custom binding name."
262262
}
263263
},
264264
"additionalProperties": false

‎packages/void/src/cli/deploy.ts‎

Lines changed: 21 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -883,8 +883,8 @@ const DEPLOY_RAW_SNAPSHOT_FIELDS: ReadonlyArray<string> = [
883883
'migrations',
884884
];
885885

886-
/** Void's own injected Workers AI binding shape (`mergeBindings` in index.ts). */
887-
const VOID_EMITTED_AI_BINDING = { binding: 'AI' };
886+
/** Void's default injected Workers AI binding name (`mergeBindings` in index.ts). */
887+
const DEFAULT_VOID_AI_BINDING = 'AI';
888888

889889
/**
890890
* Sub-fields on a `durable_objects` binding that make it target ANOTHER worker's Durable Objects.
@@ -1565,12 +1565,14 @@ export function verifyGeneratedDeployConfig(
15651565
passthroughRawFields?: Map<string, unknown>;
15661566
/**
15671567
* Whether the pre-build inference found Workers AI use in the source (the SAME `needsAI`
1568-
* signal `shouldBindWorkersAi` gates on in index.ts). Authorizes an emitted `{binding:'AI'}`
1569-
* ai binding when the root config declares no ai binding: only an AI-using app legitimately
1570-
* receives Void's injected Workers AI binding, so a build that injects one into a non-AI app
1571-
* (billable inference the source never asked for) fails closed.
1568+
* signal `shouldBindWorkersAi` gates on in index.ts). Authorizes Void's emitted AI binding
1569+
* (default `AI`, or `aiBindingName`) when the root config declares none: only an AI-using app
1570+
* legitimately receives one, so a build that injects it into a non-AI app (billable inference
1571+
* the source never asked for) fails closed.
15721572
*/
15731573
needsAI?: boolean;
1574+
/** Custom `inference.bindings.ai` name authorized for Void's emitted AI binding. */
1575+
aiBindingName?: string;
15741576
/**
15751577
* The exact `containers` set Void's build emits, reconstructed PRE-BUILD
15761578
* ({@link reconstructExpectedContainers}) from the root-declared containers plus — when the
@@ -1812,8 +1814,8 @@ export function verifyGeneratedDeployConfig(
18121814
// • `ai` — when the root declared its own ai binding it must SURVIVE byte-identical (a build
18131815
// that REMOVED or rewrote it is rejected); otherwise the emitted binding must be ABSENT or —
18141816
// ONLY when the source's pre-build inference says the app uses Workers AI — Void's own
1815-
// `{binding:'AI'}`. An injected `{binding:'AI'}` on an app that does not use AI would incur
1816-
// account-billed inference from build-injected worker code.
1817+
// its configured Workers AI binding. An injected AI binding on an app that does not use AI
1818+
// would incur account-billed inference from build-injected worker code.
18171819
// • `migrations` — an emitted Durable Object migration carrying a DESTRUCTIVE key (deleting,
18181820
// renaming, or transferring classes) must match a root-declared migration byte-for-byte;
18191821
// Void emits only creation-key migrations, so a destructive one is a build injection that
@@ -1857,10 +1859,10 @@ export function verifyGeneratedDeployConfig(
18571859
};
18581860
}
18591861
} else if (emittedAi !== undefined) {
1860-
// No root ai binding: the only ai binding Void's own build emits is `{binding:'AI'}`, and
1861-
// ONLY when the pre-build inference says the source uses Workers AI (the SAME signal
1862-
// `shouldBindWorkersAi` gates on). Anything else here is a build-injected billable binding.
1863-
if (!(expected.needsAI === true && deepEqual(emittedAi, VOID_EMITTED_AI_BINDING))) {
1862+
// No root ai binding: Void emits the configured binding name (default `AI`) ONLY when
1863+
// pre-build inference says the source uses Workers AI. Anything else is build-injected.
1864+
const expectedAi = { binding: expected.aiBindingName ?? DEFAULT_VOID_AI_BINDING };
1865+
if (!(expected.needsAI === true && deepEqual(emittedAi, expectedAi))) {
18641866
return {
18651867
ok: false,
18661868
reason: `generated config declares an ai binding the build injected — the pre-build inference found no Workers AI use in the source and the root config declares no ai binding.`,
@@ -2481,10 +2483,12 @@ export async function computeDesiredCloudflareResources(
24812483
migrations: Array<{ name: string; sql: string }>;
24822484
/**
24832485
* Whether the pre-build inference found Workers AI use in the source — the SAME `needsAI`
2484-
* signal `shouldBindWorkersAi` (index.ts) gates the emitted `{binding:'AI'}` on. Threaded
2485-
* into {@link verifyGeneratedDeployConfig} to authorize that emitted ai binding.
2486+
* signal `shouldBindWorkersAi` (index.ts) gates the emitted AI binding on. Threaded into
2487+
* {@link verifyGeneratedDeployConfig} to authorize that binding.
24862488
*/
24872489
needsAI: boolean;
2490+
/** Workers AI binding emitted by the build (default `AI`). */
2491+
aiBindingName: string;
24882492
/**
24892493
* Whether Void's build emits the sandbox container application — the SAME signal `mergeBindings`
24902494
* (index.ts) gates emission on: `isSandboxEnabled(config)` (a `sandbox` field in `void.json`) OR
@@ -2642,6 +2646,7 @@ export async function computeDesiredCloudflareResources(
26422646
migrationsDir: paths.migrationsDir,
26432647
migrations,
26442648
needsAI: bindings.needsAI,
2649+
aiBindingName: bindingNames?.ai ?? DEFAULT_VOID_AI_BINDING,
26452650
expectedCrons,
26462651
// Mirror index.ts's emission signal EXACTLY (line 658): the sandbox container is emitted when
26472652
// `void.json` enables it OR the source imports `void/sandbox` (the import-walk signal). Using
@@ -3625,6 +3630,7 @@ export async function runCloudflareDeploy(
36253630
migrationsDir,
36263631
migrations: preBuildMigrations,
36273632
needsAI,
3633+
aiBindingName,
36283634
needsSandbox,
36293635
expectedCrons,
36303636
} = await computeDesiredCloudflareResources(root, voidConfig, configPath);
@@ -4550,6 +4556,7 @@ export async function runCloudflareDeploy(
45504556
passthroughIdentitySnapshot,
45514557
passthroughRawFields,
45524558
needsAI,
4559+
aiBindingName,
45534560
expectedContainers,
45544561
expectedCrons,
45554562
});

‎packages/void/src/config.ts‎

Lines changed: 11 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ export type InferenceConfig = {
6363
db?: boolean | string;
6464
kv?: boolean | string;
6565
storage?: boolean | string;
66-
ai?: boolean;
66+
ai?: boolean | string;
6767
};
6868
scanDirs?: ReadonlyArray<string>;
6969
build?: string;
@@ -762,18 +762,10 @@ export function readConfig(root: string): VoidConfig {
762762
`config: Unknown binding '${key}' in ${CONFIG_FILE}. Valid: db, kv, storage, ai.`,
763763
);
764764
}
765-
if (key === 'ai') {
766-
if (typeof b[key] !== 'boolean') {
767-
throw new Error(
768-
`config: "inference.bindings.${key}" in ${CONFIG_FILE} must be a boolean.`,
769-
);
770-
}
771-
} else {
772-
if (typeof b[key] !== 'boolean' && typeof b[key] !== 'string') {
773-
throw new Error(
774-
`config: "inference.bindings.${key}" in ${CONFIG_FILE} must be a boolean or a string custom binding name.`,
775-
);
776-
}
765+
if (typeof b[key] !== 'boolean' && typeof b[key] !== 'string') {
766+
throw new Error(
767+
`config: "inference.bindings.${key}" in ${CONFIG_FILE} must be a boolean or a string custom binding name.`,
768+
);
777769
}
778770
}
779771
}
@@ -988,11 +980,11 @@ export function isNodeTarget(target: VoidConfig['target']): boolean {
988980
*/
989981
export function resolveBindingNames(
990982
bindings: InferenceConfig['bindings'],
991-
): { d1?: string; kv?: string; r2?: string } | undefined {
983+
): { d1?: string; kv?: string; r2?: string; ai?: string } | undefined {
992984
if (!bindings) {
993985
return undefined;
994986
}
995-
const names: { d1?: string; kv?: string; r2?: string } = {};
987+
const names: { d1?: string; kv?: string; r2?: string; ai?: string } = {};
996988
let hasCustom = false;
997989
if (typeof bindings.db === 'string') {
998990
names.d1 = bindings.db;
@@ -1006,6 +998,10 @@ export function resolveBindingNames(
1006998
names.r2 = bindings.storage;
1007999
hasCustom = true;
10081000
}
1001+
if (typeof bindings.ai === 'string') {
1002+
names.ai = bindings.ai;
1003+
hasCustom = true;
1004+
}
10091005
return hasCustom ? names : undefined;
10101006
}
10111007

‎packages/void/src/index.ts‎

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,17 @@ export function shouldBindWorkersAi(
384384
return needsAI && command === 'build' && !resultAi && !resolvedAi;
385385
}
386386

387+
/** Resolve the runtime Workers AI binding, preserving a root Wrangler declaration. */
388+
export function resolveWorkersAiBindingName(resolvedAi: unknown, configuredName?: string): string {
389+
if (typeof resolvedAi === 'object' && resolvedAi !== null) {
390+
const binding = (resolvedAi as { binding?: unknown }).binding;
391+
if (typeof binding === 'string') {
392+
return binding;
393+
}
394+
}
395+
return configuredName ?? 'AI';
396+
}
397+
387398
const CLOUDFLARE_BUILTIN_IMPORT = /^cloudflare:/;
388399
const SANDBOX_SDK_IMPORT = '@cloudflare/sandbox';
389400
const SANDBOX_SDK_IMPORT_FILTER = /^@cloudflare\/sandbox$/;
@@ -1051,22 +1062,29 @@ export function voidPlugin(options?: {
10511062
// wins, so `!resolvedAi` is load-bearing (prevents clobbering a user's own
10521063
// root-wrangler `ai` binding); `!resultAi` is harmless defense-in-depth
10531064
// (nothing else in mergeBindings ever sets `result.ai`).
1065+
const resolvedAi = (resolved as { ai?: unknown }).ai;
1066+
const aiBindingName = resolveWorkersAiBindingName(resolvedAi, bindingNames?.ai);
10541067
if (
10551068
shouldBindWorkersAi(
10561069
viteCommand,
10571070
bindings.needsAI,
1058-
(resolved as { ai?: unknown }).ai,
1071+
resolvedAi,
10591072
(result as { ai?: unknown }).ai,
10601073
)
10611074
) {
1062-
(result as Record<string, unknown>).ai = { binding: 'AI' };
1075+
(result as Record<string, unknown>).ai = { binding: aiBindingName };
10631076
}
10641077

1065-
// AI Gateway id → runtime var (harmless in serve; only read by the P3 direct
1066-
// backend that routes Workers AI inference through the user's own Gateway).
1067-
if (config.ai?.gateway) {
1078+
// AI binding name / Gateway id → runtime vars (harmless in serve; only read
1079+
// by the P3 direct backend). The default binding needs no extra variable.
1080+
if (aiBindingName !== 'AI' || config.ai?.gateway) {
10681081
const vars = (result.vars as Record<string, string>) ?? {};
1069-
vars.__VOID_AI_GATEWAY = config.ai.gateway;
1082+
if (aiBindingName !== 'AI') {
1083+
vars.__VOID_AI_BINDING = aiBindingName;
1084+
}
1085+
if (config.ai?.gateway) {
1086+
vars.__VOID_AI_GATEWAY = config.ai.gateway;
1087+
}
10701088
result.vars = vars;
10711089
}
10721090

‎packages/void/src/plugin-inference.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ export type InferenceOptions = {
8888
db?: boolean | string;
8989
kv?: boolean | string;
9090
storage?: boolean | string;
91-
ai?: boolean;
91+
ai?: boolean | string;
9292
};
9393
paths?: ProjectPaths;
9494
};

‎packages/void/src/runtime/ai.ts‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -407,8 +407,12 @@ function resolveBackend(): AiBackend {
407407
return makeHttpBackend(token, projectId, runtimeEnv);
408408
}
409409

410-
// Priority 3: Direct AI binding (user's own CF account)
411-
const directAi = runtimeEnv.AI as Ai | undefined;
410+
// Priority 3: Direct AI binding (user's own CF account). Void's generated
411+
// wrangler config stamps custom binding names into a private runtime var;
412+
// default and managed deployments continue to use `AI`.
413+
const aiBindingName =
414+
typeof runtimeEnv.__VOID_AI_BINDING === 'string' ? runtimeEnv.__VOID_AI_BINDING : 'AI';
415+
const directAi = runtimeEnv[aiBindingName] as Ai | undefined;
412416
if (directAi) {
413417
return makeDirectBackend(
414418
directAi,
@@ -418,7 +422,7 @@ function resolveBackend(): AiBackend {
418422
}
419423

420424
throw new Error(
421-
'ai: Not available. Run `void auth login` and `void project link` first, or bind Workers AI (env.AI) for self-hosted deploys.',
425+
`ai: Not available. Run \`void auth login\` and \`void project link\` first, or bind Workers AI (env.${aiBindingName}) for self-hosted deploys.`,
422426
);
423427
}
424428

0 commit comments

Comments
 (0)