Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions docs/guide/queues.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,9 @@ Use queues to process work asynchronously, such as sending emails or handling up

## Defining queues

Create a file such as `queues/emails.ts` or `queues/order-notifications.ts`. The filename becomes the queue name. `.mts`, `.js`, and `.mjs` files are also supported.
Create files in `queues/**/*.ts`; `.mts`, `.js`, and `.mjs` also work. The queue name is inferred from its path, with nested path segments joined by `-`. For example, `queues/emails.ts` creates a queue named `"emails"`, and `queues/order/notifications.ts` creates `"order-notifications"`. A nested path must not normalize to the same name as another file: `queues/order/notifications.ts` and `queues/order-notifications.ts` conflict, so Void reports the collision and asks you to rename one.

::: warning Keep deployable queue files in one directory
Nested paths become part of the queue name: `queues/order/notifications.ts` produces `"order/notifications"`. That works locally, but Cloudflare rejects `/` in queue names. Use `queues/order-notifications.ts` for a queue you intend to deploy.
:::
The resulting name must follow Cloudflare's queue naming rules: 1–63 characters, only letters, digits, and `-`, beginning and ending with a letter or digit.

Each queue file should export a default handler wrapped with [`defineQueue`](../reference/api.md#definequeue-t-handler). The generic `<T>` parameter defines the message body type. That is the type of each `msg.body` in the batch, and it is also used by the typed `queues` proxy for `send()` calls.

Expand Down Expand Up @@ -56,7 +54,7 @@ export const POST = defineHandler(async (c) => {
});
```

The binding name is derived automatically: `QUEUE_` + queue name uppercased with non-alphanumeric characters replaced by `_`. For example, `queues/emails.ts` creates binding `QUEUE_EMAILS`.
The binding name is derived automatically: `QUEUE_` + queue name uppercased with non-alphanumeric characters replaced by `_`. For example, `queues/emails.ts` creates binding `QUEUE_EMAILS`, while `queues/order/notifications.ts` creates binding `QUEUE_ORDER_NOTIFICATIONS`.

## Per-message acknowledgment

Expand Down
2 changes: 1 addition & 1 deletion packages/void/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ src/
db-types.ts # (removed — Drizzle schema is now the type source)
queue-types.ts # Generates .void/queues.d.ts (QueueMap with payload types)
queues/
scan.ts # Queue file scanner (queues/ dir, name from filename + consumer options)
scan.ts # Queue file scanner (name from relative path, nested segments joined by `-`, consumer options)
runtime/
handler.ts # defineHandler(), defineHandler.withValidator(), defineMiddleware(), defineHead(), defineQueue()
queues.ts # QueueMap stub interface + typed `queues` proxy using env.QUEUE_*
Expand Down
11 changes: 6 additions & 5 deletions packages/void/src/cli/deploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4274,10 +4274,11 @@ export async function runCloudflareDeploy(
if (options.provision || initiallyMissing.length > 0 || d1NeedsConfigReconciliation) {
await provisionAndVerify();
} else {
// Queue names are taken VERBATIM from filenames (`queues/<name>.ts` → `<name>`, see
// `queues/scan.ts`) with no sanitization, and the probe below passes each one to wrangler
// as an ARGV positional. Gate them on the SAME `assertValidResourceName('queue', …)` the
// provisioner applies, BEFORE the probe runs: a name beginning with `-` is
// Queue names come from file paths (`queues/order/jobs.ts` → `order-jobs`, see
// `queues/scan.ts`), and the probe below passes each one to wrangler as an ARGV positional.
// Apart from joining path segments, names are not sanitized. Gate them on the SAME
// `assertValidResourceName('queue', …)` the provisioner applies, BEFORE the probe runs:
// a name beginning with `-` is
// parsed by wrangler as a FLAG (`queues info --help` exits 0 printing usage), which the
// probe would classify as inconclusive and wave through — silently forfeiting the
// early-ordering guarantee this whole step exists to provide. Collect EVERY offending name
Expand All @@ -4296,7 +4297,7 @@ export async function runCloudflareDeploy(
'Deploy blocked — these queue names are not valid Cloudflare queue names:',
...invalidQueues.map((name) => ` • ${name} (queue)`),
'',
'A queue takes its name from its filename in queues/ (queues/<name>.ts → "<name>").',
'A queue takes its name from its path in queues/; nested segments are joined with "-".',
'A queue name must be 1–63 characters, contain only letters, digits and "-", and',
'begin and end with a letter or digit.',
'',
Expand Down
34 changes: 30 additions & 4 deletions packages/void/src/queues/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ const QUEUE_EXTENSIONS = ['ts', 'mts', 'js', 'mjs'];
export type QueueDefinition = {
/** Relative file path from queues/ */
filePath: string;
/** Deterministic name (path without extension) */
/** Deterministic name (path without extension, with directories joined by `-`) */
name: string;
/** Binding name: QUEUE_ + uppercased name */
bindingName: string;
Expand All @@ -22,7 +22,7 @@ export type QueueDefinition = {
};

function extractName(filePath: string): string {
return filePath.replace(/\.[^.]+$/, '');
return filePath.replace(/\.[^.]+$/, '').replaceAll('/', '-');
}

export function deriveQueueBindingName(name: string): string {
Expand All @@ -41,9 +41,9 @@ function parseQueueFiles(
files: Array<string>,
): Array<QueueDefinition> {
const queuesDirPath = isAbsolute(queuesDir) ? queuesDir : join(root, queuesDir);
return files
const definitions = files
.filter((filePath) => !filePath.split('/').some((segment) => segment.startsWith('_')))
.sort((leftFilePath, rightFilePath) => leftFilePath.localeCompare(rightFilePath))
.sort()
.map((filePath) => {
const absPath = join(queuesDirPath, filePath);
const source = readFileSync(absPath, 'utf-8');
Expand All @@ -58,6 +58,32 @@ function parseQueueFiles(
retryDelay: extractConsumerOption(source, 'retryDelay'),
};
});

const filesByName = new Map<string, Array<string>>();
for (const definition of definitions) {
const matchingFiles = filesByName.get(definition.name) ?? [];
matchingFiles.push(definition.filePath);
filesByName.set(definition.name, matchingFiles);
}

const collisions = [...filesByName]
.filter(([, matchingFiles]) => matchingFiles.length > 1)
.sort(([leftName], [rightName]) => (leftName < rightName ? -1 : leftName > rightName ? 1 : 0))
.map(
([name, matchingFiles]) =>
` ${JSON.stringify(name)}: ${matchingFiles.map((file) => JSON.stringify(file)).join(', ')}`,
);
if (collisions.length > 0) {
throw new Error(
[
'queue: Multiple queue files resolve to the same queue name:',
...collisions,
'Rename the conflicting files so each queue has a unique name.',
].join('\n'),
);
}

return definitions;
}

export async function scanQueues(
Expand Down
21 changes: 19 additions & 2 deletions packages/void/test/unit/cloudflare-deploy.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2661,8 +2661,8 @@ describe('runCloudflareDeploy — pre-build queue existence probe', () => {
describe('runCloudflareDeploy — pre-build queue name validation', () => {
/**
* A worker-bearing app whose desired queues come from the given `queues/` filenames.
* Queue names are the filenames verbatim (`queues/scan.ts` strips only the extension),
* so this fixture is how an adversarial name actually reaches the deploy.
* Queue names are derived from those paths (`queues/scan.ts` strips the extension and joins
* nested segments with `-`), so this fixture is how an adversarial name actually reaches deploy.
*/
function queueNameProject(...fileNames: Array<string>): string {
return makeProject({
Expand Down Expand Up @@ -2763,6 +2763,23 @@ describe('runCloudflareDeploy — pre-build queue name validation', () => {
expect(trace.indexOf('wrangler:queues')).toBeLessThan(trace.indexOf('build'));
expect(trace).toContain('wrangler:deploy');
});

it('probes a nested queue using its normalized Cloudflare resource name', async () => {
const root = queueNameProject('order/notifications.ts');
const { deps, wranglerArgs, trace } = harness();
await runCloudflareDeploy(root, { backend: 'cloudflare' }, silentCliLog, deps);

expect(wranglerArgs.find((args) => args[0] === 'queues')).toEqual([
'queues',
'info',
'order-notifications',
'-c',
join(root, 'wrangler.jsonc'),
'--env=',
]);
expect(trace.indexOf('wrangler:queues')).toBeLessThan(trace.indexOf('build'));
expect(trace).toContain('wrangler:deploy');
});
});

describe('runCloudflareDeploy — skipped desired resource is fatal', () => {
Expand Down
30 changes: 30 additions & 0 deletions packages/void/test/unit/queue-scan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,34 @@ export default defineQueue(async (batch, env) => {});`,
expect(queues[0].maxRetries).toBe(3);
expect(queues[0].retryDelay).toBe(60);
});

it('normalizes nested paths while preserving binding derivation', () => {
mkdirSync(join(root, 'queues', 'order'), { recursive: true });
writeFileSync(
join(root, 'queues', 'order', 'notifications.ts'),
`export default async () => {};`,
);

expect(scanQueuesSync(root)).toEqual([
{
filePath: 'order/notifications.ts',
name: 'order-notifications',
bindingName: 'QUEUE_ORDER_NOTIFICATIONS',
},
]);
});

it('reports normalized name collisions in deterministic path order', () => {
mkdirSync(join(root, 'queues', 'order'), { recursive: true });
writeFileSync(join(root, 'queues', 'order-notifications.ts'), 'export default {};');
writeFileSync(join(root, 'queues', 'order', 'notifications.ts'), 'export default {};');

expect(() => scanQueuesSync(root)).toThrow(
[
'queue: Multiple queue files resolve to the same queue name:',
' "order-notifications": "order-notifications.ts", "order/notifications.ts"',
'Rename the conflicting files so each queue has a unique name.',
].join('\n'),
);
});
});
4 changes: 2 additions & 2 deletions packages/void/test/unit/queue-types.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ describe('generateQueueTypes', () => {
{ filePath: 'emails.ts', name: 'emails', bindingName: 'QUEUE_EMAILS' },
{
filePath: 'order/notifications.ts',
name: 'order/notifications',
name: 'order-notifications',
bindingName: 'QUEUE_ORDER_NOTIFICATIONS',
},
];

const result = generateQueueTypes(queues, '/project/queues', '/project/.void');
expect(result).toContain('"emails": Queue<');
expect(result).toContain('"order/notifications": Queue<');
expect(result).toContain('"order-notifications": Queue<');
expect(result).toContain('import("../queues/order/notifications.ts")');
});

Expand Down
16 changes: 15 additions & 1 deletion packages/void/test/unit/scan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,10 +192,24 @@ describe('scanRoutes', () => {

const result = await scanRoutes(TMP);
expect(result.queues).toHaveLength(1);
expect(result.queues[0].name).toBe('order/notifications');
expect(result.queues[0].name).toBe('order-notifications');
expect(result.queues[0].bindingName).toBe('QUEUE_ORDER_NOTIFICATIONS');
});

it('rejects nested queue names that collide after normalization', async () => {
createFile('routes/api/hello.ts');
createFile('queues/order-notifications.ts', 'export default async () => {}');
createFile('queues/order/notifications.ts', 'export default async () => {}');

await expect(scanRoutes(TMP)).rejects.toThrow(
[
'queue: Multiple queue files resolve to the same queue name:',
' "order-notifications": "order-notifications.ts", "order/notifications.ts"',
'Rename the conflicting files so each queue has a unique name.',
].join('\n'),
);
});

it('extracts maxBatchSize and maxBatchTimeout', async () => {
createFile('routes/api/hello.ts');
createFile(
Expand Down