Skip to content

CLI gen: detect webhook-shaped sub-objects and emit composite --webhook-* builder #348

Description

@HavenDV

Part of #338. Depends on gap #1 (per-parameter flags).

Today

Many APIs have a webhook config nested inside the request body:

webhook:
  type: object
  properties:
    url: { type: string, format: uri }
    headers: { type: object, additionalProperties: { type: string } }
    metadata: { type: object }
    events: { type: array, items: { type: string, enum: [...] } }

Per-property flattening (gap #1) skips nested objects today. Firecrawl hand-writes BuildCrawlWebhookAsync / BuildBatchWebhookAsync to compose flags like --webhook-url, --webhook-header k=v, --webhook-metadata k=v, --webhook-event ... into a single webhook struct. ~150 lines, repeated twice with cosmetic differences.

Target

firecrawl crawl https://example.com \
  --webhook-url https://my.app/hook \
  --webhook-header authorization=Bearer\\ xyz \
  --webhook-header x-request-id=abc \
  --webhook-metadata customer-id=42 \
  --webhook-event completed --webhook-event errored

Proposed approach

  1. Detection heuristic: a nested object property is webhook-shaped when it has:
    • a property of string/format: uri (any name; url/endpoint/callback/webhook preferred)
    • AT LEAST ONE of: a string-map (headers), a generic-object (metadata), an enum-array (events)
  2. Generate --<prefix>-* flag set where <prefix> is the property's kebab-cased name (webhook, notification-url, etc.):
    • URL → single --<prefix>-url (string)
    • headers map → repeatable --<prefix>-header KEY=VALUE (Option<Dictionary<string,string>> with custom parser)
    • metadata map → repeatable --<prefix>-metadata KEY=VALUE
    • events array → repeatable --<prefix>-event with FromAmong(...) for enum constraint
  3. Composite builder in generated invocation: when --<prefix>-url is present, construct the nested object from all --<prefix>-* flags and assign to the request body.
  4. Add helpers to CliRuntime: ParseKeyValueAsync(IEnumerable<string> tokens) -> Dictionary<string,string>.
  5. Vendor extension x-cli-webhook: false to opt out (fall back to nested --<prop>-json).

Acceptance criteria

  • Firecrawl crawl and batch-scrape expose --webhook-url, --webhook-header, --webhook-metadata, --webhook-event after regen.
  • Repeated flags accumulate (--webhook-header a=1 --webhook-header b=2).
  • Without any --webhook-* flag, the webhook field stays unset in the request.
  • Snapshot test covers an operation with a webhook-shaped property.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions