Skip to content

feat: treat OpenClaw as a first-class agent source - #5929

Open
bradcypert wants to merge 3 commits into
mainfrom
brad/dno-959-openclaw-first-class
Open

feat: treat OpenClaw as a first-class agent source#5929
bradcypert wants to merge 3 commits into
mainfrom
brad/dno-959-openclaw-first-class

Conversation

@bradcypert

@bradcypert bradcypert commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Closes DNO-959.

Split out of the original combined PR so the schema change can be rolled forward or back independently — see the sibling PRs below.

Why

OpenClaw sessions weren't treated as a real, named source: the agent-type filter didn't list OpenClaw as one surface, and prompt-to-response turn correlation skipped it.

What changed

  • sourceAliases: an openclaw entry, so the agent-type filter lists it once rather than leaking raw casing variants as separate surfaces
  • canonicalAgentTurnID: accept the openclaw adapter, plus the proxied agent-turn prefix for LiteLLM-routed OpenClaw turns

The issue gated turn correlation on OpenClaw's turn IDs being "proven stable." They are: ctx.runId is identical across before_agent_run, before_tool_call, llm_output and agent_end in the DNO-950 spike fixtures. If a future OpenClaw build breaks that, correlation degrades silently rather than erroring, so it's called out in the fixture README (speakeasy-api/agenthooks#24).

The dashboard display label (openclaw: "OpenClaw") already existed.

Related PRs

Independent of each other — all three branch from main, no stacking:

Testing

server/internal/hooks turn-correlation tests pass, including new cases for the direct and proxied OpenClaw turn IDs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TiJy9DrFpsmnD2vEkDHKNv

@bradcypert
bradcypert requested a review from a team as a code owner September 1, 2026 13:49
@linear-code

linear-code Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

DNO-959

DNO-960

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ed4fd87

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
server Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Running ultrareview automatically — Hand-edited ClickHouse MV migrations change usage aggregation predicates across all agents — a missed bug could corrupt token/cost/tool-call summaries and billing data, so this warrants a deeper multi-pass review.. I'll post findings when complete.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

atlas migrate lint on server/migrations

Status Step Result
No migration files detected  
ERD and visual diff generated View Visualization
No issues found View Report
Read the full linting report on Atlas Cloud

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

atlas migrate lint on server/clickhouse/migrations

Status Step Result
1 new migration file detected 20260831204723_openclaw-usage-rows.sql
ERD and visual diff generated View Visualization
No issues found View Report
Read the full linting report on Atlas Cloud

@bradcypert bradcypert changed the title feat: OpenClaw as a first-class source + customer install runbook mig: OpenClaw as a first-class source + customer install runbook Sep 1, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultrareview completed in 18m 43s

1 issue found across 15 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/internal/hooks/ingest_hooks.go">

<violation number="1" location="server/internal/hooks/ingest_hooks.go:1635">
P1: When an embedded OpenClaw session is also observed through LiteLLM, this new correlation path leaves both assistant responses in the chat transcript. Mark correlated OpenClaw prompts as native and classify `openclaw` in `nativeAssistantTurnSource` so `proxiedTurnDuplicatesNativeStream` suppresses the proxy row.</violation>
</file>

Linked issue analysis

Linked issue: DNO-959: First-class source: alias, display label, turn correlation, usage rows

Status Acceptance criteria Notes
OpenClaw appears once in the dashboard agent-type filter with the “OpenClaw” label The PR adds the OpenClaw source alias, and the PR notes the display label already existed.
OpenClaw is included in canonical turn correlation after stable turn IDs are proven The whitelist accepts OpenClaw and tests cover both native and proxied turn IDs.
OpenClaw usage rows populate usage dashboards when the generic usage path does not already cover them Both materialized views and the mirrored session predicates admit OpenClaw per-turn usage and completed tool-call rows, with a migration applying the changes.
Existing goldens are updated The supplied diff shows new correlation tests and schema-sync coverage, but no golden-file updates or evidence of existing goldens being regenerated.

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

if adapter != "codex" && adapter != "opencode" && adapter != "litellm" {
// openclaw carries OpenClaw's own ctx.runId, which the spike proved stable
// across before_agent_run, before_tool_call, llm_output and agent_end.
if adapter != "codex" && adapter != "opencode" && adapter != "openclaw" && adapter != "litellm" {

@cubic-dev-ai cubic-dev-ai Bot Sep 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When an embedded OpenClaw session is also observed through LiteLLM, this new correlation path leaves both assistant responses in the chat transcript. Mark correlated OpenClaw prompts as native and classify openclaw in nativeAssistantTurnSource so proxiedTurnDuplicatesNativeStream suppresses the proxy row.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At server/internal/hooks/ingest_hooks.go, line 1635:

<comment>When an embedded OpenClaw session is also observed through LiteLLM, this new correlation path leaves both assistant responses in the chat transcript. Mark correlated OpenClaw prompts as native and classify `openclaw` in `nativeAssistantTurnSource` so `proxiedTurnDuplicatesNativeStream` suppresses the proxy row.</comment>

<file context>
@@ -1630,15 +1630,17 @@ func canonicalAgentTurnID(payload *gen.IngestPayload) string {
-	if adapter != "codex" && adapter != "opencode" && adapter != "litellm" {
+	// openclaw carries OpenClaw's own ctx.runId, which the spike proved stable
+	// across before_agent_run, before_tool_call, llm_output and agent_end.
+	if adapter != "codex" && adapter != "opencode" && adapter != "openclaw" && adapter != "litellm" {
 		return ""
 	}
</file context>
Fix with cubic

Comment thread docs/runbooks/openclaw-install.md Outdated
Comment thread docs/runbooks/openclaw-install.md Outdated
Comment thread server/clickhouse/schema.sql Outdated
Comment thread server/internal/hooks/ingest_hooks.go
Comment thread docs/runbooks/openclaw-install.md Outdated
Comment thread docs/runbooks/openclaw-install.md Outdated
Comment thread docs/runbooks/openclaw-install.md Outdated
Comment thread server/internal/chat/sources.go Outdated
Comment thread docs/plugins/overview.md Outdated
- sourceAliases: openclaw entry so the agent-type filter lists it once
  rather than leaking raw casing variants as separate surfaces
- canonicalAgentTurnID: accept the openclaw adapter and the proxied
  agent-turn prefix. OpenClaw's ctx.runId is stable across
  before_agent_run / before_tool_call / llm_output / agent_end, which is
  the condition this was gated on; verified against the DNO-950 spike
  fixtures

The dashboard display label already existed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiJy9DrFpsmnD2vEkDHKNv
@bradcypert bradcypert changed the title mig: OpenClaw as a first-class source + customer install runbook feat: treat OpenClaw as a first-class agent source Sep 1, 2026
@bradcypert
bradcypert force-pushed the brad/dno-959-openclaw-first-class branch from 4302391 to 3bad2ad Compare September 1, 2026 14:10
bradcypert and others added 2 commits September 1, 2026 11:06
Adding openclaw to canonicalAgentTurnID extended prompt correlation to
OpenClaw, but the correlated upsert still only promoted codex and
opencode rows. When LiteLLM recorded an OpenClaw prompt before the
native hook arrived, the native row could not promote the proxy row and
the session stayed labelled litellm.

Also pins both OpenClaw spellings in the source-alias tests so the
agent-type filter cannot regress into two entries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiJy9DrFpsmnD2vEkDHKNv
hook_source keeps its casing into ClickHouse and the summary MVs match
the lowercase literal, so an alias folding "OpenClaw" into "openclaw"
would unify the agent-type filter while those rows silently undercounted
in usage. A visible split is a better failure than a silent undercount.

The alias bought nothing regardless: the generated plugin always passes
--provider=openclaw, so there is one raw spelling, pass-through already
gives the filter a single entry, and there is no historical OpenClaw
data to reconcile. This matches opencode, which carries no alias either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiJy9DrFpsmnD2vEkDHKNv
Thegreatsura pushed a commit to Thegreatsura/gram that referenced this pull request Sep 2, 2026
…peakeasy-api#5933)

Closes DNO-960. Split from speakeasy-api#5929 so nothing here shares a rollback fate
with the schema change.

## Why

We ship an OpenClaw observability package but never told anyone how to
install it. The generated ZIP README had sections for Claude Code,
Cursor, Codex and OpenCode, and nothing for OpenClaw.

There was also no runbook for the two things that silently break an
OpenClaw install — both of which produce a *partly* working state rather
than an obvious failure, which is the worst kind.

## What changed

- **Generated package README**: an OpenClaw install section — the
directory install, `--force` when replacing, the required
`allowConversationAccess` block, the Gateway restart, and the coverage
caveat
- **`docs/runbooks/openclaw-install.md`**: laptop and server/CI install
paths, verification, enforcement semantics, version-pin policy, and the
operational gotchas from the DNO-950 spike
- **`docs/plugins/overview.md`**: OpenClaw listed alongside the other
observability packages and in the `downloadObservabilityPlugin` enum

## The two traps the runbook exists for

1. **`allowConversationAccess`.** Without it the conversation hooks
silently never fire — no prompts, no assistant responses, no usage —
while tool hooks keep working. A half-configured install looks partly
fine.
2. **The harness split.** When a model routes through the Claude CLI
harness (`agentRuntime: claude-cli`, which `openclaw models auth login`
writes *by default* when a claude-cli profile exists), OpenClaw
delegates the model and tool loop out-of-process and its tool/LLM hooks
never fire. Those sessions are still covered — by our Claude Code hooks
— so this is complementary coverage rather than a hole, but a customer
in that mode will otherwise think the integration is broken.

## Notes

- `generate.go` is a server implementation file, but the change is
purely the customer-facing install text, so it belongs with the runbook
rather than the feature PR.
- No `hooksGeneratorVersion` bump needed: the README is not part of the
rendered hook-plugin subtree that `check-generator-versions` diffs.
Confirmed by rendering `export-hook-plugin -published` and checking the
file list.
- The runbook records that our fixtures were captured from OpenClaw
2026.6.34 while 2026.7.1-2 is what installs today, and documents the
qualification procedure for that gap. The fixture-side half is
speakeasy-api/agenthooks#24.
- Deliberately no nightly canary against OpenClaw `latest`: it would
page on OpenClaw's release cadence and registry flakes without telling
us what we actually support — the same reasoning that keeps the LiteLLM
real-proxy suite on `workflow_dispatch`.

## Related PRs

Independent — all three branch from `main`, no stacking:

- speakeasy-api#5929 — `feat:` source alias and turn correlation
- speakeasy-api#5932 — `mig:` the materialized-view change

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01TiJy9DrFpsmnD2vEkDHKNv

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds OpenClaw install docs for the observability plugin — the generated
ZIP README now has an OpenClaw section, and a new runbook covers laptop
and server/CI installs. Two configuration states silently break coverage
without any error, and both are now documented.

- The README's OpenClaw section is generated only when a hooks API key
exists, so it never describes files that aren't in the ZIP.
- The runbook details the two traps: missing `allowConversationAccess`
drops prompt and usage capture while tool hooks keep working, and the
Claude CLI harness routes sessions through Claude Code hooks, which
capture them only if those hooks are also deployed on the machine.
- Documents how coverage depends on model-auth mode, the enforcement
timeout budgets, the version-pin qualification procedure, and that the
hooks binary is fetched from the org's Gram server rather than GitHub.
- Adds an OpenClaw package-format section in
`docs/plugins/package-format.md`, and lists OpenClaw in
`docs/plugins/overview.md`, the `downloadObservabilityPlugin` enum, and
the observability slug table in `docs/plugins/publishing.md`.
- Closes DNO-960.

<sup>Written for commit 024520e.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/speakeasy-api/gram/pull/5933?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant