fix(metrics): count opencode skill invocations

[bingran-you]

Problem

n_skill_invocations is 0 for every opencode rollout, including ones that
demonstrably loaded a skill. opencode's ACP skill call looks like this:

{"type": "tool_call", "kind": "other", "title": "skill", "status": "completed",
 "content": [{"type": "content", "content": {"type": "text",
   "text": "<skill_content name=\"polar-electrostatics-mentor\">\n# Skill: ..."}}]}

is_skill_invocation_event misses it four ways over:

1. kind is "other", not the canonical "skill".
2. The event carries no tool_name / name / function key, so
   _event_tool_name returns "".
3. The title check used an inline literal {"invoke_skill", "activate_skill"}
   which dropped the "skill" spelling that _SKILL_TOOL_NAMES — used by the
   kind check three lines above — already contains.
4. Content sniffing only recognized the OpenHands Tool: invoke_skill header.

This matters more than a cosmetic counter: n_skill_invocations is the field
that answers "did the agent actually use the skill?", and it is the control
variable for every with-skill vs no-skill comparison. Reading a hard 0 for an
entire harness makes those comparisons unfalsifiable — you cannot tell a skill
that didn't help from a skill that was never loaded.

Fix

Recognize both opencode signals, preserving the module's existing rule that a
tool declaring a real ACP kind is never reclassified from its title or output:

• <skill_content name="..."> counts only when it opens a structured
  tool-result text block (anchored \A, exactly as the OpenHands header is), so
  a grep whose output quotes the tag is not counted.
• Bare title == "skill" is honored only for unclassified kinds
  (""/other/tool) — the same gate content sniffing sits behind. The
  explicit invoke_skill / activate_skill spellings stay trusted outright, so
  existing behavior is unchanged. A read or execute tool titled skill
  (a filename, say) is still not a skill invocation.

Verification

Replayed against 206 recorded opencode trajectories from a 29-task,
6-arm evaluation:

	before	after
counter agrees with an independent <skill_content name=...> scan	0 / 206	206 / 206
with-skill rollouts reporting a nonzero count	0	56
no-skill rollouts falsely counted	0	0

The no-skill invariant that backstops the text-derived path still holds.

New tests cover the opencode envelope, a quoted-tag negative, and a
real-kind tool titled skill. tests/test_skill_invocation_artifacts.py
passes 10/10 (7 pre-existing unchanged); 697 tests pass across the
skill/metrics/trajectory suites. ruff check and ruff format --check clean.

🤖 Generated with Claude Code

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be6a1a03dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Name the guarded change in regression docstrings

The three newly added regression tests—this OpenCode-envelope case and the two negative cases below it—describe the behavior they protect but do not identify the PR or commit being guarded. Add the relevant PR/commit reference to each docstring so they comply with the repository's explicit regression-test convention.

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

[Morkeeth]

Corroboration from a second adapter, in case it helps this land.

I hit the same defect on claude-agent-acp while measuring a skill library on SkillsBench. That adapter emits a launch as {"type": "tool_call", "kind": "other", "title": "Skill"} with the body in a content block reading Launching skill: <name>. Each such event reports n_skill_invocations: 0 on the shipped release. The metric reads a library that fired as one that never did — the failure mode you describe.

I checked this branch against that event shape before writing. It already fixes it: count_skill_invocations returns 1 for the ACP event under your change, and still returns 0 for a bash or read tool merely titled "skill".

Worth recording why placement matters. I first tried the naive fix — widening the title branch to _SKILL_TOOL_NAMES. It counts the ACP event, but also any bash/read tool titled "skill", because that branch sits above the gate confining content inference to unclassified kinds. Your version puts the title check below that gate: on the ACP event both return 1; on a bash tool titled "skill" the naive version returns 1 where yours returns 0. Your placement is right and mine was a regression — so this supports your PR rather than offering an alternative.

If a second regression test helps, no test covers this shape yet:

{"type": "tool_call", "kind": "other", "title": "Skill",
 "content": [{"type": "content", "content": {"type": "text",
              "text": "Launching skill: document-extraction"}}]}

Happy to open it as a follow-up PR against your branch or lift it directly.

Two adapters affected rather than one seemed worth adding given this has been open since 30 July. Thanks for the fix.

Drafted with AI assistance; the probes and event capture are from my own runs.