Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,94 @@ _Avoid_: Installed skill, Skillpack-created skill
The cross-provider view of discovered skills, their provenance, availability, and health.
_Avoid_: Skill editor, authoring workspace

**Skill Invocation**:
A runtime event where a Skill Provider's agent session actually selects or loads a Skill for use during work.
_Avoid_: Skill scan, Skill Availability, installed skill

**Skill Invocation Record**:
A minimal normalized fact derived from provider session artifacts about one Skill Invocation.
_Avoid_: Session transcript, prompt, response

**Invocation Record ID**:
A deterministic identifier for a Skill Invocation Record, derived from provider session evidence so repeated imports can be idempotent.
_Avoid_: Random event ID, row number

**Skill Usage Log**:
The append-only JSONL storage for Skill Invocation Records, partitioned by Session-Producing Provider and month.
_Avoid_: Session transcript store, provider log, database

**Invocation Identity Confidence**:
The confidence Skillpack has when matching a Skill Invocation Record to a known Skill or Provider Instance, based on provider evidence such as path, resolved path, source provenance, or name.
_Avoid_: Skill Identity, duplicate confidence

**Skill Invocation Status**:
The evidence-based runtime state for a Skill Invocation, indicating whether provider session artifacts say the skill was loaded, used, failed, or unknown.
_Avoid_: Skill Availability, Inventory Issue, inventory status

**Skill Source Path**:
The `SKILL.md` path recorded from provider session evidence for an invoked skill, falling back to a resolved path only when the evidence path is unavailable.
_Avoid_: Historical status, Scan Root, inventory path

**Counted Invocation**:
A Skill Invocation Record included in default Skill Usage aggregates. Failed invocations are excluded from default intensity and ranking counts but can be shown as separate failure context.
_Avoid_: Successful invocation, completed task

**Skill Usage**:
User-facing aggregates derived from Skill Invocations, used to understand which skills are being used across Skill Providers, including Project Skills when provider session evidence identifies them.
_Avoid_: Skill Inventory, Skill Availability, inventory status, session history, cross-provider skill rollup

**Provider Skill Ranking**:
A Skill Usage aggregate that ranks invoked skill source paths within a Skill Provider by exact Counted Invocation count, with failed invocation count and recency as supporting context.
_Avoid_: Global leaderboard, inventory order

**Selected-Day Usage Detail**:
The Skill Usage detail for one selected heatmap day, scoped to one Session-Producing Provider and grouped by skill source path with exact Counted Invocation and failure context for that day.
_Avoid_: Session transcript, raw invocation history, provider-wide total

**Skill Usage Heatmap**:
A Skill Usage aggregate that shows Counted Invocation volume over time using colored block intensity, with exact counts exposed through Selected-Day Usage Detail. The overview heatmap compares Skill Providers by day using one scale across the selected range, while provider drilldowns can compare skills by day within one provider.
_Avoid_: Inventory status grid, provider availability matrix

**Skill Usage View**:
The top-level TUI workflow for Skill Usage, combining provider selection, a provider/day heatmap, Selected-Day Usage Detail, and Provider Skill Ranking for the selected Skill Provider and time range.
_Avoid_: Skill Inventory tab, update view, session browser

**Skill Usage Import**:
The read-only process that derives Skill Invocation Records from provider-owned session artifacts.
_Avoid_: Runtime instrumentation, session sync

**Usage Import Diagnostic**:
A non-inventory finding from Skill Usage Import, such as skipped ambiguous provider evidence or unsupported session artifacts.
_Avoid_: Inventory Issue, provider warning

**Usage Provider Adapter**:
A provider-specific importer that reads known session artifact formats and derives valid Skill Invocation Records.
_Avoid_: Heuristic log parser, runtime hook

**Usage Coverage State**:
Whether Skillpack can derive Skill Usage for a Skill Provider, distinguishing supported providers with zero Counted Invocations from providers that are unsupported or not configured for import.
_Avoid_: Skill Availability, zero usage

**Usage Import Consent**:
The user's persisted opt-in that allows Skillpack to read provider-owned session artifacts for Skill Usage Import.
_Avoid_: Config default, provider permission

**Skill Usage Reset**:
A user action that deletes Skillpack's derived Skill Usage Log and Usage Import Cursors without modifying provider-owned session artifacts.
_Avoid_: Provider history deletion, uninstall

**Session-Producing Provider**:
A Skill Provider with agent session artifacts from which Skillpack can derive Skill Invocation Records.
_Avoid_: Shared skill location, install source

**Usage Import Cursor**:
Provider-specific progress metadata that lets Skillpack incrementally import new session evidence without rereading all provider session artifacts on every startup.
_Avoid_: Session state, Skillpack-owned provider state

**Usage Artifact Root**:
A provider-specific directory or file root Skillpack reads during Skill Usage Import to find supported session artifacts.
_Avoid_: Scan Root, skill directory

**Provider Instance**:
A discovered skill as represented by one Skill Provider, including that provider's availability and provenance for the skill.
_Avoid_: Skill Group row, duplicate
Expand Down
5 changes: 5 additions & 0 deletions docs/adr/0039-derive-skill-usage-from-provider-sessions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Derive Skill Usage from provider sessions

Skillpack will derive Skill Usage by reading provider-owned session artifacts read-only, not by adding runtime hooks or instrumentation to agent platforms. This keeps Skillpack aligned with provider-native state: providers remain the source of truth for Skill Invocations, while Skillpack normalizes their session evidence into usage history and aggregates.

The first supported adapter is Codex. It reads the real Codex session JSONL event streams under `~/.codex/sessions/**/*.jsonl` and `~/.codex/archived_sessions/*.jsonl`, using `session_meta` and `turn_context` for session/turn/cwd context and `response_item` `function_call` tool calls for evidence that the agent directly read a `.../skills/<skill>/SKILL.md` file. Codex usage is global across configured artifact roots and is not scoped to Skillpack's startup cwd. `~/.codex/logs_2.sqlite` is not a counting source because it includes rendered skill metadata and can confuse available skills with invoked skills. Claude remains unsupported for Skill Usage until its real session artifact format is implemented conservatively.
5 changes: 5 additions & 0 deletions docs/adr/0040-persist-minimal-skill-invocation-records.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Persist minimal Skill Invocation Records

Skillpack will persist only minimal Skill Invocation Records derived from provider session artifacts: provider, skill identity evidence, session identity, timestamps, and Skill Invocation Status. It will not persist prompts, responses, tool arguments, file contents, or full transcripts, because usage analysis should not turn Skillpack into a secondary session-log store.

Provider adapters may inspect session event envelopes and tool-call arguments transiently to find Skill Invocation evidence, but persisted records must remain aggregate-ready facts only. For Codex, the adapter persists the normalized skill name/path, session id, turn id, timestamp, identity confidence, and status derived from the tool call and its output, not the command string or command output. Codex records are deduplicated by `(turn_id, skill_name)`.
3 changes: 3 additions & 0 deletions docs/adr/0041-include-project-skills-in-skill-usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Include Project Skills in Skill Usage

Skillpack will record Skill Invocations for any skill identified in provider session evidence, including Project Skills. Project Skills remain read-only and separate from the controllable Skill Inventory, but usage analysis needs to answer what was invoked in agent sessions rather than only what Skillpack can manage.
7 changes: 7 additions & 0 deletions docs/adr/0042-use-bounded-incremental-skill-usage-imports.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Use bounded incremental Skill Usage imports

Skillpack will run a cheap incremental Skill Usage Import on startup using provider-specific Usage Import Cursors, and expose deeper rescans from the Usage view for selected time ranges. This keeps usage data current during normal app launches without making inventory refresh depend on repeatedly parsing all provider session artifacts.

Cursors include an importer version as well as source artifact size and mtime. When a parser bug is fixed or a provider adapter starts recognizing additional real session evidence, Skillpack can invalidate old cursors and rescan unchanged artifacts once without requiring users to reset derived usage data.

Startup import failures are recorded in the Usage view state and surfaced when the user opens Skill Usage; they do not block normal inventory startup.
3 changes: 3 additions & 0 deletions docs/adr/0043-use-jsonl-for-skill-usage-records.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Use JSONL for Skill Usage Records

Skillpack will store Skill Invocation Records in an append-only Skill Usage Log using JSONL files partitioned by Session-Producing Provider and month. JSONL avoids native database dependencies for the `npx`-distributed TUI, is easy to inspect and test, and is sufficient for the default 7-day top-10 usage aggregate; a future migration to SQLite can be considered if range queries become too slow.
5 changes: 5 additions & 0 deletions docs/adr/0044-keep-skill-invocation-status-evidence-based.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Keep Skill Invocation Status evidence-based

Skillpack will normalize Skill Invocation Status to provider-evidence states such as loaded, used, failed, and unknown. It will not infer task success from later session outcomes, because usage analytics should report what provider session artifacts prove rather than imply that a skill solved the user's work.

For Codex, a direct `SKILL.md` read with a zero exit code is recorded as `used`; a direct `SKILL.md` read with a non-zero exit code is recorded as `failed`; missing output evidence remains `unknown`. This means failed reads are preserved as facts but do not inflate normal invocation counts.
3 changes: 3 additions & 0 deletions docs/adr/0045-require-consent-for-skill-usage-import.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Require consent for Skill Usage Import

Skillpack will require Usage Import Consent before reading provider-owned session artifacts for Skill Usage Import. After consent is granted, bounded incremental imports may run on startup, but the first import must be explicit because provider session artifacts are private even when Skillpack persists only minimal Skill Invocation Records.
3 changes: 3 additions & 0 deletions docs/adr/0046-retain-skill-usage-records-until-reset.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Retain Skill Usage Records until reset

Skillpack will retain Skill Invocation Records in the local Skill Usage Log until the user performs a Skill Usage Reset. Default Skill Usage views remain bounded to the recent 7-day top-10 output, but Skillpack will not silently expire derived usage data because that would make rankings hard to explain and could discard history the user expected to keep.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Store failed invocations but exclude from default aggregates

Skillpack will persist failed Skill Invocation Records with `status: "failed"` so usage diagnostics can show failure context. Default Provider Skill Ranking and Skill Usage Heatmap intensity will exclude failed records from Counted Invocation totals, because repeated failures should not make a skill appear more used.
4 changes: 4 additions & 0 deletions docs/adr/0048-keep-historical-skills-in-usage-aggregates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Keep Historical Skills in usage aggregates
status: superseded by ADR-0049

Skillpack will keep Historical Skills in Skill Usage aggregates when Skill Invocation Records reference skills that are no longer currently discovered. Historical Skills should appear in Usage views with clear labeling, but they should not appear in the controllable Skill Inventory unless a current Provider Instance or Project Skill is discovered.
5 changes: 5 additions & 0 deletions docs/adr/0049-use-source-path-in-skill-usage-rows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Use source path in Skill Usage rows

Skillpack will group Skill Usage table rows by skill name and Skill Source Path, and will show Source Path instead of current/historical status. Skill Usage should report what provider session evidence says was invoked, not whether the invoked skill is currently discovered by Skill Inventory or Project Skills scanning.

This supersedes ADR-0048. Current/historical labeling was too dependent on configured scan roots: a valid Project Skill invoked from a repository outside the current scan set could be labeled historical even though the invocation evidence contained the exact skill path. Source Path keeps deleted, moved, unscanned, and project-local skills visible without turning Usage into an Inventory status view.
Loading
Loading