Add a research register for ideas, experiments and decisions

AGENTS.md

@@ -98,7 +98,9 @@ Registries, bundled in the package (`dokimi/dokimi/`) so they ship in the wheel:
 - A file the package reads at run time goes inside `dokimi/dokimi/` and into
   `[tool.setuptools.package-data]`, or the installed wheel goes without it.
 - Design rationale, status and plans go in the PR description or an issue;
-  the tree has no ADRs or status reports.
+  the tree has no ADRs or status reports. The one exception is the research
+  register, `dokimi/research/records/`: one record per idea, experiment,
+  finding, decision or implementation link, negative results included (`dokimi/docs/research-register.md`).
 - A new file starts with the SPDX header (`Copyright (C) <year> Intel
   Corporation` / `SPDX-License-Identifier: Apache-2.0`) in its comment syntax.
   CI's license header step (static job) checks it and `make ci-local` does

dokimi/benchmarks/spec.v1.yaml

@@ -76,11 +76,11 @@ future:
 # on the include/exclude globs, not a target: min_docs catches a glob that
 # matched almost nothing, max_docs catches one that escaped into the tree (8
 # markdown files live outside docs/). Measured selection at the time of
-# writing: 30 files. Adding a doc past the ceiling is meant to be a deliberate
+# writing: 31 files. Adding a doc past the ceiling is meant to be a deliberate
 # edit here, so raise max_docs in the same commit that adds the doc.
 golden_manifest:
   min_docs: 10
-  max_docs: 30
+  max_docs: 31
   include:
     - docs/**/*.md
   exclude:

dokimi/docs/README.md

@@ -321,3 +321,5 @@ code.
   metric extracts and runs fenced code examples in isolation.
 - [ingestion-docling.md](ingestion-docling.md) — ingesting non-Markdown
   documents (PDF, HTML, …) via Docling, installed with the optional `ocr` extra.
+- [research-register.md](research-register.md) — the register of Dokimi's own
+  ideas, experiments (negative ones included), findings and decisions.

dokimi/docs/research-register.md

@@ -0,0 +1,117 @@
+# Research register
+
+Dokimi's own research is itself a set of experiments, and many of them come back negative or
+inconclusive. The register in `research/records/` keeps every one of them findable, so the
+next person who wants to try an idea can see what was already tried, what came out of it and
+why a direction was kept or dropped.
+
+The register is plain files in this repository: no server, history from git, review through
+pull requests, and any agent with file access can read and write it.
+
+## Record types
+
+Each record is one Markdown file, `research/records/<type>/<id>.md`. The YAML front matter
+holds the typed fields that `research/record.v1.json` defines; the body is free Markdown for
+the setup, result, rationale and anything else a reader needs, after the
+license header every file in the repository carries.
+
+| Type | Id | What it holds | Points at |
+|---|---|---|---|
+| `idea` | `I001` | A question worth answering, its map theme (`P`…`W`) and `open`/`parked`/`closed` status | — |
+| `experiment` | `X001` | Hypothesis, what was run, and the `outcome`: `planned`, `running`, `supported`, `refuted`, `inconclusive` or `aborted` | `idea` |
+| `finding` | `F001` | What was learned, with `low`/`med`/`high` confidence | `experiments` |
+| `decision` | `D001` | `accepted`, `rejected`, `deferred` or `superseded`, and why | `findings`, `ideas`, `superseded_by` |
+| `implref` | `R001` | The issue or pull request that carries a decision out, with its own `status` | `decision` |
+
+Three states stay separate on purpose: an experiment's `outcome` (what the evidence showed),
+a decision's `state` (what the team chose) and an implementation reference's `status` (what
+was built). A closed issue does not tell which of the three happened.
+
+An experiment that turns out wrong is not deleted: set `retracted: true` and say in the body
+what was wrong with it. `retracted` means the whole experiment is invalid, so `check` fails
+any finding that still cites it; query, export and render keep it, marked. When only part of a
+result is withdrawn, keep the outcome, leave `retracted` unset and list the withdrawn claims
+under a `## Corrections` heading. Every record has a `date` (`YYYY-MM-DD`) and optional `links` (required on
+`implref`), which must be public `github.com/intel/` or `github.com/napetrov/` URLs. The register is public:
+leave out internal hosts, paths, names and raw traces, and describe local evidence in words
+("local run, 96 runs").
+
+## Working with it
+
+Run from the `dokimi/` project directory:
+
+```bash
+python scripts/research_register.py check
+python scripts/research_register.py next experiment
+python scripts/research_register.py query --type experiment --ref I001 --where outcome=refuted,inconclusive
+python scripts/research_register.py query --type decision --where state=rejected
+python scripts/research_register.py export > register.json
+python scripts/research_register.py render > map.md
+python scripts/research_register.py render --html --live > map.html
+python scripts/research_register.py gaps --live
+```
+
+- `check` validates every record against its schema, its path and id prefix, and every
+  reference. `tests/test_research_register.py` runs it in CI, so a broken record fails the PR.
+- `next` prints the next free id for a type. Two PRs can claim the same id; both add the same
+  file, so the second PR to merge gets an add/add conflict (`check` never sees a duplicate)
+  and renumbers then. To renumber: take the new id
+  from `next`, rename the file and change its `id:`, then rewrite every reference to the old
+  id in that PR's own records (`idea`, `experiments`, `findings`, `ideas`, `superseded_by`,
+  `decision`). Find them with `query --ref OLDID`; `check` must pass afterwards.
+- `query` lists the records matching every filter: `--where FIELD=V1,V2` matches any of the
+  values, `--ref ID` keeps the records that point at `ID`. An unknown field, a clause
+  without `=` or a missing `ID` exits 2. Booleans are spelled as in the records:
+  `--where retracted=true`.
+- `export` writes all records, the reference edges and the ideas grouped by map theme as one
+  JSON document, stamped with the commit, for the plans-and-research map and other views.
+  `dirty` is true when `research/` has uncommitted changes, and `render` says so next to the
+  commit.
+- `render` writes the plans-and-research map from the same tree: a Mermaid mindmap (theme,
+  idea, experiments marked by outcome) and a table, as one Markdown file. It is generated on
+  demand and not committed; open it in any Mermaid-aware viewer, such as GitHub or an editor
+  preview.
+- `render --html` writes the same register as one self-contained page to open in a browser,
+  with no network and no library: a collapsible tree (theme, idea, experiment, finding,
+  decision, implref), a search box (`/` focuses it; every word must match the id, a field or
+  the body), filters for type, theme, outcome or state and implementation, a sortable table,
+  and a panel with the clicked record and the records that cite it. `#X003` in the URL opens
+  that record.
+- `gaps` lists accepted decisions with nothing merged and implrefs still `open` or
+  `proposed`. An implref's `status` is written by hand and goes stale when its pull request
+  merges, so `--live` (here and on `render --html`) asks GitHub through `gh` for the state of
+  every link, reports implrefs whose recorded status differs, and lists the open
+  `intel/dokimi` issues that no record links to. It only reads: fix a stale status in a PR.
+  `check` stays offline.
+- An open issue that is engineering work with no research behind it (a dependency
+  dashboard, a build chore, a bug) goes in `research/untracked.yaml` with its reason
+  instead of a record; `gaps --live` skips those and reports an entry whose issue has closed.
+  An implref may also point at the file that carries the change when it landed before the
+  public history and has no pull request.
+
+## Adding a record
+
+1. `python scripts/research_register.py next <type>` for the id.
+2. Write `research/records/<type>/<id>.md`:
+
+    ```markdown
+    ---
+    id: X012
+    type: experiment
+    title: Compressed skill keeps step ordering
+    date: 2026-10-14
+    idea: I004
+    hypothesis: A skill cut to 40% of its tokens keeps the ordering constraints.
+    outcome: refuted
+    ---
+    <!--
+    Copyright (C) 2026 Intel Corporation
+    SPDX-License-Identifier: Apache-2.0
+    -->
+    ## Setup
+    ...
+    ## Result
+    ...
+    ```
+
+3. `python scripts/research_register.py check`, then open the PR.

dokimi/mkdocs.yml

@@ -135,5 +135,6 @@ nav:
       - difficulty-rubric.md
       - example-execution.md
       - ingestion-docling.md
+      - research-register.md
   - CLI reference: reference/cli/
   - API reference: reference/api/

dokimi/research/record.v1.json

@@ -0,0 +1,89 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/intel/dokimi/research/record.v1.json",
+  "title": "Research register record (front matter of research/records/<type>/<id>.md)",
+  "description": "One schema per record type under $defs; scripts/research_register.py picks it by `type`. References are ids; the register checks they resolve to a record of the right type.",
+  "$defs": {
+    "id": {"type": "string", "pattern": "^[A-Z]+[0-9]{3}$"},
+    "ids": {"type": "array", "items": {"$ref": "#/$defs/id"}, "uniqueItems": true},
+    "text": {"type": "string", "minLength": 1},
+    "date": {"type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"},
+    "url": {"type": "string", "pattern": "^https://github\\.com/(intel|napetrov)/(?!(.*/)?\\.\\.(/|$))[A-Za-z0-9_./#?=&~+:@,-]+$"},
+    "theme": {
+      "description": "Branch of the plans-and-research map.",
+      "enum": ["P", "E", "V", "K", "G", "O", "C", "D", "W"]
+    },
+    "common": {
+      "type": "object",
+      "properties": {
+        "id": {"$ref": "#/$defs/id"},
+        "type": {"type": "string"},
+        "title": {"$ref": "#/$defs/text"},
+        "date": {"$ref": "#/$defs/date"},
+        "links": {"type": "array", "items": {"$ref": "#/$defs/url"}, "uniqueItems": true}
+      },
+      "required": ["id", "type", "title", "date"]
+    },
+    "idea": {
+      "$ref": "#/$defs/common",
+      "properties": {
+        "type": {"const": "idea"},
+        "theme": {"$ref": "#/$defs/theme"},
+        "question": {"$ref": "#/$defs/text"},
+        "status": {"enum": ["open", "parked", "closed"]}
+      },
+      "required": ["theme", "question", "status"],
+      "unevaluatedProperties": false
+    },
+    "experiment": {
+      "$ref": "#/$defs/common",
+      "properties": {
+        "type": {"const": "experiment"},
+        "idea": {"$ref": "#/$defs/id"},
+        "hypothesis": {"$ref": "#/$defs/text"},
+        "outcome": {
+          "description": "What the evidence showed. planned/running: no result yet. Negative and inconclusive outcomes stay in the register.",
+          "enum": ["planned", "running", "supported", "refuted", "inconclusive", "aborted"]
+        },
+        "retracted": {"type": "boolean"}
+      },
+      "required": ["idea", "hypothesis", "outcome"],
+      "unevaluatedProperties": false
+    },
+    "finding": {
+      "$ref": "#/$defs/common",
+      "properties": {
+        "type": {"const": "finding"},
+        "experiments": {"$ref": "#/$defs/ids", "minItems": 1},
+        "confidence": {"enum": ["low", "med", "high"]}
+      },
+      "required": ["experiments", "confidence"],
+      "unevaluatedProperties": false
+    },
+    "decision": {
+      "$ref": "#/$defs/common",
+      "properties": {
+        "type": {"const": "decision"},
+        "findings": {"$ref": "#/$defs/ids"},
+        "ideas": {"$ref": "#/$defs/ids"},
+        "state": {"enum": ["accepted", "rejected", "deferred", "superseded"]},
+        "superseded_by": {"$ref": "#/$defs/id"}
+      },
+      "required": ["state"],
+      "unevaluatedProperties": false
+    },
+    "implref": {
+      "$ref": "#/$defs/common",
+      "properties": {
+        "type": {"const": "implref"},
+        "decision": {"$ref": "#/$defs/id"},
+        "status": {
+          "description": "Implementation state, kept apart from the decision state.",
+          "enum": ["proposed", "open", "merged", "closed"]
+        }
+      },
+      "required": ["decision", "status", "links"],
+      "unevaluatedProperties": false
+    }
+  }
+}

dokimi/research/records/decision/D001.md

@@ -0,0 +1,16 @@
+---
+id: D001
+type: decision
+title: Do not ship the local defect battery as a judge replacement
+date: '2026-10-19'
+findings:
+- F001
+- F004
+state: rejected
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Corrected baseline: the hosted judge was not saturated (0.93 pairwise) so the battery only tied it at higher cost.

dokimi/research/records/decision/D002.md

@@ -0,0 +1,15 @@
+---
+id: D002
+type: decision
+title: Recompute aggregate in code; weight correctness x2; zero-weight specificity
+date: '2026-10-19'
+findings:
+- F002
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Judged from pre-public history, not from a record here: the five dimensions appeared to move together, and averaging diluted correctness. F002 is cited only for the judge-prompt context (enumeration beats description); it does not support the weights themselves.

dokimi/research/records/decision/D003.md

@@ -0,0 +1,15 @@
+---
+id: D003
+type: decision
+title: Persist judge material by digest and validate the judge in the arms setting
+date: '2026-10-19'
+findings:
+- F005
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Issue #48; material kept in artifact (#69), gate --context stored (#73, open).

dokimi/research/records/decision/D004.md

@@ -0,0 +1,15 @@
+---
+id: D004
+type: decision
+title: Do not build stratified probe sampler, SliCK split, LLMLingua shortener
+date: '2026-10-19'
+findings:
+- F009
+state: rejected
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Gains were negligible or refuted. The LLMLingua shortener was refuted (X009). F009 reports the grader spread, and stratified probe sampling is judged not to pay for its probe budget. The SliCK split was dropped for the same reason. The sampling and SliCK figures come from local runs not recorded here.

dokimi/research/records/decision/D005.md

@@ -0,0 +1,15 @@
+---
+id: D005
+type: decision
+title: Infer/measure mechanism; [utility] descriptor is an override only
+date: '2026-10-19'
+findings:
+- F010
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+No skill standard declares mechanisms; few public skills carry a type field.

dokimi/research/records/decision/D006.md

@@ -0,0 +1,15 @@
+---
+id: D006
+type: decision
+title: Move dokimi guidance to AGENTS.md; CLAUDE.md removed
+date: '2026-10-19'
+findings:
+- F012
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Measured process gains; merged.

dokimi/research/records/decision/D007.md

@@ -0,0 +1,15 @@
+---
+id: D007
+type: decision
+title: Writing-for-agents prose rewrite rejected; value is the lint gate
+date: '2026-10-19'
+findings:
+- F013
+state: rejected
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Rewrite dropped facts; gate-only kept them.

dokimi/research/records/decision/D008.md

@@ -0,0 +1,16 @@
+---
+id: D008
+type: decision
+title: Ship LLM-free context tools (trace, budget) first
+date: '2026-10-19'
+findings:
+- F015
+- F016
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Measurement tools are deterministic; behavioural fixes stay experimental.

dokimi/research/records/decision/D009.md

@@ -0,0 +1,15 @@
+---
+id: D009
+type: decision
+title: Product repo keeps evals only; mechanics move to dokimi; policy rules dropped
+date: '2026-10-19'
+findings:
+- F018
+state: accepted
+---
+<!--
+Copyright (C) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+## Rationale
+Reviewer objections to generic harness in the product repo; tracking #41.