feat(novelty): analyze states the measured share of readable words on stdout

dokimi/changelog.d/20261021_215500_napetrov_novelty-measured-share.md

@@ -0,0 +1,18 @@
+### Added
+
+- **`novelty analyze` states the measured share on stdout** — the headline now prints
+  `coverage: N% of the artifact's readable words were measured`, with `(entry point only)`
+  for a run without `--include-resources`. The novelty.v1 bundle record stamps the same figure
+  as `bundle.measured_share`, so a script can read it without recomputing it from
+  `measurable_outside_share`.
+
+### Changed
+
+- **A run that read half or less of the readable words warns on stdout** — the `WARNING: only
+  N% ...` line goes to stdout, because a headline piped elsewhere never shows stderr. The
+  threshold of the stderr warning is unchanged, and an entry-point run warns on both channels
+  as before. The stdout line is wider than the stderr one in two cases: a `--include-resources`
+  run whose word budgets left half or more of the readable words unread now warns on stdout
+  (its `warnings` list is unchanged), and `coverage: unknown` (a bundled file did not decode)
+  keeps the `(entry point only)` scope and warns when the entry point is half or less of the
+  readable words. `bundle.measured_share` is `null` when the share is unknowable.

dokimi/docs/skill-novelty.md

@@ -490,9 +490,14 @@ Things to know before reading a bundle result:
   deliver if the agent read all of it. Whether the agent chooses to load `references/x.md`
   at runtime is the agentic track's question (`skill-agent:`), not this one.
 - **The two modes are different cells.** `include_resources` is recorded in `run_config`;
-  an entry-point run and a bundle run of the same skill are not comparable numbers, and an
-  entry-point run whose *measurable* outside share is ≥ 0.5 says so in `warnings`
-  (`measurable_outside_share` nets out the role decisions below; `outside_share` is the
+  an entry-point run and a bundle run of the same skill are not comparable numbers. A run
+  that read half or less of the readable words says so on stdout (the `coverage:` line
+  and a `WARNING`), a bundle run held back by the word budgets included; an entry-point run
+  also says so in `warnings`. When a bundled file did not decode the share is unknowable:
+  stdout prints `coverage: unknown` (still warning if the entry point is half or less of
+  the readable words) and `bundle.measured_share` is `null`. The share is
+  `bundle.measured_share` (for an entry-point run, `1 - measurable_outside_share` to 4
+  places, which nets out the role decisions below; `outside_share` is the
   on-disk figure, and the two differ for a skill whose only resources are its evals).
 - **A skill's own `evals/` are not content.** They are the question/answer keys for
   testing the skill, so extracting atoms from them would credit the skill with delivering

dokimi/dokimi/commands/novelty.py

@@ -860,5 +860,42 @@ def _print_headline(artifact: dict) -> None:
         f"degraded: {paired['lenient']['degraded']}, "
         f"dead tokens: {artifact['spans'].get('frac_carrying_nothing')}"
     )
+    _print_coverage(artifact.get("bundle") or {})
     for warning in artifact.get("warnings", []):
         print(f"WARNING {warning}", file=sys.stderr)
+
+
+def _print_coverage(bundle: dict) -> None:
+    """How much of the artifact's readable words this run read, on stdout.
+
+    The share is stated on stdout because the headline is what a user copies. A run under
+    the floor also says so on stdout: a stderr-only warning is one a user piping the
+    headline never sees, and a rate quoted off half-read skill is not a property of it.
+    """
+    from dokimi.novelty.bundle import MEASURED_SHARE_WARN_AT, Coverage
+
+    coverage = Coverage.from_record(bundle)
+    if coverage is None or not coverage.denominator("readable"):
+        return
+    share = coverage.measured_share
+    scope = "" if coverage.include_resources else " (entry point only)"
+    # The warnings below need readable words outside the entry point, so the flag always adds some.
+    rerun = "" if coverage.include_resources else "; pass --include-resources to measure the bundled files"
+    if share is None:
+        print("coverage: unknown (a bundled file did not decode, so its words are not "
+              f"counted){scope}")
+        # The share is unknowable, but the entry point's part of the readable words is not:
+        # the stderr warning fires on it, so stdout says it too.
+        if (not coverage.include_resources
+                and coverage.outside_share_on("readable") >= 1 - MEASURED_SHARE_WARN_AT):
+            print("WARNING: the entry point is half or less of the artifact's readable words, "
+                  f"so every rate here describes that part, not the artifact{rerun}")
+        return
+    # Floored on the 4-place share, in integers: a float product floors 0.29 to 28, and
+    # 49.99% must not print as "50%" beside a "below 50%" warning. A true 49.996% is already
+    # 0.5 by the time it is stored.
+    pct = "<1%" if 0 < share < 0.01 else f"{round(share * 10000) // 100}%"
+    print(f"coverage: {pct} of the artifact's readable words were measured{scope}")
+    if share <= MEASURED_SHARE_WARN_AT:
+        print(f"WARNING: only {pct} of the artifact's readable words were measured, so every "
+              f"rate here describes that part, not the artifact{rerun}")

dokimi/dokimi/metrics/bundle_coverage.py

@@ -119,6 +119,11 @@ def named_exclusions(rows: list[dict] | None) -> tuple[list[dict], int]:
 #: How a record without ``coverage_basis`` is read: it was written when the gate judged
 #: all bundled words, and re-reading it on another basis would move its stored value.
 LEGACY_COVERAGE_BASIS = "all_bundled_words"
+#: A run that read this share of the readable words or less is warned, on stdout and in the
+#: artifact: its rates describe the measured part, not the artifact. This is the line the
+#: analyzer has always warned at (an entry-point run whose outside share is 0.5 or more), and
+#: the gate's ``max_unmeasured_outside_share`` is 0.50 on the same readable basis.
+MEASURED_SHARE_WARN_AT = 0.5
 
 
 def _share(part: int, whole: int) -> float:
@@ -260,3 +265,16 @@ def unmeasured_share(self, basis: str) -> float | None:
             return None
         whole = self.denominator(basis)
         return self.unmeasured_words / whole if whole else 0.0
+
+    @property
+    def measured_share(self) -> float | None:
+        """Share of the readable words this run read, to 4 places; ``None`` when unknowable.
+
+        The complement of :meth:`unmeasured_share` on the readable basis, stated as what was
+        read rather than what was missed. ``None`` for the same reasons: a file did not
+        decode, or there are no readable words to be a share of.
+        """
+        unmeasured = self.unmeasured_share("readable")
+        if unmeasured is None or not self.denominator("readable"):
+            return None
+        return round(1 - unmeasured, 4)

dokimi/dokimi/novelty/bundle.py

@@ -48,6 +48,7 @@
     EXCLUSION_CLASS,
     EXCLUSION_CLASSES,
     LEGACY_COVERAGE_BASIS,
+    MEASURED_SHARE_WARN_AT,
     Coverage,
     _safe_int,
     _share,
@@ -220,7 +221,7 @@ def coverage_warnings(self) -> list[str]:
         # evals has nothing a bundle run would add, so pointing at the flag would be advice
         # that cannot be followed. The gate judges the same basis (`CURRENT_COVERAGE_BASIS`).
         share = coverage.outside_share_on("readable")
-        if not self.include_resources and share >= 0.5:
+        if not self.include_resources and share >= 1 - MEASURED_SHARE_WARN_AT:
             out.append(
                 f"{share:.0%} of this artifact's words are in "
                 "bundled files that were not measured (--include-resources); every rate "
@@ -284,6 +285,7 @@ def summary(self) -> dict:
             "measured_digest": self.measured_digest(),
             "outside_share": coverage.outside_share,
             "measurable_outside_share": coverage.measurable_outside_share,
+            "measured_share": coverage.measured_share,
             "unmeasured_words": coverage.unmeasured_words,
             "max_file_words": self.max_file_words,
             "max_total_words": self.max_total_words,

dokimi/dokimi/schemas/novelty.v1.json

@@ -146,7 +146,7 @@
     },
     "bundle": {
       "type": "object",
-      "description": "Which files of the artifact were measured. A skill is a bundle: measured over the 33-skill study corpus 0.72 of all words live outside SKILL.md (per skill median 0.43, worst 0.99), so an entry-point-only run (include_resources false) is not comparable with a bundle run and must not be quoted as describing the artifact. outside_share is the on-disk share outside the entry point whether or not it was read; measurable_outside_share nets out policy_excluded_words (role decisions such as the skill\u0027s own evals, the same in both modes) and is the figure the warnings and report quote, since a rerun with --include-resources cannot read those; unmeasured_words is what such a rerun would add; excluded lists every file left out with its reason and a kind (symlink, not_text, unreadable, policy, budget, not_requested), and every reader interprets a row by the kind's class (novelty.bundle.EXCLUSION_CLASS): budget and not_requested are a gap (counted in unmeasured_words and warned about), policy is a decision (never charged as unread, and outside the readable-words denominator the warnings, the report and, for a record stamped coverage_basis readable, the gate uses), unreadable is unknown (the gate reports no artifact_coverage value while one exists; the warnings, note and report still quote a figure that leaves its words out), symlink and not_text are not content; pointer_spans names the spans whose only content is a path into the bundle, which carry no knowledge by design and are therefore not waste; by_source cuts the paired outcomes by the file that carried each atom. A bundle run measures what the artifact COULD deliver if the agent read all of it -- an upper bound; whether the agent loads a file at runtime is the agentic track\u0027s question.",
+      "description": "Which files of the artifact were measured. A skill is a bundle: measured over the 33-skill study corpus 0.72 of all words live outside SKILL.md (per skill median 0.43, worst 0.99), so an entry-point-only run (include_resources false) is not comparable with a bundle run and must not be quoted as describing the artifact. outside_share is the on-disk share outside the entry point whether or not it was read; measurable_outside_share nets out policy_excluded_words (role decisions such as the skill\u0027s own evals, the same in both modes) and is the figure the warnings and report quote, since a rerun with --include-resources cannot read those; measured_share is the share of the readable words this run read (null when a file did not decode or there are no readable words); unmeasured_words is what such a rerun would add; excluded lists every file left out with its reason and a kind (symlink, not_text, unreadable, policy, budget, not_requested), and every reader interprets a row by the kind's class (novelty.bundle.EXCLUSION_CLASS): budget and not_requested are a gap (counted in unmeasured_words and warned about), policy is a decision (never charged as unread, and outside the readable-words denominator the warnings, the report and, for a record stamped coverage_basis readable, the gate uses), unreadable is unknown (the gate reports no artifact_coverage value while one exists; the warnings, note and report still quote a figure that leaves its words out), symlink and not_text are not content; pointer_spans names the spans whose only content is a path into the bundle, which carry no knowledge by design and are therefore not waste; by_source cuts the paired outcomes by the file that carried each atom. A bundle run measures what the artifact COULD deliver if the agent read all of it -- an upper bound; whether the agent loads a file at runtime is the agentic track\u0027s question.",
       "properties": {
         "include_resources": {"type": "boolean"},
         "n_files": {"type": "integer", "minimum": 0},
@@ -155,6 +155,7 @@
         "resource_words_on_disk": {"type": "integer", "minimum": 0},
         "outside_share": {"type": "number", "minimum": 0, "maximum": 1},
         "measurable_outside_share": {"type": "number", "minimum": 0, "maximum": 1},
+        "measured_share": {"type": ["number", "null"], "minimum": 0, "maximum": 1},
         "policy_excluded_words": {"type": "integer", "minimum": 0},
         "unmeasured_words": {"type": "integer", "minimum": 0},
         "coverage_basis": {

dokimi/scripts/growth_baseline.json

@@ -1,6 +1,6 @@
 {
   "files": {
-    "dokimi/commands/novelty.py": {"lines": 864, "rationale": "W3 adds routing no_queries handling, W10 a --catalog help line; file split tracked separately to avoid conflicts across the W-series"},
+    "dokimi/commands/novelty.py": {"lines": 901, "rationale": "stdout coverage line and warning in novelty CLI; file split tracked separately"},
     "dokimi/harnesses/benchflow.py": {"lines": 869},
     "dokimi/harnesses/docker_solver.py": {"lines": 927, "rationale": "pre-existing when the gate was introduced"},
     "dokimi/llm.py": {"lines": 1118, "rationale": "grew on main in #180 (replay a stored retry) after #93 seeded this baseline; split it in its own PR"},