Skip to content

feat(novelty): routing takes the catalog field, skips disabled and mirrored skills, adds all-vs-all - #172

Open
napetrov wants to merge 19 commits into
intel:mainfrom
napetrov:fix/skill-gap-W4
Open

napetrov wants to merge 19 commits into
intel:mainfrom
napetrov:fix/skill-gap-W4

Conversation

@napetrov

Copy link
Copy Markdown
Contributor

Part of #4.
Depends on #164 and #168: rebase after both merge (routing.py and the routing tests overlap with those branches).

Summary

novelty routing builds its field the way a harness does: the catalog's own manifest, without skills the model cannot select, and with each mirror counted once. A new --all-vs-all mode routes every skill's queries against the whole field and reports who takes whose queries.

 cmd_novelty_routing
+  --all-vs-all --queries-by-skill <file> --out <file>
+    every skill with queries is the gold in turn
+    confusion matrix, own_rate per skill, attractors (skill picked for other skills' queries)
+    (none) = no card shares a word with the query
   field = _routing_field(args)
-    --catalog: every SKILL.md under root
+    --catalog: skills.yaml when present, else every SKILL.md
+    skip disable-model-invocation        (--include-disabled keeps)
+    skip non-SKILL.md files and dirs without SKILL.md, with a warning
+    one copy per byte-identical SKILL.md (nearest the catalog root)
+    same name, different bytes -> refuse, naming both paths

Evidence

Eleven new tests fail against intel/main; the other 68 in the file pass there. The run reverts commands/novelty.py only, because the new tests import all_vs_all, load_queries_by_skill and disables_model_invocation from novelty/routing.py, so reverting that file too would fail the whole module on import:

before: commands/novelty.py reverted to intel/main, rest of this branch kept
  FAILED test_a_non_skill_file_among_competitors_is_skipped_with_a_warning
  FAILED test_a_disable_model_invocation_skill_is_not_a_competitor_unless_asked
  FAILED test_an_explicit_disabled_competitor_is_skipped_with_a_warning
  FAILED test_a_disabled_artifact_is_still_routed_but_warned_about
  FAILED test_a_mirror_of_a_catalog_skill_is_counted_once
  FAILED test_two_catalog_skills_with_one_name_and_different_content_are_refused_by_name
  FAILED test_all_vs_all_writes_the_matrix_and_prints_the_attractors
  FAILED test_all_vs_all_with_model_names_its_model_calls_on_stderr
  FAILED test_all_vs_all_without_model_says_nothing_about_model_calls
  FAILED test_all_vs_all_takes_no_artifact
  FAILED test_a_single_routing_run_still_needs_an_artifact
  11 failed, 68 passed

The routing.py-level tests (the attractor table, the (none) rule, the disable-flag spelling in test_the_disable_flag_is_read_the_way_the_repo_budget_reads_it) need the new module and cannot be run against intel/main at all.

Local checks on the rebased head:

uv run ruff check .                              All checks passed!
uv run mypy dokimi                               Success: no issues found in 222 source files
uv run python -m dokimi.schema_check             exit 0
uv run python -m dokimi.config_check             exit 0
uv run pytest -n auto -q                         7099 passed, 22 skipped

Merge Danger

Door: two-way. Every mode is additive except the default field, which now differs for catalogs with disabled or mirrored skills. Reverting restores the old field.

Blast Radius: medium. Existing routing figures change for any catalog that has disabled or duplicated skills, so older numbers are not comparable with new ones.

  • Default excludes disable-model-invocation skills. A --catalog or --competitors field no longer includes a skill that sets disable-model-invocation, and an artifact with that flag is routed with a warning. To keep the old field, pass --include-disabled.
  • Catalog reads the manifest. With skills.yaml present, --catalog takes its entries, and a declared skill that does not load is skipped with a note rather than aborting the run.
  • Mirrors counted once. A byte-identical SKILL.md under another path is dropped, keeping the copy nearest the catalog root.
  • Name clash refuses. Two catalog skills with one name and different bytes now raise, naming both paths.
  • Non-skill competitors skipped. A skill: reference to a file that is not SKILL.md, or to a directory without one, is skipped with a warning. text: and plain paths are unchanged.
  • New mode. --all-vs-all makes one selector call per query over the whole field when --model is given. The CLI prints the call count on stderr before it starts.
  • --artifact is now optional only with --all-vs-all; a single run without it still fails.
  • No persisted-format change beyond the new all_vs_all payload written to its own --out file.

Checklist

  • Tests added/updated and pytest passes
  • ruff, mypy, and schema checks pass
  • Docs updated: docs/skill-novelty.md and docs/skill-novelty-walkthrough.md
  • No ad-hoc run artifacts committed
  • New code and test files carry the SPDX license header (no new code files; changelog fragments follow the existing fragments, which carry none)

🤖 Generated with Claude Code

…anifest

--catalog already took the competitors from a skill directory; this adds the
name the flag is asked for under, and makes the catalog reader the one
catalog scan uses. A skills.yaml at the root now decides which skills are
competitors. A declared skill that cannot be loaded is skipped with a
warning instead of making the whole field unreadable.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
A shell glob over a skills directory passes every file beside the skills,
and the first helper script (activation-rule.mjs) aborted the whole routing
run. A skill: competitor that is not a SKILL.md, or a directory without one,
is now left out of the field with a warning. The artifact under test is still
refused, and a text: competitor is unchanged.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…the field

No harness lists a skill that sets disable-model-invocation, so the model
cannot select it, and a routing card for one measured a choice that does not
exist. Such skills are now left out of a --catalog and of --competitors, with
a warning on stderr. --include-disabled keeps them. A disabled artifact is
still routed, with a warning. The flag is read with the truthy set
repos.budget already uses.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
A plugin copy of a skills/ tree is a byte-identical SKILL.md under another
path with the same name. The catalog read both, so the field named the skill
twice and the run was refused. Catalog entries are now grouped by the SHA-256
of SKILL.md; the copy nearest the catalog root stands for the card, and the
others are named in the warnings.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
… table

Every skill with queries becomes the gold in turn and its queries are routed against
the whole field. The output names the skills picked for other skills' queries
(attractors), with how many of those queries each took and from which skills.
Lexical arm always; the model arm when --model is given. Takes --queries-by-skill
and --out, and no --artifact. The --artifact flag is now optional for a single run.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…-model counts its calls

--competitors-from-catalog was an alias of --catalog, which is already on main. Drop
the alias. The changelog and docs now describe the --catalog change itself: a
skills.yaml at the catalog root decides which skills are competitors.

Two catalog skills with one name and different content are refused, naming both
paths, instead of falling through to check_field's "pass each competitor once".

With --model, all-vs-all prints the number of model calls (one per query) to
stderr before it makes them, and the docs note the cost.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 3ab7f1b. The field logic holds up: the manifest read, the name-clash check before the mirror dedupe, and the disable-flag spellings behave as described. The problems are in the new --all-vs-all mode and its tests. Five things are worth fixing before merge, and one nit.

What I ran, on a Linux review host at 3ab7f1b:

  • make ci-local: 20 passed, 8 failed, 7 skipped. The 8 failures are exit 127 (ruff, zizmor, pre-commit, bandit, mypy, deptry, mkdocs, pip-audit missing from my venv): make dev stops at the tools install on this base, which #179 fixed on main. Static checks, mypy and the strict docs build pass on GitHub's CI at this head. The test job passed: 7,100 passed, 21 skipped, changed-line coverage gate passed.
  • tests/test_novelty_routing.py: 79 passed.
  • Reproductions for the code findings, with the LLM transport mocked.
  • A mutation run of about 60 mutants on the new code: 35 killed, the survivors are in the test comment.

Checked, no issues

  • Disable flag: true, True, 'true', yes, on, 1 read as disabled; false, no, y do not. This matches repos/budget.py.
  • Clash and mirrors: a disabled skill that shares a name with an enabled one is dropped first, so there is no false clash. Ordering is deterministic.
  • Call count: the stderr count equals the calls made on a valid queries file.
  • Confusion and attractor figures: own_rate, wrongly_picked, sibling_queries match hand counts, and a query with no shared word is (none).
  • Changelog fragments: format and location match the existing ones.

Comment thread dokimi/dokimi/novelty/routing.py Outdated
Comment thread dokimi/dokimi/novelty/routing.py Outdated
Comment thread dokimi/dokimi/commands/novelty.py Outdated
Comment thread dokimi/docs/skill-novelty.md Outdated
Comment thread dokimi/tests/test_novelty_routing.py
Comment thread dokimi/docs/skill-novelty.md Outdated
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
@codecov-commenter

codecov-commenter commented Oct 22, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 89.81818% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.96%. Comparing base (9a93635) to head (898e053).

Files with missing lines Patch % Lines
dokimi/dokimi/commands/novelty.py 85.71% 9 Missing and 7 partials ⚠️
dokimi/dokimi/novelty/routing.py 92.59% 6 Missing and 6 partials ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
- Coverage   91.98%   91.96%   -0.02%     
==========================================
  Files         223      223              
  Lines       27743    27999     +256     
  Branches     4560     4622      +62     
==========================================
+ Hits        25520    25750     +230     
- Misses       1574     1588      +14     
- Partials      649      661      +12     
Files with missing lines Coverage Δ
dokimi/dokimi/external/catalog.py 96.82% <100.00%> (ø)
dokimi/dokimi/novelty/routing.py 95.80% <92.59%> (-3.05%) ⬇️
dokimi/dokimi/commands/novelty.py 94.33% <85.71%> (-1.51%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a commit per nit, up to 70b7a72, which fix the nit from my review:

  • 70b7a72: dokimi/docs/skill-novelty.md:602: the sentence now says a disabled artifact is still routed, with a warning, and the flag does not change that; the doubled "before quoting either:" at line 647 is dropped.

make ci-local at 70b7a72: 20 passed, 8 failed, 7 skipped.

Not approving yet. Still open: the bug and the four risks from my review.

… file, max-usd doc

Rank with the gold last so a lexical tie is no win; count n_failed and
n_abstained in the model arm, warn on failures and exit non-zero when every
call failed; catch OSError/UnicodeDecodeError on --queries-by-skill; say
routing has --max-usd; add tests for the surviving mutants.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…rowth caps

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 924aba9. All five points from my earlier review are fixed and tested, and the field logic still holds. Two things are worth fixing before merge, both in the new --all-vs-all command, and there are two nits inline plus one in the body.

What I ran, on a Linux review host at 924aba9:

  • make ci-local: 28 passed, 1 failed, 7 skipped. The failure is pip-audit on gitpython 3.2.0 (CVE-2026-114145, fix 3.2.1), a pinned dependency the PR does not touch. Tests: 7,258 passed, 1 skipped. The changed-line coverage gate passed.
  • tests/test_novelty_routing.py and tests/test_novelty_routing_all_vs_all.py: 99 passed.
  • Reproductions for the code findings, with the LLM transport mocked.
  • A mutation run of about 55 mutants on the new code. The survivors from my earlier review are killed, except gold=0, which you explained. Several new ones survive (below).

Nit: the all-vs-all tests leave a few outputs unpinned. With the full suite at 99 passed, these mutants survive: seed=0 passed to all_vs_all from _cmd_routing_all_vs_all; the top-five cap in _print_attractors changed to three; the name tie-break on equal wrongly_picked (and in from) dropped or reversed; the gold != name filter removed from sibling_queries, which only changes rate for an attractor that has queries of its own (the exact-row tests use a hub with none). Suggestion: add one case with two attractors tied on count and one attractor that also has queries, and assert seed in the saved payload.

Checked, no issues

  • Tie scoring: two identical cards give confusion {"a": {"b": 1}, "b": {"a": 1}} and own_rate 0.0 in both field orders. A query with no shared word is (none).
  • Failed calls: a transport that raises on every call gives n_failed equal to the query count, a stderr warning and a non-zero exit with the partial result written.
  • Queries file: a missing file, a non-UTF-8 file, a top-level list, null, non-string items, an empty file and an unknown skill name each exit with a message, not a traceback.
  • --max-usd: with a cap of 1.0 and a mock priced at 0.6 per call, the run stops after 2 calls with spend cap reached, and no --out file is written. The docs now say so.
  • Field logic: true, True, 'true', yes, on, 1 read as disabled; false and 'off' do not. Byte-identical mirrors collapse to the copy nearest the root, and a same-name clash with different bytes names both paths.
  • Docs and changelog: the flags, outputs and defaults in skill-novelty.md, the walkthrough and the five fragments match the code.

Comment thread dokimi/dokimi/commands/novelty.py Outdated
payload = {"mode": "all_vs_all", "queries_by_skill": args.queries_by_skill,
"n_candidates": len(field), "result": result}
out.write_text(json.dumps(payload, indent=2), encoding="utf-8")
if model_arm and failed and failed >= result["n_queries"]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A partial model failure exits 0 and prints OK Saved.

The non-zero exit only fires when failed >= result["n_queries"]. With a transport that answers once and then raises rate limited, five queries over db and perf print:

WARNING: 4 model call(s) failed (first error: rate limited); ...
OK Saved all-vs-all routing: ...
model: hub picked for 1 of 5 sibling queries (from db)

Exit code 0. The saved model arm has own_rate {db: 0.0, perf: 0.0} and n_failed 4. perf never got an answer, yet it reads as a measured 0.0. A rate limit mid-run in a script or CI step looks like success, and the attractor line looks like a measurement. The warning scrolls past, and the figures are lower bounds only if the reader knows to look at n_failed.

Suggestion: exit non-zero whenever failed is non-zero (the file is still written), or add a --allow-failed-calls escape. Report own_rate as null for a skill whose queries all failed. Add a test with one success and several failures; the existing tests only cover all-fail.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5e27afa: a run with any failed model call now writes the result and exits non-zero (no 'OK Saved'). Failed calls are left out of every rate: a skill whose queries all failed has own_rate null (n_scored 0, n_unscored n), not 0.0. Test: test_a_partial_model_failure_exits_non_zero_and_leaves_failed_queries_out (one success, two failures).

Comment thread dokimi/dokimi/novelty/routing.py Outdated
of strings.
"""
try:
data = json.loads(Path(path).read_text(encoding="utf-8-sig"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A queries file with a repeated skill key silently keeps only the last list.

{"a": ["profile"], "a": ["slow"]} loads as {'a': ['slow']} with no error. A hand-merged file measures fewer queries than the author wrote, and n_queries is the only trace.

Suggestion: parse with object_pairs_hook and raise RoutingInputError(f"{path}: skill {key!r} appears more than once") on a repeated key. Add a test for it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5e27afa: load_queries_by_skill parses with an object_pairs_hook and raises RoutingInputError ": skill 'a' appears more than once". Test: test_a_skill_named_twice_in_the_queries_file_is_refused.

raise RoutingInputError("all-vs-all: the field names a skill more than once")
unknown = sorted(set(queries_by_skill) - set(names))
if unknown:
raise RoutingInputError("the queries name skills that are not in the field: "

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: a queries file written for the whole catalog fails on a disabled skill without naming the cause. The field drops disable-model-invocation skills by default, so a file that names one ends with the queries name skills that are not in the field: `gamma` . The earlier "left out catalog skill" warning has scrolled past by then; the same run with --include-disabled succeeds.

Suggestion: append "; a skill that sets disable-model-invocation is left out unless you pass --include-disabled" to this message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5e27afa: the message now ends with '; a skill that sets disable-model-invocation is left out unless you pass --include-disabled'. Test: test_the_not_in_the_field_error_names_the_disable_flag_and_its_override.


def _check_all_vs_all_args(args: argparse.Namespace) -> None:
"""Refuse the flag combinations ``--all-vs-all`` cannot run with."""
if args.artifact:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: --queries is accepted with --all-vs-all and ignored. --queries /nonexistent.json --all-vs-all ... prints OK Saved all-vs-all routing. --artifact is refused here for the same reason.

Suggestion: add if args.queries: raise SystemExit("--all-vs-all takes --queries-by-skill, not --queries") next to the --artifact check.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5e27afa: --all-vs-all now refuses --queries and --queries-from-headings with 'takes --queries-by-skill, not --queries or --queries-from-headings'. Test: test_all_vs_all_refuses_the_single_artifact_query_flags (both flags).

Single-mode routing.json keeps the selector model; the all-vs-all model
arm did not, so files from two different models were indistinguishable.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
… its own

The existing guard never ran: in its fixture no attractor has queries, so
"rate over all queries" and "own queries not excluded" mutants passed.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
@xaleryb

xaleryb commented Oct 23, 2026

Copy link
Copy Markdown
Contributor
  • Checks run: the full suite, ruff, mypy and bandit on the head merged with current main; every point below was reproduced on a toy catalog.
  • --all-vs-all own_rate counts unscored queries as misses. alpha queries ["convert my json file to yaml","json to yaml please","the of and"] -> single mode R@1 1.0 over 2 of 3; all-vs-all own_rate 0.6667. A skill whose model calls all failed reads 0.0, the same as one the model really never picked. Expected: the single-mode denominator (scored/attempted queries), plus per-skill counts.
  • A lexical tie between two or more non-gold cards goes to the first card in the field. Three identical cards: order [fft, linalg, random] -> top attractor fft; reversed order -> random. Expected: the same result in any order, e.g. a (tie) bucket.
  • routing.json does not record the field rule. A default run and an --include-disabled run write the same keys. Expected: include_disabled, the competitor names and the left-out names in both payloads.
  • Pushed: d992d46 (the model arm records model); adbe542 (test that pins the attractor rate for a skill that has queries of its own; the old guard never ran).
  • Left for the author: the three points above.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at adbe542. The two commits since 924aba9 are fine: model is recorded in the model arm and the new attractor-rate test fails when the gold != name filter is removed. The four threads from my last review are unchanged at this head, so the PR is not ready: T1 and T2 are still risks.

What I ran, on a Linux review host at adbe542:

  • make ci-local: 28 passed, 1 failed, 7 skipped. The failure is pip-audit, which fails on the lock files; this PR does not touch them. Tests: 7,261 passed, 1 skipped.
  • Reproductions for the open threads, with the LLM transport mocked.
  • A mutation check of the two new tests: removing model=client.model fails both model-a/model-b cases (2 failed, 31 passed); removing the gold != name filter in sibling_queries fails the new attractor test (1 failed, 32 passed).

Still open from my earlier review:

  • dokimi/dokimi/commands/novelty.py:364: a partial model failure still exits 0 and prints OK Saved. Re-run at this head: 5 queries over db and perf, a transport that answers once then raises rate limited, gives exit 0, own_rate {db: 0.5, perf: 0.0} and n_failed 4.
  • dokimi/dokimi/novelty/routing.py:541: {"a":["x"],"a":["y"]} still loads as {'a': ['y']} with no error.
  • dokimi/dokimi/novelty/routing.py:600: the "not in the field" error still does not name disable-model-invocation or --include-disabled.
  • dokimi/dokimi/commands/novelty.py:375: --queries and --queries-from-headings are still ignored under --all-vs-all when --queries-by-skill is also given; --queries /nonexistent.json --queries-by-skill q.json --all-vs-all --out o.json prints OK Saved all-vs-all routing.

@xaleryb's points on the own_rate denominator, the lexical tie among non-gold cards and the missing field rule in the payload wait for the author.

Checked, no issues

  • Docs and changelog: the flags, outputs and defaults in skill-novelty.md, the walkthrough and the five fragments match the code, including the new model sentence; mkdocs build --strict passes.
  • Repo rules: SPDX header on the new test file, no schema or with_docs changes, no generated artifacts, changelog fragments in changelog.d/.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

# Conflicts:
#	dokimi/scripts/growth_baseline.json
#	dokimi/tests/test_novelty_routing.py
…ree, partial failures exit non-zero

Rates exclude unscored queries (no content word, failed model call) and
report null when a skill has none scored. A tie among other cards is (tie).
Repeated skill keys in the queries file are refused, the not-in-the-field
error names disable-model-invocation, --queries is refused with
--all-vs-all, and routing results record the field rule.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

Review-body items from @xaleryb's comment, in 5e27afa: (1) own_rate denominator: rates are now over scored queries; a query with no content word or a failed model call is unscored, reported in n_scored/n_unscored per skill, and a skill with none scored has own_rate null (test_a_query_with_no_content_word_is_not_scored_as_a_miss). (2) lexical tie among non-gold cards: two or more other cards tied at the top are a (tie) bucket in every field order, never an attractor (test_a_tie_among_other_cards_is_a_tie_in_any_field_order). (3) field rule: routing.json, single and all-vs-all, now carries field {include_disabled, competitors, left_out} (test_the_saved_payload_records_the_field_rule, test_a_single_artifact_routing_run_records_the_field_rule_too).

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

# Conflicts:
#	dokimi/docs/skill-novelty-walkthrough.md
#	dokimi/docs/skill-novelty.md
#	dokimi/dokimi/commands/novelty.py
#	dokimi/scripts/growth_baseline.json
@github-actions github-actions Bot added the cli-interface-change Changes dokimi's user-facing CLI surface label Oct 23, 2026
@github-actions

Copy link
Copy Markdown

CLI interface change detected

This PR changes dokimi's user-facing CLI surface (commands and/or options):

~ `dokimi novelty routing` parameters changed:
    + added option `--all-vs-all`
    + added option `--include-disabled`
    + added option `--queries-by-skill`
    ~ changed option `--artifact`: required: True -> False

Updated on every push; the cli-interface-change label is on while the diff is non-empty.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli-interface-change Changes dokimi's user-facing CLI surface

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants