Skip to content

fix(repo): tasks check --warm judges task host requirements - #182

Open
napetrov wants to merge 4 commits into
intel:mainfrom
napetrov:host-under-warm
Open

napetrov wants to merge 4 commits into
intel:mainfrom
napetrov:host-under-warm

Conversation

@napetrov

Copy link
Copy Markdown
Contributor

Part of #41 (D3 follow-up: host requirements under --warm).

Summary

 check_task(task, warm)
-  reason = None if warm else host_reason(task, probed_host)
+  reason = host_reason(task, probed_host)     # same container view (affinity, cgroup quota) with or without network
   null   runs only when the host meets the task
   oracle runs when the host meets it, or only CPUs are short
   verdict: host miss -> unsupported (allowed by --allow-unsupported)

repo run already judged the host on every path; only tasks check --warm skipped it. The null/oracle step moves to _check_step (radon CC of check_task 21 -> 13).

Evidence

Merge Danger

Door: two-way

Blast Radius: narrow

CI that runs tasks check --warm on a host missing a task's requirements now sees unsupported (exit 1 unless --allow-unsupported) instead of whatever null/oracle gave.

🤖 Generated with Claude Code

--warm skipped the host verdict, so on a host that misses a task's
requirements the untouched-tree null run passed and the task was
reported broken instead of unsupported; --allow-unsupported could not
pass it. The probe sees the same container view (affinity, cgroup
quota) with or without network. repo run already judged the host.
check_task's null/oracle step moves to _check_step (CC 21 -> 13).

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@codecov-commenter

codecov-commenter commented Oct 22, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.66%. Comparing base (5064dc2) to head (aeacdc0).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #182      +/-   ##
==========================================
+ Coverage   91.64%   91.66%   +0.02%     
==========================================
  Files         222      222              
  Lines       27409    27410       +1     
  Branches     4508     4508              
==========================================
+ Hits        25118    25126       +8     
+ Misses       1620     1615       -5     
+ Partials      671      669       -2     
Files with missing lines Coverage Δ
dokimi/dokimi/repos/selfcheck.py 95.95% <100.00%> (+3.10%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The fix is right: --warm now judges the host through the same _host_reason path as an offline check, and the _check_step extraction keeps the old behaviour for the null/oracle steps. Two nits inline.

What I ran, on a Linux review host at 9598a29:

  • make ci-local: 27 passed, 1 failed, 7 skipped. The failure is pip-audit, reporting a new gitpython 3.2.0 advisory (CVE-2026-114145) in the existing locks; it is not caused by the PR. 7089 tests passed; the changed-line coverage gate passed.
  • Mutation run on selfcheck.py: 24 mutants against the host/runner/run test files. Every mutant that changes which steps run, including restoring None if warm, is killed. 8 survive, all in the moved _check_step failure paths (see the second inline comment).

Checked, no issues

  • Equivalence: the old loop and _check_step match on prepare failure ("{kind}: {exc}", no verdict problems, other step still runs), on oracle non-zero/timeout, and on the skip condition (not reason / run_oracle).
  • Probe cost: run_check probes once per check and passes host to every check_task, so --warm adds no probes.
  • Probe view: probe_host runs with network="none" and the evaluation's runner flags, so --warm's network does not change the affinity or cgroup quota view.

host verdict, so on a host that misses a task's requirements the untouched-tree null run
could pass and the task was reported `broken` instead of `unsupported`, and
`--allow-unsupported` could not pass it. The probe is the same container view (affinity,
cgroup CPU quota) with or without network; `repo run` already judged it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the D3 fragment still says the opposite. dokimi/changelog.d/20261018_120000_napetrov_d3-host-requirements.md:22-23 (outside this diff) ends with "--warm does not judge the host." It is unreleased, so both entries ship in the same notes and contradict each other.

Suggestion: drop that last sentence from the D3 fragment. The module docstring at selfcheck.py:21-22 (outside this diff) could also say that --warm judges the host, for consistency with the check_task docstring.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d459193: the "--warm does not judge the host." sentence is gone from the D3 fragment and the selfcheck module docstring now says --warm judges the host.

f"; see {rd / 'oracle.log'}")
return
res[kind] = outcome(task, rd, run_entry(ctx, "grade", task, rd, net, container_name(rd.name, "grade")))
except (PrepError, ContainerError) as exc:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the failure paths of the extracted _check_step have no test. They moved here unchanged, but this PR is what makes them a function worth pinning (Codecov shows 6 changed lines uncovered, all here).

Mutation check: removing the return at selfcheck.py:119 (after an oracle failure), removing the return at :123, narrowing the except at :121 to PrepError only or to ContainerError only, dropping the {kind}: prefix, and replacing the grade run's net with "none" all pass the full host/runner/run test files (208 passed each). Removing the :123 return makes a prepare error fall through to res[kind] and raise KeyError.

Suggestion: add one test where _prepare raises PrepError for the null step and one where the oracle exits non-zero, asserting the null: ... / oracle exited N problem and that --warm passes the bridge network to the grade run.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Covered in 10e5aa8: new tests in tests/test_repo_host.py pin the _check_step failure paths (oracle failure return, prepare error prefix and the other step still running, grade-run net) plus --warm's network.

antonwolfy and others added 3 commits October 22, 2026 17:18
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants