DR-008 Option 4: two-stage test-execution workflow (PR 2 of 2) by Subramanian-K812 · Pull Request #280 · eclipse-score/reference_integration

Subramanian-K812 · 2026-07-07T08:48:54Z

DR-008 Option 4: two-stage test-execution workflow (PR 2 of 2)

Closes #264. This is the second of two PRs. It is stacked on PR 1
(Subramanian-K812_resolve_override_mechanism, "resolved-dependency resolve + override
mechanism") — please review/merge that one first. Until PR 1 merges, this PR's base branch
is the PR-1 branch, so the diff shown here is only the test-execution delta.

What this PR does

Rewires the quality workflow to the DR-008 Option 4 flow, using the resolve + override
mechanism added in PR 1:

Stage 1 — integration: builds the platform, runs Feature Integration Tests, and
exports the resolved dependency set as the stage1-resolved-deps artifact
(resolved_versions.json).
prepare_matrix: derives the Stage-2 module list dynamically from
known_good.json (target_sw) — never hardcoded.
Stage 2 — per module: checks the module out at its known_good commit, overrides
its MODULE.bazel with the Stage-1 resolved set (PR 1's mechanism), and runs the
module's own unit tests + coverage as the Bazel root (//...).
aggregate: consolidates Stage 1 + Stage 2 into one quality report (and the
release-tag test-report ZIP).

Files

Workflow & runners

.github/workflows/test_and_docs.yml — two-stage restructure (+ parallel docs build)
scripts/quality_runners.py — module-context mode (--module-dir / --resolved-deps); calls ResolvedDependencies.overwrite
scripts/aggregate_quality_report.py — new, consolidated report
scripts/known_good/list_modules.py — new, dynamic Stage-2 matrix from known_good.json
scripts/integration_test.py — derive build targets from known_good.json (drop build_config.json)

Resolved pins + their config ripple (kept together so the tree is self-consistent)

known_good.json — pin bumps (baselibs, lifecycle) + per-module bazel_config + lifecycle code_root_path/extra_test_config
bazel_common/score_basic_bazel.MODULE.bazel — flatbuffers bump
bazel_common/score_modules_target_sw.MODULE.bazel — regenerated (baselibs, lifecycle)
MODULE.bazel.lock — updated lock
rust_coverage/BUILD — lifecycle query //src/... → //score/... (layout moved by the bump)
showcases/simple_lifecycle/BUILD — lifecycle bin path //src/... → //score/launch_manager
.bazelrc — coverage atomic-gcov / -no-coverage; eb-aarch64 outline-atomics link fix

Known Stage 2 integration findings (surfaced, not fixed here)

The workflow surfaces two module-owned integration failures caused by the
baselibs bump (score_logging CheckSizeValid stale uint8_t assumption;
score_persistency to_string on std::string_view).

…ing it, and make resolved_dependencies importable standalone

… markers, simplify imports

…directive inline, add BUILD for bazel run

…wn_good.py, and tooling BUILD format

…wn_good tests

…own MODULE.bazel

github-actions · 2026-08-03T04:15:18Z

The created documentation from the pull request is available at: docu-html

…nstead of only its declared deps

…mechanism (rebased onto upstream)

…s, fix exit-code masking, wire transitive-dep fetch, and correct score_communication/score_time known_good pins

…CI verification

…ef_int's centralized bazelrc over the module's own

…lementation files

…un the unit tests in CI

…CI verification

PiotrKorkus

as the workflow has been completely reworked please create a private fork and execute it there, so we can see the proof of working in CI

pull_request_target wont allow to execute it here

PiotrKorkus · 2026-08-14T13:46:45Z

@@ -0,0 +1,221 @@
+# Stage 2 — Constraint Sheet
+
+Living document. Amend it when a constraint turns out to be wrong, and record why in the


If thats AI generated and maintained, where is a skill / agent for that purpose?

PiotrKorkus · 2026-08-14T13:47:09Z

where is this file generated from? All modules should be from known_good

not generated — known_good.json pins first-party repos by commit, these are registry deps pinned by version (confirmed via bazel mod graph --verbose: 2 of the 3 pins do move MVS's selection)

PiotrKorkus · 2026-08-14T13:47:54Z

+# 0.9.1 declares the four *_rules_rust_miri toolchains score_lifecycle_health and
+# score_persistency register in their own .bazelrc; 0.8.0 has none. ref_int cannot un-register
+# them, so the version must match here. Locked in score_test_artifact_versions.MODULE.bazel.


remove this comment

PiotrKorkus · 2026-08-14T13:50:32Z

+      - name: Set up Python 3
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'


why needed?

PiotrKorkus · 2026-08-14T13:51:25Z

+
+# One label for every Python unit test, so CI runs all of them by naming a single target.
+test_suite(
+    name = "python_tests",


change name, its meaningless without looking into target

Renamed to all_python_unit_tests.

PiotrKorkus · 2026-08-14T13:52:40Z

  push:
    branches:
      - main
+      - Subramanian-K812_test_execution_upgrade # TEMP: remove after CI verified


it doesnt work because the same workflow name is with pull_request_target on main

PiotrKorkus · 2026-08-14T13:53:36Z

+    runs-on: ubuntu-latest
+    steps:
+      - name: Clean disk space
+        uses: eclipse-score/more-disk-space@v1


use 1.1 version

Bumped to v1.1

PiotrKorkus · 2026-08-14T13:55:11Z

+          if [ ! -s _module/module_graph.json ]; then
+            echo "::warning::no module graph captured for ${{ matrix.module.name }}"; exit 0
+          fi
+          PYTHONPATH=scripts python3 scripts/known_good/verify_stage2_resolution.py \


if script needs a PYTHONPATH to be set its very bad design. Fix with proper modules imports

PiotrKorkus · 2026-08-14T14:14:25Z

+                    "bazel_config": [
+                        "stage2-gcc",
+                        "stage2-rust"


it feels like it doesnt need to be placed in this json as its common for every module

score_time and score_communication are exceptions for the above.

PiotrKorkus · 2026-08-14T14:14:42Z

                        "//score/mw/com/impl/bindings/lola/tracing:tracing_runtime_test"
+                    ],
+                    "bazel_config": [
+                        "stage2-rust"


why com has only rust?

score_communication passes use_base_constraints_only=True to its own gcc.toolchain(), which changes its generated toolchain target name leading to referencing a target that doesn't exist.

Subramanian-K812 · 2026-08-17T10:48:14Z

Ran the reworked workflow end-to-end on a private fork:
https://github.com/Subramanian-K812/reference_integration-ci-verify/actions/runs/32011971817

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from c9462ea to fd3df76 Compare July 7, 2026 08:49

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch 2 times, most recently from d293277 to b6df34d Compare July 24, 2026 06:56

Subramanian-K812 added 7 commits August 3, 2026 03:10

fix: overwrite() replaces a module's own override instead of duplicat…

fce6a51

…ing it, and make resolved_dependencies importable standalone

feat: add DR-008 resolved-dependency resolve + override mechanism

a8dcbb7

fix: remove bazel_config, always overwrite overrides, clean injection…

6f83e03

… markers, simplify imports

fix: revert update_module_from_known_good.py, move generate_override_…

e6ddcc9

…directive inline, add BUILD for bazel run

fix: ruff format fix for scripts/tooling/BUILD

e131bdd

fix: revert out-of-scope changes to module.py, update_module_from_kno…

529351d

…wn_good.py, and tooling BUILD format

feat: warn on unresolved declared deps, add bazel test target for kno…

4311861

…wn_good tests

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from b6df34d to e51b340 Compare August 3, 2026 03:12

feat: reach one level of transitive git_override deps via a module's …

d14e228

…own MODULE.bazel

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from e51b340 to 640dfee Compare August 3, 2026 03:26

feat: pin a module's full transitive closure from the Stage-1 graph i…

2959000

…nstead of only its declared deps

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch 3 times, most recently from 75e4c57 to cb3fbcc Compare August 10, 2026 07:26

Subramanian-K812 added 12 commits August 11, 2026 09:59

fix: key the Stage-2 pin scope on ref_int's resolved set

f9b4afe

feat: lock the versions of deps Stage 2 collects test artifacts from

fb20ff4

feat: DR-008 Option 4 two-stage test-execution workflow on canonical …

92fed52

…mechanism (rebased onto upstream)

fix: skip non-lcov genhtml output, tolerate non-UTF-8 subprocess byte…

eb0c90e

…s, fix exit-code masking, wire transitive-dep fetch, and correct score_communication/score_time known_good pins

ci: temp push trigger on Subramanian-K812_test_execution_upgrade for …

0f8ea37

…CI verification

feat: centralize Stage 2 module builds on ref_int's bazelrc/config

9b9c9a5

fix: split gcc toolchain config for score_communication

0b08e04

fix: register score_communication's libclang toolchain in Stage 2 config

7be8e62

feat: Stage 2 pins each module's full transitive closure and layers r…

df1aebb

…ef_int's centralized bazelrc over the module's own

feat: run Rust coverage inside Stage-2 checkout

8bbe458

fix: fail fast and accurately when a Stage-2 module runs zero tests

33f49cd

fix: floor Stage-2's Bazel version instead of overwriting

a5466d0

Subramanian-K812 added 5 commits August 11, 2026 10:07

fix: format issues in known_good scripts

4b06e8e

fix: attribute integration conflicts instead of ref_int harness defect

9268f85

chore: rebase onto resolve/override mechanism and scope to DR-008 imp…

e26c47d

…lementation files

fix: report the owner Stage 2 computed, remove dead code paths, and r…

3d06a2a

…un the unit tests in CI

fix: move the score_toolchains_rust lock and bazel_dep to 0.9.1

e816620

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 04a7a73 to e816620 Compare August 11, 2026 10:22

ci: temp push trigger on Subramanian-K812_test_execution_upgrade for …

c44af1f

…CI verification

PiotrKorkus requested changes Aug 14, 2026

View reviewed changes

fix: rename python test target, drop PYTHONPATH in Stage 2 verify script

104e15b

Subramanian-K812 added 2 commits August 18, 2026 09:56

fix: bump ref_int's score_crates pin

c992cdf

fix: re-add score_baselibs' aborts_upon_exception exclusion

913bc97

Subramanian-K812 force-pushed the Subramanian-K812_test_execution_upgrade branch from 1a9a26e to 913bc97 Compare August 18, 2026 11:02

		@@ -0,0 +1,221 @@
		# Stage 2 — Constraint Sheet

		Living document. Amend it when a constraint turns out to be wrong, and record why in the

Uh oh!

Conversation

Subramanian-K812 commented Jul 7, 2026

DR-008 Option 4: two-stage test-execution workflow (PR 2 of 2)

What this PR does

Files

Known Stage 2 integration findings (surfaced, not fixed here)

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

PiotrKorkus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Subramanian-K812 commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants