[nej1gotnochill]

Description

This PR adds lightweight GitHub Actions validation to the OpenMS-benchmarking repository, addressing the absence of automated pull-request checks and the fact that the existing benchmark test suites were not being executed by any workflow.

Changes:

• Add .github/workflows/pr-checks.yml, triggered on pull_request and pushes to main.
• Use Ubuntu 24.04 and Python 3.12 with read-only repository permissions (contents: read).
• Validate benchmark definitions before running the test suites.
• Add tests/validate_ci_test_coverage.py to ensure every tracked benchmark/test_*.py file is accounted for in the workflow's explicit test list.
• Detect missing, stale, duplicated, out-of-scope, and stale-exclusion entries in the test inventory.
• Execute the five existing Python test suites, shell syntax checks, and both shell contract tests.
• Keep the existing full benchmark workflows separate, avoiding expensive OpenMS builds and benchmark execution on every pull request.

The coverage guard deliberately does not execute newly discovered tests automatically. New test files must be explicitly reviewed and either added to the lightweight CI list or documented as deliberate exclusions.

Related issues:

Closes #17

Closes #18

Checklist

• Make sure that you are listed in the AUTHORS file (The target repository does not currently contain an AUTHORS file, so this item appears not to be applicable.)

• I have commented my code, particularly in hard-to-understand areas

• New and existing unit tests pass locally with my changes

• Updated or added python bindings for changed or new classes (No Python binding updates were necessary.)

• No CHANGELOG entry needed (The target repository contains no CHANGELOG file; the upstream note about entries being generated automatically once per week applies to OpenMS/OpenMS.)

Validation performed

The following checks passed locally:

• python benchmark/validate_definitions.py
• python tests/validate_ci_test_coverage.py
• python benchmark/test_openswath_metrics.py
• python benchmark/test_prose_metrics.py
• python benchmark/test_proteobench_compare.py
• python benchmark/report/test_generic_renderer.py
• python benchmark/report/test_v2_schema.py
• bash -n checks for all benchmark/*.sh and tests/*.sh scripts
• bash tests/test_id_validation.sh
• bash tests/test_dispatch_step.sh
• actionlint .github/workflows/pr-checks.yml
• git diff --check

The coverage guard was additionally tested against simulated missing, stale, duplicated, out-of-scope, nested, and stale-exclusion scenarios.

CI status: These checks were executed locally. A GitHub-hosted Actions run has not yet been verified; it will be evaluated after the PR is opened.

How can I get additional information on failed tests during CI

Click to expand

If your PR is failing you can check out

• The details of the action statuses at the end of the PR or the "Checks" tab.
• http://cdash.seqan.de/index.php?project=OpenMS and look for your PR. Use the "Show filters" capability on the top right to search for your PR number. If you click in the column that lists the failed tests you will get detailed error messages.

Advanced commands (admins / reviewer only)

Click to expand

• /reformat (experimental) applies the clang-format style changes as additional commit. Note: your branch must have a different name (e.g., yourrepo/XYZ) than the receiving branch (e.g., OpenMS). Otherwise, reformat fails to push. (Not applicable to this repository: it contains no C/C++ sources for clang-format to act on.)

⚠️ Note: Once you opened a PR try to minimize the number of pushes to it as every push will trigger CI (automated builds and test) and is rather heavy on our infrastructure (e.g., if several pushes per day are performed).

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[coderabbitai]

Important

• Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: b9a1daca-6f9e-4f17-a39f-737a6853c0ce

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn