ci: add daily scheduled Test run that posts results to Slack#440
Conversation
Adds a `Daily CI Test` workflow on a 12:00 UTC cron that runs the workspace test suite and reports pass/fail to Slack, so a green main is verified daily independent of PR activity. To avoid duplicating the ~70 lines of fixture-download + Rust setup logic, the CI Test job body is extracted into a reusable composite action (`.github/actions/run-fixture-tests`) that both `ci.yml` and the new daily workflow consume. The CI Test job behavior is unchanged. Slack wiring mirrors daily_loc_report.yml: scheduled runs post to the prod webhook, manual workflow_dispatch defaults to the test channel. The notify job runs with `if: always()` so failures are reported too.
🤖 Kimi Code ReviewOverall Assessment: This PR cleanly extracts fixture test logic into a reusable composite action and adds daily CI with Slack notifications. The changes improve maintainability but introduce minor security and reliability considerations typical of CI/CD workflows. Security & Reliability1. Supply Chain Risk — Unpinned Third-Party ActionsFile: The composite action uses mutable references for third-party actions:
Recommendation: Pin to specific commit SHAs: uses: dtolnay/rust-toolchain@7f2b3f8a6e1f4a5c2d8e9f0a1b2c3d4e5f6a7b8c # v1.92.02. Fragile JSON Parsing Without ValidationFile: The Python one-liners use Recommendation: Add error handling: fixtures_url=$(echo "$json" | python3 -c "
import sys, json
j = json.load(sys.stdin)
assets = j.get('assets', [])
url = next((a.get('browser_download_url') for a in assets if a.get('name') == 'fixtures-prod-scheme.tar.gz'), None)
if not url:
print('Error: fixtures-prod-scheme.tar.gz not found in release assets', file=sys.stderr)
sys.exit(1)
print(url)
")3. Missing Referenced ScriptFile: The workflow executes
4. Workspace Path AssumptionsFile:
rm -rf "${{ github.workspace }}/leanSpec/fixtures"Correctness & Maintainability5. Cache Poisoning ProtectionFile: The logic to save cache only on successful download ( 6. Daily CI ConcurrencyFile: The concurrency group prevents overlapping daily runs, which is correct for resource-intensive consensus tests. However, consider adding 7. Slack Webhook Selection LogicFile: The ternary expression for webhook selection is correct: ${{ (github.event_name == 'schedule' || inputs.target == 'prod') && secrets.ETHLAMBDA_GENERAL_SLACK_WEBHOOK || secrets.ETHLAMBDA_TEST_SLACK_WEBHOOK }}The Minor Improvements8. Shell Quoting ConsistencyWhile most variables are properly quoted, line 52-53 in the action use unquoted static paths. Though low risk, consistent quoting is better practice. 9. Python vs jqUsing Python for JSON parsing is portable but verbose. If fixtures_url=$(echo "$json" | jq -r '.assets[] | select(.name=="fixtures-prod-scheme.tar.gz") | .browser_download_url')Consensus-Specific ConsiderationsSince this PR only touches CI infrastructure, there are no direct changes to attestation processing, XMSS signatures, or fork choice. However, ensure that the Status: Approve with minor comments. The SHA256 verification of fixtures and proper cache invalidation logic are well-implemented. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code ReviewNo consensus-path Rust code is touched here; this PR is CI/workflow-only. I found a few infrastructure issues worth fixing before merge.
Aside from those points, the extracted composite action in Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewHere is my analysis of PR #440: PR #440 Review: Daily Scheduled CI Test with Slack NotificationOverviewThis PR refactors the existing CI test job into a reusable composite action ( CorrectnessComposite action extraction (
Daily workflow (
SecuritySupply-chain risks (both workflows)
Unauthenticated GitHub API call The fixture release info fetch: curl -sL "https://api.github.com/repos/leanEthereum/leanSpec/releases/latest"runs unauthenticated, which is subject to GitHub's 60 requests/hour rate limit for the runner's IP. When multiple workflow runs share a runner pool this can hit the limit. Passing Heredoc JSON injection In "text": "$emoji *Daily CI Test* $status on \`${{ github.ref_name }}\`\n<$run_url|View run #${{ github.run_number }}>"
Minor / Style
SummaryThe core design is sound: the composite action extraction is clean, the cache logic is correct, and the daily workflow correctly handles failure notification. The main actionable items are:
Automated review by Claude (Anthropic) · sonnet · custom prompt |
Greptile SummaryThis PR introduces a daily scheduled CI test workflow that runs the full workspace test suite at 12:00 UTC and posts pass/fail results to Slack. To avoid duplicating ~70 lines of fixture-download and Rust setup logic, that logic is extracted into a new reusable composite action (
Confidence Score: 4/5Safe to merge — the composite action is a clean extraction with no behavioral changes to the existing CI, and the daily workflow wires Slack notifications correctly following the established pattern. The refactor of the Test job into a composite action is mechanical and well-executed. The daily workflow mirrors
|
| Filename | Overview |
|---|---|
| .github/actions/run-fixture-tests/action.yml | New composite action extracting fixture-download + Rust-setup + test-run logic from ci.yml. Logic is verbatim from the original job; adds shell: bash to all run steps (a correct requirement for composite actions), and the cache-save guard is preserved correctly. |
| .github/workflows/ci.yml | Test job body replaced with a single uses: ./.github/actions/run-fixture-tests call. Behavior is unchanged; the composite action step does not need extra inputs or permissions beyond what the job already has. |
| .github/workflows/daily_ci.yml | New daily-scheduled workflow. Slack-webhook routing and concurrency config mirror daily_loc_report.yml correctly. One P2: the JSON payload is built via heredoc with unescaped ${{ github.ref_name }} — consider using jq for safe construction. |
Sequence Diagram
%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Cron as Cron / workflow_dispatch
participant Test as Job: test
participant Action as .github/actions/run-fixture-tests
participant GH as GitHub API (leanSpec)
participant Cache as Actions Cache
participant Notify as Job: notify
participant Slack as Slack Webhook
Cron->>Test: trigger (12:00 UTC or manual)
Test->>Action: uses: run-fixture-tests
Action->>GH: GET /releases/latest (fixtures URL + SHA)
Action->>Cache: "restore leanspec-fixtures-{sha}"
alt cache miss
Action->>GH: curl fixtures tarball + .sha256
Action->>Action: verify SHA256
Action->>Cache: "save leanspec-fixtures-{sha}"
end
Action->>Action: make test (Rust workspace)
Action-->>Test: outcome (success / failure)
Test-->>Notify: needs.test.result
Notify->>Notify: build Slack JSON payload
Notify->>Slack: POST ci_result_slack.json
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Cron as Cron / workflow_dispatch
participant Test as Job: test
participant Action as .github/actions/run-fixture-tests
participant GH as GitHub API (leanSpec)
participant Cache as Actions Cache
participant Notify as Job: notify
participant Slack as Slack Webhook
Cron->>Test: trigger (12:00 UTC or manual)
Test->>Action: uses: run-fixture-tests
Action->>GH: GET /releases/latest (fixtures URL + SHA)
Action->>Cache: "restore leanspec-fixtures-{sha}"
alt cache miss
Action->>GH: curl fixtures tarball + .sha256
Action->>Action: verify SHA256
Action->>Cache: "save leanspec-fixtures-{sha}"
end
Action->>Action: make test (Rust workspace)
Action-->>Test: outcome (success / failure)
Test-->>Notify: needs.test.result
Notify->>Notify: build Slack JSON payload
Notify->>Slack: POST ci_result_slack.json
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
.github/workflows/daily_ci.yml:56-72
**Heredoc JSON construction embeds expressions without sanitization**
`${{ github.ref_name }}` is expanded by the Actions expression engine before the shell runs, and the resulting string is written verbatim into a JSON value without escaping. If the ref name ever contains a `"` or `\` character (unlikely on GitHub but possible with other git hosting), the output file will be malformed JSON and the subsequent `curl` to Slack will fail silently (the `publish_slack.sh` script doesn't validate JSON before posting). Using `jq` to assemble the payload is the standard mitigation — it handles all escaping automatically. The `$run_url` and `${{ github.run_number }}` expansions share the same pattern but are lower risk since those values are always safe strings.
Reviews (1): Last reviewed commit: "ci: add daily scheduled Test run that po..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| cancelled) emoji="⚠️"; status="was cancelled";; | ||
| *) emoji="❔"; status="finished with status \`$TEST_RESULT\`";; | ||
| esac | ||
| run_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" | ||
| cat > ci_result_slack.json <<EOF | ||
| { | ||
| "blocks": [ | ||
| { | ||
| "type": "section", | ||
| "text": { | ||
| "type": "mrkdwn", | ||
| "text": "$emoji *Daily CI Test* $status on \`${{ github.ref_name }}\`\n<$run_url|View run #${{ github.run_number }}>" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| EOF |
There was a problem hiding this comment.
${{ github.ref_name }} is expanded by the Actions expression engine before the shell runs, and the resulting string is written verbatim into a JSON value without escaping. If the ref name ever contains a " or \ character (unlikely on GitHub but possible with other git hosting), the output file will be malformed JSON and the subsequent curl to Slack will fail silently (the publish_slack.sh script doesn't validate JSON before posting). Using jq to assemble the payload is the standard mitigation — it handles all escaping automatically. The $run_url and ${{ github.run_number }} expansions share the same pattern but are lower risk since those values are always safe strings.
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/daily_ci.yml
Line: 56-72
Comment:
**Heredoc JSON construction embeds expressions without sanitization**
`${{ github.ref_name }}` is expanded by the Actions expression engine before the shell runs, and the resulting string is written verbatim into a JSON value without escaping. If the ref name ever contains a `"` or `\` character (unlikely on GitHub but possible with other git hosting), the output file will be malformed JSON and the subsequent `curl` to Slack will fail silently (the `publish_slack.sh` script doesn't validate JSON before posting). Using `jq` to assemble the payload is the standard mitigation — it handles all escaping automatically. The `$run_url` and `${{ github.run_number }}` expansions share the same pattern but are lower risk since those values are always safe strings.
How can I resolve this? If you propose a fix, please make it concise.
The heredoc embedded ${{ github.ref_name }} (user-controlled on
workflow_dispatch) directly into a JSON value. A ref name containing a
quote, backslash, or newline would produce malformed JSON and a silent
Slack failure. Pass context values through the environment and assemble
the payload with jq, which escapes them correctly.
Addresses PR #440 review comment r3423800540.
2 participants
What
Adds a Daily CI Test workflow that runs the workspace test suite on a
0 12 * * *(12:00 UTC) cron and posts the result (pass/fail) to Slack.To avoid duplicating the ~70 lines of fixture-download + Rust setup logic, the CI
Testjob body is extracted into a reusable composite action (.github/actions/run-fixture-tests) that bothci.ymland the new daily workflow consume.Changes
.github/actions/run-fixture-tests/action.ymlmake test. Extracted verbatim from CI's Test job..github/workflows/ci.ymluses: ./.github/actions/run-fixture-tests. Behavior unchanged..github/workflows/daily_ci.ymlnotifyjob (if: always()) posting to Slack with a link to the run.Slack wiring
Mirrors
daily_loc_report.yml:ETHLAMBDA_GENERAL_SLACK_WEBHOOK(prod)workflow_dispatch→ defaults toETHLAMBDA_TEST_SLACK_WEBHOOK(test channel), selectable viatargetinputNo new secrets required.
Testing
workflow_dispatch(target: test) to dry-run against the test channel before the first scheduled run.