ci: add e2e-staging workflow for nightly staging tests

[piotr-iohk]

Fixes synonymdev/bitkit-e2e-tests#221 (PR A — android)

Description

Adds a new workflow .github/workflows/e2e-staging.yml that runs staging E2E tests:

• Triggers:

  • schedule: runs daily at 04:00 UTC (after migration cron at 02:00)
  • workflow_dispatch: manual dispatch with inputs:
    • e2e_branch — branch of bitkit-e2e-tests to use
    • post_to_slack — checkbox to post summary to Slack (off by default)
  • No PR trigger — nightly + dispatch only

• Staging shards:

  • @multi_address_2
  • @pubky
  • @hardware_wallet
  • @transfer_max
  • @transfer_1 — temporarily omitted; will return in PR B / e2e-tests fix

• Build: Uses BACKEND=regtest with TREZOR_BRIDGE: true matching existing staging build config

• Slack: Posts summary via shared scripts/slack_summary.py e2e-staging from bitkit-e2e-tests repo (merged in ci: add shared e2e-staging Slack summary script bitkit-e2e-tests#223) to #bitkit-staging-nightly:

  • On schedule: always posts (success, failure, or cancelled)
  • On workflow_dispatch: posts only when post_to_slack checkbox is checked

• Concurrency: group is workflow-ref-event_name with cancel-in-progress: true — schedule and dispatch cannot cancel each other; back-to-back same-trigger runs still cancel in progress

• Timeout: timeout-minutes: 90 on e2e-tests-staging job (matches iOS)

• Test coverage: Added e2e-staging.yml to ARTIFACT_CONSUMERS in BuildOutputContractTest.kt

The existing e2e.yml workflow remains unchanged — the e2e-status merge gate is unaffected.

Design

N/A — no UI changes.

Preview

N/A

QA Notes

Manual Tests

N/A

Automated Checks

• Unit tests modified: BuildOutputContractTest.kt now includes e2e-staging.yml in ARTIFACT_CONSUMERS.
• Workflow concurrency (after merge): schedule and dispatch share a ref but not a group, so a master dispatch cannot cancel the nightly.

To dispatch after merge:

gh workflow run e2e-staging.yml -f e2e_branch=main
# or with Slack summary:
gh workflow run e2e-staging.yml -f e2e_branch=main -f post_to_slack=true

The workflow runs nightly at 04:00 UTC or via manual dispatch.

  

[github-actions]

Regtest APK

Built from 8ea050b (run).

Download bitkit-dev-debug universal APK (expires in 30 days).

[greptile-apps]

Confidence Score: 4/5

The workflow should not merge until scheduled and manual build or branch-selection failures reliably trigger the promised Slack notification.

Findings

1.  Upstream Failures Skip Alerts ▶
2.  Pull Requests Duplicate Shards ▶

Summary

• Builds a network-backed regtest APK with Trezor Bridge support.
• Runs multi-address, Pubky, hardware-wallet, and transfer shards with retries.
• Uploads failure artifacts and attempts to notify the staging Slack channel.
• The failure notification currently misses upstream failures, and the pull-request trigger duplicates existing staging shards on future PRs.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Trigger[Schedule / Manual / Pull Request] --> Build[build-staging]
  Trigger --> Branch[e2e-branch]
  Build --> Tests[e2e-tests-staging matrix]
  Branch --> Tests
  Tests -->|failure| Notify[notify-failure]
  Build -. failure skips tests .-> Missed[No Slack notification]
  Branch -. failure skips tests .-> Missed

Loading

_{Reviews (1) · Last reviewed commit: "ci: add e2e-staging workflow for nightly..."}

[ovi-reviewer]

Reviewed 1 file: adds a nightly and manually dispatched E2E Staging workflow that builds the staging regtest APK and runs four staging shards, notifying Slack when any upstream job fails. 1 non-blocking finding inline.

Coverage

Total: 0% (whole PR, 1 file)

• Journeys: 0% - the change is CI workflow config and exercises no user journey
• Unit tests: 0% - no production Kotlin changes; the repo's only test that reads workflow artifact paths is BuildOutputContractTest.kt
• QA: 0% - Manual Tests not run

---

Reviewed by Codex (deepseek-v4.1-flash-high) via gh-pr-review-loop skill

[jvsena42]

Two LOW observations inline, neither blocking. The staging targeting is right and the matrix does fail the run — traces below.

Cross-repo consistency with the iOS twin synonymdev/bitkit-ios#747, which I reviewed in the same pass:

• The Slack action is pinned to a different release on each side. Here, e598089… (line 248) is v2.0.0; iOS line 269 pins b0fa283… = v2.1.0. Both are SHA-pinned and both parse the YAML payload: form, so nothing breaks — but the pins were clearly meant to match, and one of the two should move.
• The Slack message bodies diverge. This one carries a title plus the run link (lines 253-258); iOS also includes workflow, run number, branch and github.event_name. Both post to the same SLACK_WEBHOOK_URL_STAGING, so the channel gets two differently-shaped alerts and the Android one will not say which trigger fired it.
• notify-failure conditions are logically identical on both sides (lines 237-243 here vs 257-263 there, just reordered). Shard sets differ only by hardware_wallet, which is Android-only and expected.

Checked and clean:

• Staging targeting is correct. The build (lines 55-65) sets E2E_BACKEND: network on assembleDevDebug, and every test attempt sets BACKEND: regtest (lines 186, 204, 221) — which per the e2e repo's README means the Blocktank API over the internet, not the docker stack. Matching build to runner backend is what that README requires, and this workflow correctly omits e2e.yml's Run regtest setup / Wait for electrum docker steps.
• All four shard tags resolve to real tests on bitkit-e2e-tests@main: @multi_address_2 (multiaddress.e2e.ts:116, also @regtest_only — consistent with BACKEND: regtest), @pubky (paykit.e2e.ts:138, pubky-profile.e2e.ts:47), @hardware_wallet (hardware-wallet.e2e.ts:33), @transfer_max (transfer.e2e.ts:303). No shard silently matches zero tests and passes green.
• The matrix really does fail the run. continue-on-error: true is on attempts 1 and 2 only (lines 174, 191); attempt 3 (line 208) has none, so a shard failing all three retries fails its job, and fail-fast: false still leaves e2e-tests-staging.result == 'failure'. The absence of an e2e-status-style aggregator is fine here — that job exists to give PRs a stable required-check name.
• E2E branch resolution on a cron run: github.event.inputs.e2e_branch is null on schedule, so line 82 falls through to default-feature-branch; determine-e2e-branch.yml probes refs/heads/master in bitkit-e2e-tests, which does not exist, and falls back to main as intended.
• No injection surface or secret exposure. No pull_request_target. The only ${{ }} values reaching a run: shell are matrix.shard.grep (workflow-literal) and needs.e2e-branch.outputs.branch, which is piped through an env: var at line 102 first. Secrets appear only in env: blocks and action inputs. GOOGLE_SERVICES_JSON_BASE64 decoding (lines 44-53) uses set -euo pipefail and an env indirection, same as the sibling.
• The schedule will fire — the workflow lands on master, the repo's default branch.
• Artifact names do not collide with e2e.yml (bitkit-e2e-apk-staging_* vs -regtest_*), and upload/download share the same github.run_number.

[jvsena42]

Re-reviewed at 2bcb3695c. Both of my notes are addressed and the timeout fix is right. The concurrency one needs a different shape — my original suggestion was misdirected, and I've corrected that comment. Three LOW notes, none blocking.

timeout-minutes: 90 — correct. Line 87, on e2e-tests-staging (not the build job), so it applies per matrix leg, and it matches iOS line 122. I sanity-checked the value against real data rather than eyeballing it: recent green e2e.yml runs (34816482602, 34810970657, 34792240593) give per-leg wall times of 10-24 min, so a worst-case 3-attempt leg lands around 75 min, inside the cap.

Concurrency — see the corrected comment on line 24. Short version: cancel-in-progress is the incoming run's knob, and the incoming run is the one that needs restraining, so conditioning it on github.event_name cannot protect a run already in flight — the discriminator has to be in the group key. That was my error last round, not yours; I gave you that expression.

The good news is that the half that actually mattered is fixed. The old gate was needs.e2e-tests-staging.result == 'failure', which never fires on a cancellation — confirmed empirically in cancelled run 34628569879, where notify-failure concluded cancelled. The new slack-report job runs on always() with STAGING_RESULT computed as success-only-if-all-three-succeeded, so a cancelled nightly now does post failure, and BUILD_RESULT: cancelled / E2E_TESTS_RESULT: cancelled in the payload make the message self-diagnosing. I verified always() survives concurrency cancellation using the structurally identical e2e-status job in e2e.yml: in cancelled runs 34485911442 and 34482039396 it ran and reported failure while its upstreams were cancelled. So the silent-nightly outcome is genuinely closed, which is why the residual is LOW.

Worth knowing: e2e_migration.yml in both repos has the same ref-only group with both schedule and workflow_dispatch, so the same latent issue predates this PR. If you change the shape here, it's worth doing all four. And the mechanism is demonstrably live on master — in iOS e2e_migration.yml on 2026-09-10, run 34531572432 was cancelled 21 seconds after 34531814945 was created, both workflow_dispatch on refs/heads/master.

The slack_summary.py refactor is a genuine improvement and I have no objection to it: the env-var contract matches exactly (all nine names the script reads are supplied, no orphans in either direction), and it strictly reduces injection surface — every ${{ }} now lands in an env: block or an action input, and line 252 is a literal command. It also removed a third-party action from both repos. Two notes on it inline, both non-blocking.

Convergence with iOS synonymdev/bitkit-ios#747 — both of last round's divergences are resolved. The differing Slack action pins are moot (slackapi/slack-github-action is gone from both files entirely), and the differently-shaped payloads into the same channel are gone too — both now render through the one shared script, differing only by the platform title. The timeout-minutes gap is closed. The notify condition and concurrency block are now byte-identical across the two repos, which unfortunately also means the concurrency note applies verbatim to both.

Re-verified clean at head: all four shard tags still resolve; continue-on-error on attempts 1-2 only with attempt 3 bare (175, 192, 209-211), so the matrix can still fail the run; E2E_BACKEND: network (61) and BACKEND: regtest (187, 205, 222) unchanged; GOOGLE_SERVICES_JSON_BASE64 decode still set -euo pipefail + env indirection (45, 51); artifact names still collision-free vs e2e.yml. The BuildOutputContractTest.kt addition is correct — the file contains app/build/outputs/bitkit/ at 69, 70 and 76 and neither forbidden AGP-native path.

One unrelated thing so it doesn't cost you time: the red build check on this head is not yours. It is QuickPayRepoTest > reconcile during live dispatched op does not steal completion, a Pending-vs-Success race — and master commit 50b2c4cab ("test: synchronize quickpay dispatch race") was already an attempt to fix that same race. #1224 and #1239 share this exact merge-base (57dea75e7) and their build checks passed. A re-run should clear it.

[piotr-iohk]

Suggest we merge with the remaining Lows and enhance later.

This is a new schedule + workflow_dispatch flow — hard to fully exercise Slack/nightly behaviour until it lands on master. The Low notes (Slack script checkout on feature e2e branches; fail-open webhook) are valid follow-ups, but they should not block landing PR A.

Happy to follow up in a small PR after we have a real scheduled/dispatch run to look at.

[ovi-reviewer]

Reaudit 3fa1500..2bcb369, 1 file. e2e-staging.yml: the Slack job is rewritten as slack-report, running under always() and reporting every outcome through the shared scripts/slack_summary.py, with timeout-minutes: 90 added to e2e-tests-staging. No findings.

Notes

• BuildOutputContractTest.kt now lists .github/workflows/e2e-staging.yml in ARTIFACT_CONSUMERS, and the workflow reads app/build/outputs/bitkit/ with neither AGP-native path the test forbids.
• The old gate keyed on needs.e2e-tests-staging.result == 'failure', which never fired on a cancellation. STAGING_RESULT now resolves to success only when all three upstream jobs succeeded, so a cancelled or skipped upstream reports failure instead of going silent.
• On the checkout ref at line 249: scripts/slack_summary.py is present on all seven branches of synonymdev/bitkit-e2e-tests at this head, and codex/pubky-ring-signup does not exist in that repository. A scheduled run resolves master to main, since that repository has no master branch, so the feature-branch dispatch failure described earlier is not reachable as stated.
• slack_summary.py fails open: main() calls post_to_slack(message) and returns 0 regardless, so a revoked webhook leaves the job green. That is upstream in synonymdev/bitkit-e2e-tests, not in this diff.
• cancel-in-progress gates the incoming run, so conditioning it on github.event_name cannot protect a run already in flight. The ref-only concurrency group predates this PR and is shared by e2e.yml and e2e_migration.yml.

Coverage

Total: 33% (delta diff since 3fa1500, 1 file)

• Journeys: 0% - the change is CI workflow config and a build-contract test, exercising no user journey
• Unit tests: 100% - the delta's only Kotlin change is covered by artifact consumers read the collected outputs in BuildOutputContractTest.kt
• QA: 0% - Manual Tests not run

---

Reviewed by Claude Code (claude-opus-5-xhigh) via gh-pr-review-loop skill

[cursor]

Concurrency is now aligned with Ovi's iOS review: event_name is in the group key so a master dispatch cannot cancel the scheduled nightly. Slack checkout stays on the resolved e2e branch.

[ovi-reviewer]

Verdict: ✅ Approve

---

Reaudit: delta diff
Reviewed: 1 file. e2e-staging.yml

Findings:
N/A
Security audit skipped - no security surface in the diff (score 0, threshold 3)

Coverage:
N/A

---

Reviewed by claude-opus-5-xhigh via gh-pr-review-loop skill.