Repository navigation
Run CI "fire-and-forget" with --github-actions on Bencher Cloud #1046
Description
Hi! We're exploring adding Bencher to our GitHub CI (Pro plan): pgdogdev/pgdog#1617
We want to be able to run Bencher "fire-and-forget" instead of keeping a runner alive polling, so that we can terminate early (detach) from our Blacksmith runner where we run the Bencher CLI to start the task, and still allow Bencher to comment when it's done if there's any regressions. This doesn't seem possible right now with a GITHUB_TOKEN as it'll expire when the Action job ends (and, regardless, it doesn't look like that's sent to Bencher's servers)
Curious if we have any options around this? I don't mind contributing something up to address this, e.g., a per-project webhook that fires repository_dispatch to trigger a secondary workflow to add/update a comment.
Thanks!
Activity
epompeii commentedon Sep 24, 2026
@jkaczman thank you for using Bencher, and I'm sorry things aren't working as expected!
This is definitely a use case that I want Bencher to support!
Currently, you have to keep a CI runner waiting for Bencher Bare Metal results to complete if you want to have the results posted. There's already the
--detachflag however there's currently no mechanism for Bencher to push the results back to you.My current plan is to implement a per-
bencher runcallback mechanism. If you use the--detachflag, you can also specify a callback: URL, headers, and JSON body template. When the bare metal job reaches a terminal job lifecycle state, Bencher sends the callback POST.To make the GitHub Actions use case easier, if you specify the
--detachflag + the--github-actionsoption a callback will be created for you that invokesrepository_dispatchwith the necessary client payload. The--github-actionstoken would be aGITHUB_TOKENwith at leastpermissions: { checks: write }to set the check at the start of the run. A new--ci-callback-tokenoption would be added to pass in a fine-grained PAT for your repository with thecontents: writepermission to invoke therepository_dispatchin the callback.A new
--joboption would also be added that allows you to reattach to a job. In arepository_dispatchworkflow you would have a secondbencher runinvocation that uses the new--joboption to reattach to the completed job and report the results (PR comment, GitHub Check, etc) using that workflow'sGITHUB_TOKENwithpermissions: { pull-requests: write, checks: write }.Please, let me know what you think! cc @levkk
Add the Unknown job status (#1048)
Mark a job Unknown, not Failed, when its runner goes silent (#1049)
Make Failed and Canceled final job states (#1050)
Add the report and timeout to JsonJob (#1052)
Derive a callback key from the secret key (#1053)
Add the callback request type (#1055)
Add a callback sender that only reaches public addresses (#1056)
Store job callbacks and show their state on JsonJob (#1057)
Attach to a submitted job with bencher run --job (#1058)
Fire job callbacks once a job reaches a terminal state (#1059)
Add OpenTelemetry metrics for job callbacks (#1060)
Accept a run job callback, sealed for a paid plan or a GitHub dispatch (
Add callback flags to a detached bencher run (#1062)
Hand a detached GitHub Actions run's check to its attach (#1063)
Document job callbacks and pull requests on bare metal (#1064)
Add job callback steps to the runner smoke test (#1065)
epompeii commentedon Sep 28, 2026
@jkaczman the new callback functionality is now released as a part of
v0.6.13!Please, see the new Pull Requests on Bare Metal docs for an example of how to use it. You can also see the example live in the
examplerepo.You'll need to use the new
--ci-callback-tokenoption set to a PAT withcontents: writepermissions for your repository along with the--detachoption in the "sending" workflow. And in the "receiving"repository_dispatchworkflow, you will need to use the new--joboption.Please, try things out and let me know how it goes either way!
jkaczman commentedon Sep 28, 2026
Thank you! It looks good: pgdogdev/pgdog#1677
Send a free job callback only as the GitHub Actions dispatch from a c…
Send a free job callback only as the GitHub Actions dispatch from a c…
Pace job callbacks per credential and per host, and wait out rate limits
Send a free job callback only as the GitHub Actions dispatch from a c…
Pace job callbacks per credential and per host, and wait out rate lim…