Skip to content

ci: speed up build - #615

Merged
gilescope merged 40 commits into
mainfrom
ci-speed-up-build
Aug 9, 2026
Merged

ci: speed up build#615
gilescope merged 40 commits into
mainfrom
ci-speed-up-build

Conversation

@janishorsts

@janishorsts janishorsts commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

🛠️ Benefits, Issues Resolved & DX Impact

  • Benefits: Consolidated EarthBuild binary and container image builds into a single upfront job (fast-check-and-build), preventing redundant build steps across matrix jobs in CI.
  • Issues Eliminated: Eliminated duplicate compilation in downstream test matrixes (ci-docker-ubuntu, ci-podman-ubuntu, and ci-earthly-next-docker-ubuntu).
  • Developer Experience (DX) Impact: Dramatically reduced total CI pipeline execution time, faster PR feedback loops, and added repository naming standards to AGENTS.md clarifying earth and EarthBuild usage.

WARNING: This PR requires the change to "Protect default branch" ruleset.

NOTE: This PR preserves the existing workflow behaviour across all frontends. The goal is not to investigate why specific jobs run for frontend X and not frontend Y.

⚡ Performance Impact

Total time:

  • Before: ~24 minutes (3 parallel pipelines compiling standard and next Earth binaries independently, taking ~8 minutes each).
  • After: ~11 minutes (1 consolidated compilation phase with shared caching).

Reduces total execution time by 13 minutes.

Before

graph TD
    A[Code Commit / PR] --> B[Fast Check: Lint, Unit, Fuzz]
    B --> C[Docker Pipeline]
    B --> D[Podman Pipeline]
    B --> E[Next Pipeline]
    
    subgraph Docker Pipeline
        C --> C_Build["Build Earth (Docker tag) [~8 min]"]
        C_Build --> C_Tests["Run Docker Test Matrix"]
    end
    subgraph Podman Pipeline
        D --> D_Build["Build Earth (Podman tag) [~8 min]"]
        D_Build --> D_Tests["Run Podman Test Matrix"]
    end
    subgraph Next Pipeline
        E --> E_Build["Build Earth (Next tag) [~8 min]"]
        E_Build --> E_Tests["Run Next Test Matrix"]
    end
Loading
Screenshot 2026-06-29 at 16 43 02

After

graph TD
    A[Code Commit / PR] --> B[Fast Check & Build: Lint, Unit, Fuzz, Compile]
    
    subgraph Fast Check & Build Runner
        B --> B_BuildStd["Build Standard Earth"]
        B --> B_BuildNext["Build Next Earth"]
    end
    B_BuildStd --> C[Docker Pipeline]
    B_BuildStd --> D[Podman Pipeline]
    B_BuildNext --> E[Next Pipeline]
    subgraph Docker Pipeline Runner
        C --> C_Tests["Run Docker Test Matrix"]
    end
    subgraph Podman Pipeline Runner
        D --> D_Tests["Run Podman Test Matrix"]
    end
    subgraph Next Pipeline Runner
        E --> E_Tests["Run Next Test Matrix"]
    end
Loading
Screenshot 2026-07-29 at 08 43 04

@janishorsts janishorsts self-assigned this Jun 29, 2026
@janishorsts janishorsts added the ai-assisted Authored with AI assistance label Jun 29, 2026
@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

🎉 Are we earthbuild yet?

Great progress! You've reduced "earthly" occurrences by 250 (5.81%)

📈 Overall Progress

Branch Total Count
main 4300
This PR 4050
Difference -250 (5.81%)

📁 Changes by file type:

File Type Change
Go files (.go) ➖ No change
Documentation (.md) ❌ +1
Earthfiles ➖ No change

Keep up the great work migrating from Earthly to Earthbuild! 🚀

💡 Tips for finding more occurrences

Run locally to see detailed breakdown:

./.github/scripts/count-earthly.sh

Note that the goal is not to reach 0.
There is anticipated to be at least some occurences of earthly in the source code due to backwards compatibility with config files and language constructs.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the GitHub Action setup in .github/actions/stage2-setup/action.yml by hardcoding the string 'docker' instead of using the dynamic inputs.BINARY input for both the artifact name and the TAG environment variable. There are no review comments, so we have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@janishorsts

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the dynamic inputs.BINARY parameter with a hardcoded docker string in both the artifact download name and the image tag suffix within the stage 2 setup action. The reviewer notes that hardcoding docker reduces the action's reusability and flexibility, especially when running podman tests, and suggests using a fallback expression like ${{ inputs.ARTIFACT_SUFFIX || 'docker' }} instead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread .github/actions/stage2-setup/action.yml Outdated
Comment thread .github/actions/stage2-setup/action.yml Outdated
@janishorsts
janishorsts marked this pull request as ready for review July 29, 2026 13:52
@janishorsts
janishorsts requested a review from a team as a code owner July 29, 2026 13:52
@janishorsts
janishorsts requested review from gilescope and kmannislands and removed request for a team July 29, 2026 13:52
Comment thread .github/workflows/ci.yml Outdated
@kmannislands

Copy link
Copy Markdown

Seems more efficient (less runners used) but wait time to pass doesn't seem to decrease much. Last run on this branch was ~37 mins where the main runs are 37-44 mins over the past few.

I think we could decrease "wait time to green" by running more test steps in parallel

@janishorsts

Copy link
Copy Markdown
Collaborator Author

Seems more efficient (less runners used) but wait time to pass doesn't seem to decrease much. Last run on this branch was ~37 mins where the main runs are 37-44 mins over the past few.

I think we could decrease "wait time to green" by running more test steps in parallel

@kmannislands, yes, we can run more in parallel. I asked AI to do a performance analysis for successful runs.

🚀 GitHub Actions Performance Analysis (ci.yml)

Base Statement

Baseline Execution Time: Prior to the GitHub Actions workflow optimizations in this PR, successful full runs of the ci.yml workflow averaged 65.19 minutes (median 57.80 minutes / 3,468 seconds) across 173 analyzed baseline runs.


Impact Statement

Impact Statement: The workflow refactoring on this branch reduced the median successful CI execution time from 57.80 minutes to 37.97 minutes, achieving a 34.3% reduction in total run duration (~19.8 minutes faster per run). On average across all iterations, total run duration was reduced by 21.29 minutes (32.7% mean speedup).


Performance Comparison Breakdown

Metric Baseline (main / Other Runs) This PR (ci-speed-up-build) Improvement
P50 (Median) 57.80 min (3,468s) 37.97 min (2,278s) ⚡ 19.83 min faster (-34.3%)
Mean Duration 65.19 min (3,911s) 43.90 min (2,634s) ⚡ 21.29 min faster (-32.7%)
P99 (Worst Case) 167.31 min (10,039s) 111.85 min (6,711s) ⚡ 55.46 min faster (-33.1%)

Recent Successful PR Runs Benchmark

janishorsts and others added 3 commits July 30, 2026 12:57
Signed-off-by: Giles Cope <gilescope@gmail.com>

# Conflicts:
#	.github/workflows/reusable-secrets-integrations.yml
#	.github/workflows/reusable-test.yml
@gilescope
gilescope enabled auto-merge (squash) August 9, 2026 07:04
@gilescope
gilescope disabled auto-merge August 9, 2026 13:05
@gilescope
gilescope merged commit 49dbef3 into main Aug 9, 2026
92 checks passed
@gilescope
gilescope deleted the ci-speed-up-build branch August 9, 2026 13:05
gilescope added a commit that referenced this pull request Aug 9, 2026
Conflict in .github/actions/stage2-setup/action.yml: main gated the
"Set fixed buildkitd image for Docker 29+" step on BINARY == 'docker'
(#615). This branch replaces that step's body with the PR's own staging
image, so the docker-only guard no longer applies -- podman jobs must
exercise the PR's buildkitd too. Kept this branch's step, dropped the guard.

Also realigned the new "Save buildkitd image tarball" step's TAG_SUFFIX
with #615's removal of the -${BINARY} suffix; git merged it clean but the
tag it built no longer existed.

Assisted-by: Claude:claude-opus-5 claude-code
Signed-off-by: Giles Cope <gilescope@gmail.com>
gilescope added a commit that referenced this pull request Aug 9, 2026
Per review on #737: the tag suffix carried a runner dimension
(ubuntu-latest[-${BINARY}]) that never varied -- #615 already dropped
${BINARY}, leaving 'ubuntu-' as dead width in every staging tag and
artifact name. Two tags is the whole truth: latest and latest-ticktock.

Producer and consumer moved together, since a mismatch is only
observable as a 404 on the ghcr pull at test time:
  - ci.yml, build-earthly.yml       TAG_SUFFIX + upload-artifact name
  - stage2-setup/action.yml         download-artifact name, TAG, EARTHLY_BUILDKIT_IMAGE
  - ci-staging-deploy.yml           ${GITHUB_SHA}-<suffix> upgrade/mv pairs

build-earthly.yml now hardcodes the suffix rather than deriving it from
RUNS_ON, so RUNS_ON is a runner label only. That removes the silent
coupling to stage2-setup, which had 'ubuntu-latest' hardcoded on the
consumer side while the producer templated it.

runs-on: values are untouched.

Assisted-by: Claude:claude-opus-5 claude-code
Signed-off-by: Giles Cope <gilescope@gmail.com>
gilescope added a commit that referenced this pull request Aug 9, 2026
#615 moved the live CI build out of build-earthly.yml and into ci.yml's
fast-check-and-build, where the save+upload steps are still gated on
fork PRs. This branch made stage2-setup download unconditionally, so
after the merge every non-fork run (push, merge_group, same-repo PR)
had a consumer with no producer -- download-artifact v8 fails hard on a
missing artifact, which is every reusable-* test job.

Split each variant into an always-run tarball step and a fork-only
binary build, mirroring build-earthly.yml:
  - non-fork: image lives only in ghcr, so pull-if-absent before save
  - fork: --image already put it in the local daemon, inspect hits
  - binary stays fork-only; non-fork gets it via earth upgrade

Assisted-by: Claude:claude-opus-5 claude-code
Signed-off-by: Giles Cope <gilescope@gmail.com>
gilescope added a commit that referenced this pull request Aug 10, 2026
The earthly-count check flagged +7 lines. All seven were prose this
branch introduced -- comments and one step display name -- where
'earthly' meant the tool, not an identifier.

Left alone, because there the word IS the identifier and a rename would
make the comment wrong: DEFAULT_INSTALLATION_NAME=earthly, the
earthly-buildkitd container name tests assert on, +earthly/earthly,
./earthly upgrade, ~/.earthly, EARTHLY_BUILDKIT_IMAGE, and the
earthly-build-* artifact names.

Also drops a reference to build-earthly from stage2-setup, which #615
made stale -- the artifact it downloads now comes from ci.yml.

Net delta against main: 0.

Assisted-by: Claude:claude-opus-5 claude-code
Signed-off-by: Giles Cope <gilescope@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Authored with AI assistance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants