Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4d51e0d
feat: render patched upstream workflow templates
vitormattos Sep 20, 2026
2865eaa
feat: report all upstream patch failures
vitormattos Sep 20, 2026
0576322
test: cover partial upstream patch failures
vitormattos Sep 20, 2026
3447d4e
ci: report upstream patch failures in refresh PR
vitormattos Sep 20, 2026
8a7a4cb
fix: avoid partial upstream refresh pull requests
vitormattos Sep 20, 2026
e3b19d2
chore: license upstream template manifest
vitormattos Sep 20, 2026
97fb19d
docs: describe patch-aware upstream refresh
vitormattos Sep 20, 2026
aad909e
refactor: publish generated GitHub workflow templates
vitormattos Sep 20, 2026
1fc9b08
refactor: publish appstore workflow as organization template
vitormattos Sep 20, 2026
edaa746
feat: add appstore workflow template metadata
vitormattos Sep 20, 2026
f4649de
chore: license workflow template metadata
vitormattos Sep 20, 2026
40c12f8
refactor: remove nonstandard generated template path
vitormattos Sep 20, 2026
245546c
refactor: include GitHub workflow templates in refresh PRs
vitormattos Sep 20, 2026
ec5f68f
docs: use native workflow template destination
vitormattos Sep 20, 2026
2fe51c8
docs: describe native GitHub workflow template publishing
vitormattos Sep 20, 2026
cd69c00
test: cover native workflow template destinations
vitormattos Sep 20, 2026
bc33da5
docs: define source and catalog responsibilities
vitormattos Sep 20, 2026
e6bd63b
docs: describe workflow template catalog publishing
vitormattos Sep 20, 2026
546e1e4
refactor: move refresh PR rendering into tested Python
vitormattos Sep 20, 2026
c84bce3
refactor: keep refresh workflow orchestration-only
vitormattos Sep 20, 2026
edd9abf
test: cover refresh pull request report rendering
vitormattos Sep 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 32 additions & 10 deletions .github/workflows/refresh-upstream.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,35 @@ jobs:
persist-credentials: false

- name: Refresh upstream pins
id: refresh
env:
GITHUB_TOKEN: ${{ github.token }}
run: python3 scripts/sync_upstream.py refresh upstream/sources.json

- name: Verify refreshed sources
- name: Verify refreshed immutable sources
run: python3 scripts/sync_upstream.py check upstream/sources.json

- name: Apply downstream patches
id: render
continue-on-error: true
run: |
python3 scripts/sync_upstream.py check upstream/sources.json
python3 -m unittest discover -s tests -p 'test_*.py'
python3 scripts/render_upstream.py sync upstream/templates.json \
--report render-report.json

- name: Run unit tests
if: always()
run: python3 -m unittest discover -s tests -p 'test_*.py'

- name: Build pull request report
if: steps.refresh.outcome == 'success'
run: >-
python3 scripts/render_upstream.py pr-body
--report render-report.json
--output refresh-upstream-pr.md

- name: Create update pull request
id: pull-request
if: steps.refresh.outcome == 'success'
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.WORKFLOW_UPDATE_TOKEN }}
Expand All @@ -45,14 +64,17 @@ jobs:
signoff: true
branch: automated/refresh-upstream-workflows
delete-branch: true
title: 'chore: refresh upstream workflow pins'
body: |
Automated refresh of tracked upstream workflow sources.

The committed source URLs remain pinned to immutable commit SHAs and
SHA-256 hashes. Review upstream changes and any downstream patches
before merging.
title: 'chore: refresh upstream workflows'
body-path: refresh-upstream-pr.md
draft: ${{ steps.render.outcome == 'failure' }}
labels: dependencies
add-paths: |
upstream/sources.json
upstream/vendor/**
workflow-templates/**

- name: Fail when patches need manual updates
if: steps.render.outcome == 'failure'
run: |
echo "One or more downstream patches could not be applied."
exit 1
5 changes: 4 additions & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,8 @@ jobs:
- name: Run unit tests
run: python3 -m unittest discover -s tests -p 'test_*.py'

- name: Verify generated templates
- name: Verify vendored upstream sources
run: python3 scripts/sync_upstream.py check upstream/sources.json

- name: Verify rendered upstream templates
run: python3 scripts/render_upstream.py check upstream/templates.json
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@ reference consumers and upstream sources, not hard-coded engine concepts.

## Repository layout

- `templates/` — generated or maintained reusable workflow templates.
- `workflow-templates/` — generated GitHub-native organization workflow templates ready for catalog publication.
- `upstream/` — immutable source manifests.
- `patches/` — explicit downstream adaptations.
- `scripts/` — deterministic synchronization/check tooling.
- `tests/` — tests for synchronization and template behavior.
- `docs/` — architecture, adoption and security guidance.

`LibreCodeCoop/.github` is the organization catalog used by GitHub's **Actions → New workflow** UI. This repository remains the source of truth; catalog publication should mirror generated templates rather than make `.github` a second editing source.

## Development

Run:
Expand Down
22 changes: 17 additions & 5 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ SPDX-License-Identifier: AGPL-3.0-or-later

## Responsibility boundary

`github-workflows` owns reusable CI and release automation.
`github-workflows` is the source of truth for reusable CI, imported workflow adaptations, generated workflow templates and release automation.

It does not manage repository rulesets. That responsibility belongs to
`LibreCodeCoop/github-governance`.
`LibreCodeCoop/.github` is the organization-facing catalog. Generated workflow templates can be published there so developers can discover them through GitHub's **Actions → New workflow** experience. The catalog is a distribution target, not the editing source.

Repository rulesets remain the responsibility of `LibreCodeCoop/github-governance`.

Consumer repositories own:

Expand All @@ -34,11 +35,13 @@ hash verification
↓
explicit downstream patches
↓
generated template
generated `workflow-templates/` artifact
↓
tests + actionlint + zizmor
↓
versioned release
publish catalog copy to `LibreCodeCoop/.github`
↓
versioned release / consumer update
```

The source manifest is authoritative. A network response that does not match the
Expand All @@ -61,3 +64,12 @@ Release automation is split into two stages:
- **apply:** explicit mutation and publication.

Credentials remain in the consumer repository or protected environment.

## Developer experience

The distribution model has two complementary entry points:

1. **Discovery / first install:** `LibreCodeCoop/.github/workflow-templates/` provides the GitHub-native template cards, metadata and optional icons.
2. **Ongoing updates:** consumer repositories receive reviewable update pull requests generated from the tested templates in this repository.

When a workflow can be expressed as a thin caller of a reusable workflow, prefer that model because fixes remain centralized. When GitHub Actions semantics require a full installed workflow, publish the generated workflow template and keep its downstream differences as explicit patches here.
62 changes: 34 additions & 28 deletions docs/upstream-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,23 +19,9 @@ The tracking ref can be mutable. The effective source cannot: after refresh, the
manifest is rewritten to a full commit SHA and content hash before the vendored
file is accepted.

Example:

```json
{
"sources": [
{
"name": "example",
"repository": "example/project",
"ref": "main",
"path": ".github/workflows/example.yml",
"url": "https://raw.githubusercontent.com/example/project/<commit>/.github/workflows/example.yml",
"sha256": "<64 lowercase hex characters>",
"destination": "upstream/vendor/example/example.yml"
}
]
}
```
Rendered downstream templates are declared separately in
`upstream/templates.json`. Each template points to one vendored source, an
ordered patch list and a generated workflow under GitHub's native `workflow-templates/` directory. Template metadata (`*.properties.json`) is maintained locally so LibreCode can provide its own names, descriptions, categories and icons without inheriting upstream branding.

## Commands

Expand All @@ -58,17 +44,37 @@ vendored files:
python3 scripts/sync_upstream.py refresh upstream/sources.json
```

The scheduled `refresh-upstream.yml` workflow runs this refresh weekly, validates
the result and opens a pull request when upstream changed.
Render vendored workflows with downstream patches:

A dedicated `WORKFLOW_UPDATE_TOKEN` secret is required for pull-request creation.
Using only the workflow's `GITHUB_TOKEN` would prevent the resulting pull request
from triggering the normal CI workflows. The refresh itself only uses the
read-only `GITHUB_TOKEN` to resolve public upstream commits.
```bash
python3 scripts/render_upstream.py sync upstream/templates.json
```

The renderer processes every declared template. Successful templates are updated.
If one or more patches no longer apply, those templates are left unchanged and
the renderer returns a structured report containing every failure.

Both `sync` and `check` verify the recorded source hash before accepting
content. `refresh` only records bytes fetched from the exact commit it resolved.
## Automated refresh

Patch application is intentionally a separate layer: upstream bytes remain
verbatim under `upstream/vendor/`, while downstream adaptations should be stored
as reviewable patches and rendered into generated templates.
The scheduled `refresh-upstream.yml` workflow:

1. resolves each tracked upstream workflow to its latest commit;
2. updates the immutable URL, SHA-256 and vendored bytes;
3. verifies the vendored sources;
4. attempts every downstream patch;
5. runs the test suite;
6. opens one pull request containing the upstream and successfully rendered changes.

If all patches apply, the pull request is normal. If any patch fails, the pull
request is opened as draft and its body lists each failed template, patch path and
error. The generated template for a failed patch remains at its previous known-good
version. The workflow then fails after creating the pull request so the problem is
also visible in Actions.

Failures while resolving, downloading or verifying upstream sources are treated
as fatal and do not create a partial update pull request.

A dedicated `WORKFLOW_UPDATE_TOKEN` secret is required for pull-request creation.
Using only the workflow's `GITHUB_TOKEN` would prevent the resulting pull request
from triggering the normal CI workflows. The refresh itself uses the read-only
`GITHUB_TOKEN` to resolve public upstream commits.
21 changes: 18 additions & 3 deletions patches/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,21 @@ SPDX-License-Identifier: AGPL-3.0-or-later

This directory contains explicit patches applied to imported upstream workflows.

The patch format and application contract will be introduced with the first
upstream workflow import and covered by tests. Generated templates must not hide
manual downstream edits.
Each rendered template is declared in `upstream/templates.json` with:

- an immutable vendored source under `upstream/vendor/`;
- zero or more ordered unified-diff patches from this directory;
- a generated destination under GitHub's native `workflow-templates/` directory.

Render all declared templates with:

```bash
python3 scripts/render_upstream.py sync upstream/templates.json
```

CI runs the corresponding `check` command and fails when a committed generated
template does not match its vendored source plus patches.

Patches should stay minimal. Product-specific behavior belongs in consumer
configuration unless the difference is required by the shared downstream
workflow contract.
13 changes: 13 additions & 0 deletions patches/nextcloud/appstore-build-publish.yml.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
--- appstore-build-publish.yml
+++ appstore-build-publish.yml
@@ -18,8 +18,8 @@ jobs:
build_and_publish:
runs-on: ubuntu-latest

- # Only allowed to be run on nextcloud-releases repositories
- if: ${{ github.repository_owner == 'nextcloud-releases' }}
+ # Downstream consumers publish from their own repositories.
+ # Repository policy is enforced by the consumer.

steps:
- name: Check actor permission
2 changes: 2 additions & 0 deletions patches/nextcloud/appstore-build-publish.yml.patch.license
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
SPDX-FileCopyrightText: 2026 LibreCode coop and contributors
SPDX-License-Identifier: AGPL-3.0-or-later
Loading
Loading