Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .coverage-baseline
Original file line number Diff line number Diff line change
@@ -1 +1 @@
80.2
80.3
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ Dockerfile.cross
# The GitOps layout fixtures check in a kustomize secretGenerator env file on purpose.
# Its contents are placeholder text, never a real credential.
!test/fixtures/gitops-layouts/**/*.env
# So do the configMapGenerator write-through fixtures: their env files are the input a live
# ConfigMap edit is written into.
!test/fixtures/layout-corpus/**/*.env
!test/e2e/fixtures/**/*.env

# Test binary, built with `go test -c`
*.test
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,16 @@ repos, branches, and folders. See [Configuration](docs/configuration.md).
| Your source in Git | What you can do |
|---|---|
| Plain Kubernetes manifests | Capture resources and update their existing YAML documents |
| Supported Kustomize layouts | Update source manifests or image/replica declarations; add and remove resources |
| Supported Kustomize layouts | Update source manifests, image/replica declarations, and `configMapGenerator` input files; add and remove resources |
| A Flux `HelmRelease` or Argo CD `Application` | Capture the declaration, including chart versions and inline values |
| Helm templates or standalone `values.yaml` | No writeback from rendered workloads |

Local Kustomize bases and overlays work, with the base kept read-only. Generators, components, and
remote bases are unsupported; such a folder is refused with `Stalled=True` before anything is
written. See the [supported subset](docs/configuration.md#kustomize-support-in-the-target-path).
Local Kustomize bases and overlays work, with the base kept read-only. A `configMapGenerator` with
`disableNameSuffixHash: true` is writable: a ConfigMap edit lands in the file, `.env` line or
literal it came from, and deleting the ConfigMap removes its generator entry. `secretGenerator`,
hash-suffixed generators, components, and remote bases are unsupported; such a folder is refused
with `Stalled=True` before anything is written. See the [supported
subset](docs/configuration.md#kustomize-support-in-the-target-path).

Watch the resources that express your intent, such as a `HelmRelease` rather than the workloads it
renders: the operator cannot tell authored resources from generated ones. See
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -1391,7 +1391,8 @@ write side is shared with live writes):

- before anything is planned, a **structure-only acceptance gate** runs over the scanned subtree; if it
finds content the operator cannot safely manage: a kustomization using an unsupported feature
(generators / inline or JSON6902 patches / components / helm / replacements / transformers /
(`secretGenerator` or a hash-suffixed or merging `configMapGenerator` / inline or JSON6902 patches /
components / helm / replacements / transformers /
name(pre|suf)fix / remote bases) or malformed `images:`/`replicas:` overrides, a duplicate manifest
identity, an impure managed file, or a standalone non-KRM / invalid YAML;
(a path-based strategic-merge `patches:` entry is tolerated as read-only build context, and an
Expand Down
4 changes: 3 additions & 1 deletion docs/bi-directional.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,9 @@ Argo CD stamps `argocd.argoproj.io/tracking-id` on every non-CRD object it appli
the live object would stop matching the committed file, and every apply would produce a commit.

GitOps Reverser strips all of it before writing, along with `argocd.argoproj.io/installation-id`,
`kro.run/`, `applyset.kubernetes.io/`, and `kcp.io/cluster`; the rules and reasons are in
`kro.run/`, `applyset.kubernetes.io/`, `kcp.io/cluster`, and kustomize's build provenance
(`config.kubernetes.io/origin`, `alpha.config.kubernetes.io/transformations`), which Flux requests
for a `kustomization.yaml` with no `resources:`; the rules and reasons are in
[`internal/sanitize/types.go`](../internal/sanitize/types.go). Under `argocd.argoproj.io/` only the
exact tracking keys go: `sync-wave`, `sync-options` and `hook` are user intent that belongs in Git.
That is also why the loop settles: an applied commit GitOps Reverser wrote matches the live object,
Expand Down
15 changes: 14 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2650,6 +2650,17 @@ A target path may contain `kustomization.yaml` files. The operator retains them
keeps its bytes. Only fields the entry already declares are updated; the operator never adds or
removes entries. Note that one entry is a shared knob, exactly as in kustomize itself: updating it
affects every resource in the build whose image matches.
- **`configMapGenerator` with `disableNameSuffixHash: true`** (in `generatorOptions:` or on the
entry): a live change to the generated ConfigMap is written into the generator's inputs. A
`files:` key's file gets the live value as its content; an `envs:` key's line and a `literals:`
key's `KEY=value` item are edited in place, and a removed key's line or item is removed. A new key
is appended to the entry's `envs:` file when it has exactly one, and otherwise to its `literals:`
when it has some. Deleting the ConfigMap removes the entry from `configMapGenerator:`, and the
input files no other entry, resource or patch reads; under `spec.prune.mode: Always` a resync
sweeps the entry of a generated ConfigMap the cluster no longer has, as it sweeps a document. A
change with no single input to land in is refused per object: `binaryData`, a label or annotation,
a new key with several `envs:` files or only `files:` sources, a removed `files:` key, and a
generator in a read-only base. The input files are build inputs, never foreign content.

Two kustomize shapes beyond that subset are **supported without being authored**:

Expand All @@ -2662,7 +2673,9 @@ Two kustomize shapes beyond that subset are **supported without being authored**

Everything else outside the modeled subset **refuses the whole target path before anything is
written**: inline or JSON6902 patches and the deprecated `patchesStrategicMerge`/`patchesJson6902`
spellings, generators, `components`, Helm fields, `replacements`, `transformers`,
spellings, `secretGenerator`, a `configMapGenerator` whose name hash is not disabled (the message
says to set `disableNameSuffixHash: true`) or whose `behavior:` is `merge` or `replace`,
`components`, Helm fields, `replacements`, `transformers`,
`namePrefix`/`nameSuffix`, remote bases, and `images:`/`replicas:` values that do not parse (those
would fail `kustomize build` too). A refusal is loud: the target reports `GitPathAccepted=False`,
`Stalled=True`, and `Ready=False` with reason `UnsupportedContent` until the path is cleaned up.
Expand Down
6 changes: 4 additions & 2 deletions docs/design/support-boundary/kustomize-support-boundary.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@ language.
| Class | Fields | Boundary |
|---|---|---|
| Editable declarations | local `resources`/`bases`, `namespace`, `images`, `replicas` | supported where the layout provides one write home |
| Editable inputs | `configMapGenerator` with `disableNameSuffixHash: true` and no `behavior: merge`/`replace` | an edit lands in the `files:` file, on the `envs:` line or in the `literals:` item it came from; a delete removes the entry |
| Tolerated build context | `patches:` with local strategic-merge `path:` documents | rendered and retained, but neither authored nor edited |
| Planned authoring surface | scalar strategic-merge patch for an inherited field | see [patch authoring](patch-authoring.md) |
| Refused | inline/JSON6902/deprecated patches, generators, `components`, `namePrefix`/`nameSuffix`, remote bases, `helmCharts`, plugins, `replacements`, `vars` | no safe or owned inverse in the current model |
| Refused | inline/JSON6902/deprecated patches, `secretGenerator`, hash-suffixed or merging `configMapGenerator`, `components`, `namePrefix`/`nameSuffix`, remote bases, `helmCharts`, plugins, `replacements`, `vars` | no safe or owned inverse in the current model |
| Refused for editor safety | `configurations`, `openapi`, `crds` | they alter merge semantics that the writer must not guess |

`labels`, `commonLabels`, and `commonAnnotations` are not field-authoring channels.
Expand Down Expand Up @@ -92,7 +93,8 @@ described in [patch authoring](patch-authoring.md) exist.
- No chart inflation or remote-base resolution.
- No plugin execution or controller-side transform emulation.
- No write-through to a shared base or another environment.
- No generic inversion of patches, generators, name references, or merge keys.
- No generic inversion of patches, name references, or merge keys, and no generator beyond a
hash-disabled `configMapGenerator`.

The fixture corpus under [`test/fixtures/gitops-layouts/`](../../../test/fixtures/gitops-layouts/)
is the executable evidence for these verdicts. The more detailed history of the renderer and
Expand Down
3 changes: 2 additions & 1 deletion docs/design/support-boundary/support-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ templates, and multi-input ResourceSet templates alike. It is specified once, in
| kustomize `patches:` — a strategic-merge document named by `path:` | **Tolerated, not authored** | the folder is accepted and the render is mirrored; the patch is read-only build context. An edit to a field the patch OWNS is refused per object, not per folder |
| kustomize `patches:` — inline, JSON6902, or a path outside the tree | **Refused** | not a sparse KRM document we can read; refused by name |
| kustomize `patchesStrategicMerge`, `patchesJson6902` (deprecated spellings) | **Refused** | kustomize does not fold them into `patches:` (measured), so they refuse under their own names |
| kustomize generators, `components`, `namePrefix`/`nameSuffix`, remote bases | **Refused** | non-invertible |
| kustomize `configMapGenerator` with `disableNameSuffixHash: true` | **Editable** | the edit lands in the `files:` file, on the `envs:` line or in the `literals:` item it came from, and a delete removes the entry, checked by re-render; a change with no single input is refused per object |
| kustomize `secretGenerator`, hash-suffixed or merging `configMapGenerator`, `components`, `namePrefix`/`nameSuffix`, remote bases | **Refused** | non-invertible |
| kustomize `helmCharts:` (inflation) | **Refused** | we never render a chart |
| A Helm chart (`Chart.yaml` + `templates/` + `values.yaml` + `crds/`) | **Planned: skipped as a unit** | this needs chart-folder detection; today `scan-repo` can report an incidental `crds/` directory as accepted |
| Helm knobs on a `HelmRelease` / `Application` (chart version, inline values, parameters) | **Editable** | the Helm surface people actually use — see below |
Expand Down
8 changes: 4 additions & 4 deletions docs/design/support-boundary/values-file-projection.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,11 @@ The same Helm values, expressed six ways in the corpus, get five different verdi
| `HelmRelease.spec.values` inline, `valuesFrom` → a hand-written `ConfigMap` | `flux-helmrelease/infrastructure/controllers/ingress-nginx/` | **Editable** |
| a `values.yaml` alone in a `values/` directory | `argocd-external-helm/values/**` | **Not seen at all** — no KRM in the dir, so never a candidate |
| a `values.yaml` **next to** KRM | `argocd-external-helm/platform/cert-manager/values.yaml` | **Refused** — and it takes the folder with it |
| a `values.yaml` wrapped by a `configMapGenerator` | `flux-helmrelease/apps/frontend/` | **Refused** — the generator is on the deny-list |
| a `values.yaml` wrapped by a `configMapGenerator` | `flux-helmrelease/apps/frontend/` | **Editable** — with `disableNameSuffixHash: true`, the file is the generator's input |

Read that column again. Values are editable when a human wraps them in a `ConfigMap`, and
refused when `configMapGenerator` does the identical wrapping — with
`disableNameSuffixHash: true`, so the result is deterministic. They are editable as a *string*
Read that column again. Values are editable when a human wraps them in a `ConfigMap`, and, since
the generator write-through, when a `configMapGenerator` with `disableNameSuffixHash: true` does
the identical wrapping: the live edit lands in the file. They are editable as a *string*
inside an Application, and refused as a *file* on disk. They are invisible when alone and
fatal when adjacent.

Expand Down
3 changes: 2 additions & 1 deletion docs/facts/expansion-provenance-markers.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,8 @@ belongs in the evidence table, not as a replacement for it.
[`internal/sanitize/types.go`](../../internal/sanitize/types.go) strips
`kustomize.toolkit.fluxcd.io/` (labels + annotations), `kro.run/` (labels only),
`applyset.kubernetes.io/`, and — by exact key — `argocd.argoproj.io/tracking-id`,
`argocd.argoproj.io/installation-id`, and `kcp.io/cluster`.
`argocd.argoproj.io/installation-id`, `kcp.io/cluster`, `config.kubernetes.io/origin`, and
`alpha.config.kubernetes.io/transformations`.

It does **not** strip, and these are all controller bookkeeping observed on real
objects:
Expand Down
184 changes: 184 additions & 0 deletions internal/git/generator_write.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
// SPDX-License-Identifier: Apache-2.0

package git

import (
"context"
"errors"
"fmt"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/log"

"github.com/ConfigButler/gitops-reverser/internal/git/manifestedit"
"github.com/ConfigButler/gitops-reverser/internal/manifestanalyzer"
"github.com/ConfigButler/gitops-reverser/internal/manifestreport"
"github.com/ConfigButler/gitops-reverser/internal/types"
)

// writeGenerated writes a live ConfigMap that a configMapGenerator renders back into the
// generator's input files: a files: source gets the live value as its content, an envs: line is
// edited in place. manifestanalyzer.PlanGeneratedWrite decides where each key goes; the oracle
// then re-renders the folder and refuses the flush unless the generator produces the live object.
//
// A change with no destination (binaryData, a label, a key with no single input to land in) is
// refused rather than written somewhere else: a new ConfigMap document beside the generator would
// collide with it in the build.
func (wb *writeBatch) writeGenerated(
ctx context.Context,
event Event,
generated []*manifestanalyzer.GeneratedObject,
) (upsertOutcome, error) {
live := event.Object
gen := generated[0]
if len(generated) > 1 {
return upsertNoChange, generatedRefusal(live, gen.GeneratedIn, fmt.Errorf(
"%d configMapGenerator entries render an object of this name", len(generated)))
}
if wb.writer.isSensitiveIdentifier(event.Identifier) {
// A generator input is plaintext; a resource the target encrypts must never land in one.
return upsertNoChange, generatedRefusal(live, gen.GeneratedIn, errors.New(
"the resource is sensitive, and a generator input is a plaintext file"))
}

desired := manifestreport.Project(live)
// The same render-fidelity fence a document's edit passes: a ${...} token the input holds that
// the live object carries expanded was substituted downstream (Flux postBuild), and writing the
// expansion back into the input is the corruption no re-render can see, since it then matches.
divergences := manifestanalyzer.RenderTokenDivergences(gen.Rendered.Object, desired.Object)
if len(divergences) > 0 {
id := manifestedit.Identity{Kind: desired.GetKind(), Name: desired.GetName()}
return upsertNoChange, renderFidelityRefusal(
gen.GeneratedIn, id, &renderFidelityRefusedError{Divergences: divergences})
}
writes, err := manifestanalyzer.PlanGeneratedWrite(gen, desired, wb.writeSubdir, wb.currentContent)
if err != nil {
return upsertNoChange, generatedRefusal(live, gen.GeneratedIn, err)
}
for _, w := range writes {
wb.buffer(w.Path).current = w.Content
log.FromContext(ctx).Info("Routed live change to configMapGenerator input",
"kustomization", gen.GeneratedIn, "file", w.Path, "resource", event.Identifier.String())
}

// Declared whether or not this event wrote anything, as for a governed document: an input
// may be shared with another entry an earlier event already wrote, and only the intent tells
// the oracle that this object's move is convergence. Governed, because a generated object no
// render root produces after the write is a contradiction, not a skip.
wb.intend(manifestanalyzer.WriteIntent{
SourcePath: gen.GeneratedIn,
Kind: desired.GetKind(),
Name: desired.GetName(),
Desired: desired,
Governed: true,
})
if len(writes) == 0 {
return upsertNoChange, nil
}
wb.putToKustomize = true
return upsertUpdated, nil
}

// deleteGenerated mirrors a live delete of an object a configMapGenerator renders; see
// removeGenerated. A delete no generator renders is a no-op.
func (wb *writeBatch) deleteGenerated(ctx context.Context, event Event) error {
obj := event.Object
if obj == nil {
id := event.Identifier
if id.Group != "" || id.Resource != "configmaps" {
return nil
}
obj = objectNamed(manifestedit.Identity{
APIVersion: "v1", Kind: "ConfigMap", Namespace: id.Namespace, Name: id.Name,
})
}
_, err := wb.removeGenerated(ctx, event.Identifier, obj, documentDeletedLive)
return err
}

// removeGenerated takes an object a configMapGenerator renders out of the folder: the entry leaves
// configMapGenerator:, and the input files only it reads are deleted, so the next render does not
// bring the object back. The oracle proves the object left the render and nothing else moved. It
// is the one removal the live delete and the resync sweep share, as removeDocument is for a
// document, and it reports whether it removed anything: an object no generator renders, or one
// this batch already removed, is a no-op.
func (wb *writeBatch) removeGenerated(
ctx context.Context,
identifier types.ResourceIdentifier,
obj *unstructured.Unstructured,
outcome string,
) (bool, error) {
generated := wb.store.GeneratedObjectsFor(obj)
switch {
case len(generated) == 0:
return false, nil
case len(generated) > 1:
return false, generatedRefusal(obj, generated[0].GeneratedIn, fmt.Errorf(
"%d configMapGenerator entries render an object of this name", len(generated)))
}
gen := generated[0]
if wb.removedGenerated[gen] {
return false, nil
}
writes, err := manifestanalyzer.PlanGeneratedDelete(gen, wb.writeSubdir, wb.currentContent)
if err != nil {
return false, generatedRefusal(obj, gen.GeneratedIn, err)
}
for _, w := range writes {
buf := wb.buffer(w.Path)
buf.current = w.Content
if w.Path == gen.GeneratedIn {
wb.recordDocumentRemoval(buf)
}
}
if wb.removedGenerated == nil {
wb.removedGenerated = map[*manifestanalyzer.GeneratedObject]bool{}
}
wb.removedGenerated[gen] = true
wb.intend(manifestanalyzer.WriteIntent{
SourcePath: gen.GeneratedIn,
Kind: obj.GetKind(),
Name: obj.GetName(),
Removed: true,
})
wb.putToKustomize = true
wb.tallyDocument(identifier, outcome)
log.FromContext(ctx).Info("Removed configMapGenerator entry for a deleted ConfigMap",
"kustomization", gen.GeneratedIn, "files", len(writes)-1, "resource", identifier.String())
return true, nil
}

// objectNamed is a bodiless object with the given identity, enough to look a generated object up.
func objectNamed(id manifestedit.Identity) *unstructured.Unstructured {
obj := &unstructured.Unstructured{}
obj.SetAPIVersion(id.APIVersion)
obj.SetKind(id.Kind)
obj.SetNamespace(id.Namespace)
obj.SetName(id.Name)
return obj
}

// generatedRefusal reports a generated object the writer cannot write back, as the write-boundary
// refusal every unplaceable edit surfaces as, naming the object and the kustomization.
func generatedRefusal(obj *unstructured.Unstructured, kustomization string, err error) error {
return &manifestanalyzer.AcceptanceRefusedError{
Issues: []manifestanalyzer.AcceptanceIssue{{
Kind: manifestanalyzer.IssueUnplaceableEdit,
Path: kustomization,
// Not solvable from either side: the change itself has nowhere to land.
Solvable: false,
Message: fmt.Sprintf("%s/%s generated in %s: %v",
obj.GetKind(), obj.GetName(), kustomization, err),
}},
}
}

// currentContent is a file's bytes as the batch has left them so far, without hydrating a buffer
// for it.
func (wb *writeBatch) currentContent(rel string) ([]byte, bool) {
if b, ok := wb.buffers[rel]; ok {
return b.current, b.current != nil
}
content, ok := wb.contentByPath[rel]
return content, ok
}
Loading
Loading