Rewrite functions and tests in Python

.github/workflows/ci.yaml

@@ -11,6 +11,11 @@ on:
         required: false
 
 env:
+  # Build functions one at a time. Every Python function build mounts the same
+  # up-python-sdk-pip-cache Docker volume, and on a fresh runner - where that volume starts
+  # empty - concurrent builds race creating its directories and fail with
+  # "mkdir ...: file exists".
+  UP_MAX_CONCURRENCY: "1"
   UP_API_TOKEN: ${{ secrets.UP_API_TOKEN }}
   UP_ROBOT_ID: ${{ secrets.UP_ROBOT_ID }}
   UP_ORG: ${{ secrets.UP_ORG }}

.github/workflows/composition-tests.yaml

@@ -6,6 +6,13 @@ on:
       - main
   pull_request: {}
 
+env:
+  # Build functions one at a time. Every Python function build mounts the same
+  # up-python-sdk-pip-cache Docker volume, and on a fresh runner - where that volume starts
+  # empty - concurrent builds race creating its directories and fail with
+  # "mkdir ...: file exists".
+  UP_MAX_CONCURRENCY: "1"
+
 jobs:
   composition-tests:
     runs-on: ubuntu-latest

.github/workflows/ruff.yaml

@@ -0,0 +1,14 @@
+name: ruff
+on: [pull_request]
+jobs:
+  ruff:
+    name: runner / ruff
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
+      # Pinned: ruff's default rule set changes between releases, and ruff.toml only
+      # adjusts it. Bump deliberately, and fix what the new version reports in the same change.
+      - name: Install ruff
+        run: python3 -m pip install --quiet ruff==0.16.9
+      - name: ruff check
+        run: ruff check --output-format=github functions tests common

.gitignore

@@ -2,3 +2,5 @@ _output
 .venv
 .up
 .agents
+.vscode
+__pycache__

README.md

@@ -505,12 +505,44 @@ delete the repository or its published packages.
 
 ## Development
 
+The composition functions and the tests are Python, on the
+[function SDK](https://github.com/crossplane/function-sdk-python). Each function is a
+`FunctionRunner` in `functions/<name>/function/fn.py`; each test is a module under
+`tests/<name>/test/` that prints its `CompositionTest` (or `E2ETest`) as YAML.
+
 ```bash
-up project build
+up project build                   # also generates the Python models under .up/python
 up test run "tests/test-*"         # composition tests
 up test run "tests/*" --e2e        # end-to-end, against a real control plane
 ```
 
+Functions and tests run in containers, so none of this needs Python on your machine. An
+editor does: without the generated models and the SDK on its interpreter path, every
+`from models.io...` import shows as unresolved on correct code. Build a venv once, after the
+first `up project build`, from the project's own pins:
+
+```bash
+python3.13 -m venv .venv && .venv/bin/pip install --upgrade pip
+# The functions' pins cover the tests too (SDK, pydantic, PyYAML). The `cd` matters: pip
+# resolves each pyproject's relative path to .up/python from the current directory.
+for d in functions/*; do (cd "$d" && ../../.venv/bin/pip install -q -e .); done
+.venv/bin/pip install -e .up/python    # last, and editable, so regenerated models need no reinstall
+```
+
+Code more than one function needs lives in `common/` at the project root, not in any one
+function. A function is packaged from its own directory alone, so each carries a
+`function/common` symlink to it, and `up` copies the symlink's target into the built function.
+Import it as `from .common.naming import truncate_iam_name`. On Windows, clone with
+`git config core.symlinks true` (and Developer Mode or admin rights), or the symlinks check
+out as plain text files.
+
+> Function directory names are the published package paths
+> (`xpkg.upbound.io/<org>/platform-ref-upbound_<name>`) — renaming one publishes a new package.
+
+> CI builds functions one at a time (`UP_MAX_CONCURRENCY=1`). Every Python function build
+> mounts the same pip-cache Docker volume, and on a fresh runner concurrent builds race creating
+> its directories.
+
 > The composition glob is `tests/test-*`, not `tests/*`. `up test run` generates manifests for
 > every directory it matches, even ones it will not execute, and `tests/e2etest-environment`
 > deliberately fails generation when its variables are unset — better than provisioning a

common/__init__.py

@@ -0,0 +1,9 @@
+"""Code shared by this project's composition functions.
+
+Each function is built and packaged on its own, from its own directory, so a function cannot
+import a sibling. This package is shared by symlink instead: every function carries a
+`function/common` symlink pointing here, and `up` follows symlinks when it packages a
+function's source, so each built function gets its own copy of this directory.
+
+Keep it free of imports from any one function, and of anything a function would not want.
+"""

common/dicts.py

@@ -0,0 +1,10 @@
+"""Reading untyped request data."""
+
+
+def dig(d, *path):
+    """Walk nested dicts, returning None at the first missing level."""
+    for key in path:
+        if not isinstance(d, dict):
+            return None
+        d = d.get(key)
+    return d

common/kcl_parity.py

@@ -0,0 +1,11 @@
+"""Output the KCL implementation produced without the functions asking for it.
+
+These functions replaced KCL ones and keep their rendered output identical. KCL's typed
+models materialised every schema default into their output, so a few provider defaults
+appear in the desired state although no function set them. They change nothing on a cluster;
+they are kept so the rendered desired state - what the composition tests assert - is unchanged.
+"""
+
+# provider-kubernetes Object defaults, emitted on every Object.
+OBJECT_FOR_PROVIDER_DEFAULTS = {"deletionPropagationPolicy": "Background"}
+OBJECT_SPEC_DEFAULTS = {"watch": False}

common/naming.py

@@ -0,0 +1,24 @@
+"""AWS IAM resource names that fit IAM's length limit."""
+
+IAM_NAME_MAX = 64
+
+
+def simple_hash(s: str) -> str:
+    """Position-weighted character sum, truncated to 8 digits.
+
+    Not a cryptographic hash, and it does not need to be: it only has to be stable, because
+    its output becomes part of an AWS resource name. It must stay identical to the KCL
+    original it replaced - a different value renames, and so replaces, the IAM resource.
+    """
+    return str(abs(len(s) * 31 + sum(ord(c) * (i + 1) for i, c in enumerate(s))))[:8]
+
+
+def truncate_iam_name(name: str, suffix: str) -> str:
+    """Fit an IAM name into 64 characters, keeping the suffix and hashing the prefix."""
+    if len(name) <= IAM_NAME_MAX:
+        return name
+    base = name[: len(name) - len(suffix)]
+    prefix_space = IAM_NAME_MAX - len(suffix) - 8 - 1
+    if prefix_space <= 0:
+        return f"{simple_hash(base)}{suffix}"
+    return f"{base[:prefix_space].rstrip('-')}-{simple_hash(base)}{suffix}"

common/policy.py

@@ -0,0 +1,10 @@
+"""managementPolicies for the XR-level deletionPolicy parameter."""
+
+# Orphan on delete. Namespaced (.m.) managed resources have no deletionPolicy; leaving
+# "Delete" out of managementPolicies is the only way to keep the external resource.
+ORPHAN = ["Create", "Observe", "Update", "LateInitialize"]
+
+
+def management_policies(deletion_policy: str) -> list[str]:
+    """Translate the XR's Delete/Orphan parameter into managementPolicies."""
+    return ["*"] if deletion_policy == "Delete" else ORPHAN

functions/environments/README.md

@@ -0,0 +1,11 @@
+# environments
+
+Composition function for `Environment` (`sa.upbound.io/v1`): an Upbound group holding one
+control plane, wired to AWS.
+
+- `function/fn.py` — initialisation from the bootstrap kubeconfig, then which resources apply
+- `function/resources/` — one builder module per area: `kubernetes`, `argo`, `team_robot`,
+  `secret_sync`, `aws`
+- `function/common` — symlink to the project's shared `common/` package
+
+Tests: `tests/test-environment*`. See the project README for how to build and run them.