Promote the generation preview to Goa v3.31.0

.cursor/skills/goa-release/SKILL.md

@@ -5,8 +5,10 @@ description: Release Goa v3, including preflight tests, dependency updates, semv
 
 # Goa Release
 
-Use this skill only from the Goa repository. `make release` pushes branches and tags for Goa,
-examples, and plugins, so do not run it unless the user explicitly wants to perform the release.
+Use this skill only from the Goa repository. A stable release and an opt-in preview are separate
+workflows. `make release` pushes branches and tags for Goa, examples, and plugins. A preview tags
+Goa alone from a reviewed feature branch. Do not run either publishing command unless the user
+explicitly approves that exact release.
 
 ## Release Contract
 
@@ -17,8 +19,11 @@ examples, and plugins, so do not run it unless the user explicitly wants to perf
 - Apply semver within v3:
   - Patch release: increment `BUILD`.
   - Minor release: increment `MINOR` and reset `BUILD=0`.
+- An opt-in preview uses `v3.MINOR.BUILD-preview.NUMBER`. Its version is a testing label, not a
+  promise of the final stable minor version. The major version remains 3.
 - The preparation commit message must be exactly `Prepare v3.x.y`, where `x` is `MINOR` and `y`
   is `BUILD`.
+- The preview preparation commit message must be exactly `Prepare v3.x.y-preview.n`.
 - Keep the working trees clean between phases. Never discard local changes unless the user
   explicitly approves it.
 - Do not push preparation commits manually. `make release-goa` runs `git pull origin <branch>` in
@@ -44,6 +49,55 @@ Determine the target version before changing any repository:
 6. Use the exact confirmed target version for every preparation commit, tag check, and release
    command.
 
+## Preview Release Contract
+
+Use this workflow only when the confirmed version ends in `-preview.N`.
+
+- Work from the reviewed feature branch. Do not switch to or move the stable `v3` branch.
+- Publish Goa only. Do not update, tag, or push the examples and plugins repositories.
+- Keep the stable README badge and stable `@latest` installation instructions unchanged.
+- Make the preview public only after the upgrade guide describes every known source, design, wire,
+  deployment, and rollback effect in plain English.
+- Treat `prepare-preview` and `release-preview` as separate approval points. The first creates the
+  version commit. The second creates and pushes the public tag.
+- Never run `make release` with `PREVIEW_NUMBER` set. That target is the stable multi-repository
+  release and deliberately rejects preview versions.
+
+### Prepare a preview
+
+1. Confirm the complete version, such as `v3.31.0-preview.1`.
+2. Fetch `origin`, verify the feature branch contains the latest stable branch, and verify the
+   working tree is clean. Preserve and report any existing local changes.
+3. Verify the exact preview tag is absent locally and on `origin`.
+4. Read `UPGRADING.md` against the current diff. Confirm it tells application and plugin authors
+   how to install, regenerate, migrate, deploy, roll back, and report a problem.
+5. Run:
+
+   ```bash
+   make prepare-preview MINOR=31 BUILD=0 PREVIEW_NUMBER=1
+   ```
+
+   The target runs the full preflight, updates the Makefile version and `pkg/version.go`, verifies
+   the command reports the preview version, and creates the exact preparation commit. It does not
+   create or push a tag.
+
+6. Push the feature branch through the normal review workflow and wait for every required check.
+7. Review the complete diff and GitHub release notes one final time.
+
+### Publish a preview
+
+1. Inventory the current branch, commit, local tag, and remote branch and tag.
+2. Get explicit maintainer approval to publish the exact preview tag.
+3. Run `make release-preview` and watch it until it exits. The target reruns the full preflight,
+   verifies the current commit is already on the same remote feature branch, creates the tag, and
+   pushes only the tag.
+4. If the command fails after creating the local tag, inspect local and remote state before any
+   rerun. Never delete a public tag or force-push without an approved recovery plan.
+5. Create a GitHub release for the tag with `--prerelease`. Link `UPGRADING.md`, describe the
+   preview as opt-in, and explain that the final stable minor version follows compatibility review.
+6. Verify the Go proxy resolves the exact preview version and that `@latest` still resolves the
+   current stable release.
+
 ## Required Repositories
 
 Before release, verify these repositories exist, are on the expected branches, are clean, and are
@@ -102,8 +156,10 @@ For each repository:
    If files changed, commit them with `Prepare v3.x.y`. Do not push; `make release-plugins`
    pushes the branch.
 
-5. In the Goa repository, edit only `MINOR` and `BUILD` in `Makefile` for the target version.
-   Leave `MAJOR=3`.
+5. In the Goa repository, edit only `MINOR`, `BUILD`, and the preview marker in `Makefile` for the
+   target version. Leave `MAJOR=3`. For a stable release, `PREVIEW_NUMBER` must be empty. The stable
+   release target clears the suffix in `pkg/version.go`; do not edit that generated version value
+   by hand.
 6. If Goa files changed from dependency updates or the version bump, commit them with
    `Prepare v3.x.y`.
 7. Re-check all three repositories are clean. They may be ahead of upstream by their preparation
@@ -180,4 +236,3 @@ Channel guidance:
 - Substack: Write a short note for readers who may not know Goa deeply. Use a friendly title, a
   brief explanation of Goa, a user-centered story for the release highlights, upgrade guidance if
   any, and contributor thanks. Keep it proportional to the release size.
-

AGENTS.md

@@ -55,10 +55,36 @@ No commented-out code—delete dead code.
 
 ### Codegen Implementation
 
+- **Critical — finish work during generation**: Use the complete design, generation plan,
+  and Go templates to decide everything that is known before the generated
+  program runs. Select branches, names, types, imports, field paths, helper
+  calls, and emitted files while generating source. Templates must write only
+  the selected code. Do not make generated programs inspect generated type
+  shapes, parse generator-made names, carry generator mode flags, or execute
+  branches whose answer was already known. Runtime code should contain only
+  logic that depends on actual runtime values. When a runtime value is truly
+  required, keep that input narrow and specialize all surrounding code during
+  generation.
 - **Use NameScope helpers** for type references: `GoTypeRef`, `GoFullTypeRef`, `GoTypeName`. Never concatenate strings for types.
 - Let Goa decide pointer/value semantics. Do not force `pointer=true` except in transport validation.
 - **Keep helper visibility minimal**: If logic is shared only inside one codegen area, keep it package-private or move it under an `internal` package. Do not export helpers from a parent package just to share them across sibling generators.
 - **Avoid pass-through wrappers**: When two helper functions differ only by forwarding arguments or hard-coding `nil`, collapse them into a single implementation instead of adding an extra layer.
+- **Generated packages own names**: When several services or plugins write to
+  one Go package, collect every package-level name before rendering and then
+  make those names final. A declaration and every HTTP, gRPC, or JSON-RPC use
+  of it must read the same name record. Do not give each service or plugin a
+  separate name scope for the same package, and do not add declarations after
+  names become final.
+- **Keep identity typed and explicit**: Do not hide a declaration's kind,
+  package, or use in a decorated name or a made-up string map key. Do not
+  change an expression's `Hash` behavior to solve a generation problem. Pass a
+  typed identifier where the generated declaration is named.
+- **Trace the complete lifecycle**: Before changing relocated types, union
+  naming, generation roots, plugins, or file merging, follow the declaration
+  from the evaluated design through service analysis, its generated Go
+  package, the emitted service code, HTTP and gRPC uses, plugin changes, and
+  the final file merge. A service-only rendering test is not enough.
+  See [`codegen/ARCHITECTURE.md`](codegen/ARCHITECTURE.md).
 
 ### Documentation
 
@@ -129,4 +155,9 @@ To reproduce a codegen issue:
 
 ### Slices/Maps and Required Fields
 
-Do not rely on nil vs empty to encode presence. Goa uses `omitempty`—both nil and empty serialize as "missing". If empty is valid, do not mark the field as required.
+Do not rely on nil versus empty slices or maps to encode domain meaning.
+Required JSON fields may contain empty collections unless a length constraint
+forbids them; keep `Required` when the JSON property must be present. Protobuf
+repeated and map fields cannot distinguish absent from empty, so their generated
+validation checks length and contents, not presence. Message, scalar, and oneof
+presence checks remain independent of collection emptiness.

Makefile

@@ -8,13 +8,18 @@
 # - "test" runs the tests
 # - "release" creates a new release commit, tags the commit and pushes the tag to GitHub.
 #   "release" also updates the examples and plugins repo and pushes the updates to GitHub.
+# - "prepare-preview" creates the version commit for an opt-in Goa preview.
+# - "release-preview" tags and publishes that preview without changing other repositories.
 #
 # Meta targets:
 # - "all" is the default target, it runs "lint" and "test"
 #
 MAJOR=3
-MINOR=30
+MINOR=31
 BUILD=0
+PREVIEW_NUMBER?=5
+
+PREVIEW_TAG=v$(MAJOR).$(MINOR).$(BUILD)-preview.$(PREVIEW_NUMBER)
 
 GOOS=$(shell go env GOOS)
 GOARCH=$(shell go env GOARCH)
@@ -25,15 +30,18 @@ GOLANGCI_LINT_VERSION?=v2.11.3
 GOLANGCI_LINT=$(GOBIN_DIR)/golangci-lint
 PROTOC_BIN=protoc
 PROTOC_DEST=$(GOBIN_DIR)/$(PROTOC_BIN)
+PROTOC_INCLUDE_DEST=$(GOPATH)/include
 
-.PHONY: all all-tests ci depend lint test test-release integration-test build-goa release release-preflight release-goa release-examples release-plugins
-.NOTPARALLEL: release release-goa release-examples release-plugins
+.PHONY: all all-tests ci depend lint test test-release integration-test build-goa
+.PHONY: release stable-release-check release-preflight release-goa release-examples release-plugins
+.PHONY: preview-version preview-check preview-preflight prepare-preview release-preview
+.NOTPARALLEL: release release-goa release-examples release-plugins prepare-preview release-preview
 
 # Only list test and build dependencies
 # Standard dependencies are installed via go get
 DEPEND=\
-	google.golang.org/protobuf/cmd/protoc-gen-go@latest \
-	google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest 
+	google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.12 \
+	google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
 
 all: lint test integration-test
 
@@ -80,6 +88,8 @@ depend:
 	$(UNZIP) $(PROTOC).zip
 	@rm -f "$(PROTOC_DEST)" && \
 		cp $(PROTOC_EXEC) "$(PROTOC_DEST)" && \
+		mkdir -p "$(PROTOC_INCLUDE_DEST)" && \
+		cp -R "$(PROTOC)/include/." "$(PROTOC_INCLUDE_DEST)" && \
 		chmod 0755 "$(PROTOC_DEST)" && \
 		rm -rf $(PROTOC) && \
 		"$(PROTOC_DEST)" --version
@@ -114,40 +124,132 @@ build-goa:
 
 release-preflight: lint test-release integration-test
 
-release: release-goa release-examples release-plugins
+stable-release-check:
+	@if [ "$(MAJOR)" != "3" ]; then \
+		echo "error: Goa releases must use major version 3"; \
+		exit 1; \
+	fi
+	@if [ -n "$(PREVIEW_NUMBER)" ]; then \
+		echo "error: PREVIEW_NUMBER is set; use the preview release targets"; \
+		exit 1; \
+	fi
+
+release: stable-release-check release-goa release-examples release-plugins
 	@echo "Release v$(MAJOR).$(MINOR).$(BUILD) complete"
 
+preview-version:
+	@if [ "$(MAJOR)" != "3" ]; then \
+		echo "error: Goa preview releases must use major version 3"; \
+		exit 1; \
+	fi
+	@case "$(MINOR)" in \
+		""|*[!0-9]*|0[0-9]*) \
+			echo "error: MINOR must be a non-negative integer without a leading zero"; \
+			exit 1; \
+			;; \
+	esac
+	@case "$(BUILD)" in \
+		""|*[!0-9]*|0[0-9]*) \
+			echo "error: BUILD must be a non-negative integer without a leading zero"; \
+			exit 1; \
+			;; \
+	esac
+	@case "$(PREVIEW_NUMBER)" in \
+		""|*[!0-9]*) \
+			echo "error: PREVIEW_NUMBER must be a positive integer"; \
+			exit 1; \
+			;; \
+		0|0*) \
+			echo "error: PREVIEW_NUMBER must not be zero or start with zero"; \
+			exit 1; \
+			;; \
+	esac
+	@echo "$(PREVIEW_TAG)"
+
+preview-check: preview-version
+	@status="$$(git status --porcelain)"; \
+	if [ -n "$$status" ]; then \
+		echo "error: goa repo has uncommitted changes:"; \
+		echo "$$status"; \
+		exit 1; \
+	fi
+	@if git rev-parse --verify --quiet "refs/tags/$(PREVIEW_TAG)" >/dev/null; then \
+		echo "error: local tag $(PREVIEW_TAG) already exists"; \
+		exit 1; \
+	fi
+	@if git ls-remote --exit-code --tags origin "refs/tags/$(PREVIEW_TAG)" >/dev/null 2>&1; then \
+		echo "error: remote tag $(PREVIEW_TAG) already exists"; \
+		exit 1; \
+	fi
+
+preview-preflight: preview-check release-preflight
+
+prepare-preview: preview-preflight
+	sed 's/^MINOR=.*/MINOR=$(MINOR)/' Makefile > _tmp && mv _tmp Makefile
+	sed 's/^BUILD=.*/BUILD=$(BUILD)/' Makefile > _tmp && mv _tmp Makefile
+	sed 's/^PREVIEW_NUMBER?=.*/PREVIEW_NUMBER?=$(PREVIEW_NUMBER)/' Makefile > _tmp && mv _tmp Makefile
+	sed 's/Minor = .*/Minor = $(MINOR)/' pkg/version.go > _tmp && mv _tmp pkg/version.go
+	sed 's/Build = .*/Build = $(BUILD)/' pkg/version.go > _tmp && mv _tmp pkg/version.go
+	sed 's/Suffix = .*/Suffix = "preview.$(PREVIEW_NUMBER)"/' pkg/version.go > _tmp && mv _tmp pkg/version.go
+	@test "$$(go run ./cmd/goa version)" = "Goa version $(PREVIEW_TAG)"
+	git add Makefile pkg/version.go
+	git commit -m "Prepare $(PREVIEW_TAG)"
+
+release-preview: preview-preflight
+	@expected="Goa version $(PREVIEW_TAG)"; \
+	actual="$$(go run ./cmd/goa version)"; \
+	if [ "$$actual" != "$$expected" ]; then \
+		echo "error: generated version is $$actual, expected $$expected"; \
+		exit 1; \
+	fi
+	@branch="$$(git branch --show-current)"; \
+	if [ -z "$$branch" ]; then \
+		echo "error: preview release must run from a branch"; \
+		exit 1; \
+	fi; \
+	remote_head="$$(git ls-remote origin "refs/heads/$$branch" | awk '{print $$1}')"; \
+	local_head="$$(git rev-parse HEAD)"; \
+	if [ "$$remote_head" != "$$local_head" ]; then \
+		echo "error: push branch $$branch and wait for its checks before publishing $(PREVIEW_TAG)"; \
+		exit 1; \
+	fi
+	git tag $(PREVIEW_TAG)
+	git push origin $(PREVIEW_TAG)
+	@echo "Preview $(PREVIEW_TAG) published; examples and plugins were not changed"
+
 release-goa:
 	# First make sure all is clean
+	@test "$$(git branch --show-current)" = "v$(MAJOR)"
 	@status="$$(git status --porcelain)"; \
 	if [ -n "$$status" ]; then \
 		echo "error: goa repo has uncommitted changes:"; \
 		echo "$$status"; \
 		exit 1; \
 	fi
 	cd $(GOPATH)/src/goa.design/examples && \
-		git checkout main && \
-		git pull origin main && \
+		test "$$(git branch --show-current)" = main && \
 		status="$$(git status --porcelain)" && \
 		if [ -n "$$status" ]; then \
 			echo "error: examples repo has uncommitted changes:"; \
 			echo "$$status"; \
 			exit 1; \
-		fi
+		fi && \
+		git pull --ff-only origin main
 	cd $(GOPATH)/src/goa.design/plugins && \
-		git checkout v$(MAJOR) && \
-		git pull origin v$(MAJOR) && \
+		test "$$(git branch --show-current)" = "v$(MAJOR)" && \
 		status="$$(git status --porcelain)" && \
 		if [ -n "$$status" ]; then \
 			echo "error: plugins repo has uncommitted changes:"; \
 			echo "$$status"; \
 			exit 1; \
-		fi
+		fi && \
+		git pull --ff-only origin v$(MAJOR)
 	go mod tidy 
 	# Bump version number, commit and push
 	sed 's/Major = .*/Major = $(MAJOR)/' pkg/version.go > _tmp && mv _tmp pkg/version.go
 	sed 's/Minor = .*/Minor = $(MINOR)/' pkg/version.go > _tmp && mv _tmp pkg/version.go
 	sed 's/Build = .*/Build = $(BUILD)/' pkg/version.go > _tmp && mv _tmp pkg/version.go
+	sed 's/Suffix = .*/Suffix = ""/' pkg/version.go > _tmp && mv _tmp pkg/version.go
 	sed 's/goa\/v3@v.*tab=doc/goa\/v3@v$(MAJOR).$(MINOR).$(BUILD)\/dsl?tab=doc/' README.md > _tmp && mv _tmp README.md
 	$(MAKE) release-preflight
 	git add .
@@ -170,11 +272,14 @@ release-examples:
 
 release-plugins:
 	cd $(GOPATH)/src/goa.design/plugins && \
-		sed 's/goa.design\/goa\/v.*/goa.design\/goa\/v$(MAJOR) v$(MAJOR).$(MINOR).$(BUILD)/' go.mod > _tmp && mv _tmp go.mod && \
+		for module in $$(find . -name go.mod -not -path './.git/*'); do \
+			(cd "$$(dirname "$$module")" && \
+				go mod edit -require=goa.design/goa/v$(MAJOR)@v$(MAJOR).$(MINOR).$(BUILD) && \
+				go mod tidy) || exit 1; \
+		done && \
 		make && \
 		git add . && \
 		git commit -m "Release v$(MAJOR).$(MINOR).$(BUILD)" && \
 		git tag v$(MAJOR).$(MINOR).$(BUILD) && \
 		git push origin v$(MAJOR) && \
 		git push origin v$(MAJOR).$(MINOR).$(BUILD)
-

README.md

@@ -182,6 +182,17 @@ curl -X POST http://localhost:8000/jsonrpc \
   -d '{"jsonrpc":"2.0","method":"hello.say_hello","params":{"name":"world"},"id":"1"}'
 ```
 
+## Upgrading to v3.31.0
+
+v3.31.0 is a substantial generator upgrade that stays on the v3 module path
+and includes intentional breaking changes. Regenerate the complete generated
+tree and review changes to handwritten code and plugins. Some gRPC, JSON-RPC,
+and streaming contracts require coordinated client/server deployment.
+
+Read the [v3.31.0 upgrade guide](UPGRADING.md) before updating. It covers the
+exact installation commands, source and design migrations, deployment order,
+and rollback.
+
 ## Documentation
 
 Our documentation site at [goa.design](https://goa.design) provides comprehensive guides and references: